Apple is sponsoring the International Conference on Machine Learning (ICML) 2024, which is taking place in person from July 21 to 27 in the Messe Wien Exhibition and Congress Center, Vienna Austria. ICML is globally renowned for presenting and publishing cutting-edge research on all aspects of machine learning used in closely related areas like artificial intelligence, statistics and data science, as well as important application areas such as machine vision, computational biology, speech recognition, and robotics. Below is the schedule of our sponsored workshops and events at ICML 2024.
Schedule
Stop by the Apple booth in Halle/Hall B, Booth #110, from 11:30 am – 6:45 pm CEST July 22; 10:00 am – 6:00 pm CEST on July 23 and 24.
Sunday, July 21
Monday, July 22
WORKSHOP & SOCIAL
Queer in AI
9:00 am – 4:00 pm CEST, Stolz 2
Isha Garg and Adam Golinski will be representing Apple.
WORKSHOP
LatinX in AI
9:00 am – 4:00 pm CEST, Stolz 1
Pau Rodriguez Lopez and Eduardo Martinez Montes will be representing Apple.
SOCIAL
LatinX in AI
5:30pm – 7:30pm CEST, The View Cafe-Bar Restaurant
Pau Rodriguez Lopez and Eduardo Martinez Montes will be representing Apple.
Tuesday, July 23
Wednesday, July 24
WORKSHOP
Women in Machine Learning (WiML)
9:00 am – 4:00 pm CEST, Schubert 1-6
Catherine Vilhauer, Pau Rodriguez Lopez, and Tillie Hands will be representing Apple.
POSTER
How Smooth Is Attention?
11:30 am – 1:00 pm CEST, Halle/Hall C 4-9
Valérie Castin (ENS), Pierre Ablin, Gabriel Peyré (ENS)
POSTER
Data-free Bootstrapping Distillation
1:30 pm – 3:00 pm CEST, Halle/Hall C 4-9
Jiatao Gu, Shuangfei Zhai, Yizhe Zhang, Lingjie Liu (University of Pennsylvania), Josh Susskind
Thursday, July 25
POSTER
Executable Code Actions Elicit Better LLM Agents
1:30 pm – 3:00 pm CEST, Halle/Hall C 4-9
Xingyao Wang (University of Illinois Urbana-Champaign), Yangyi Chen (University of Illinois Urbana-Champaign), Lifan Yuan (University of Illinois Urbana-Champaign), Yizhe Zhang, Hao Peng (University of Illinois Urbana-Champaign), Ji Heng (University of Illinois Urbana-Champaign)
Friday, July 26
WORKSHOP
Workshop on Foundation Models in the Wild 2024
9:00 am CEST, Straus 1
Aligning Text-to-Image Diffusion as GFlowNets
Dinghuai Zhang (University of Montreal), Yizhe Zhang, Jiatao Gu, Ruixiang Zhang, Josh Susskind, Navdeep Jaitly, Shuangfei Zhai
Projected Language Models: A Large Model Pre-Segmented Into Smaller Ones
David Grangier, Angelos Katharopoulos, Pierre Ablin, Awni Hannun
WORKSHOP
ES-FoMo II: 2nd Workshop on Efficient Systems for Foundation Models
9:00 am CEST, Lehar 2
OpenELM: An Efficient Language Model Family with Open Training and Inference Framework
Sachin Mehta, Mohammad Sekhavat, Qingqing Cao, Max Horton, Yanzi Jin, Frank Sun, Iman Mirzadeh, Mahyar Najibi, Dmitry Belenko, Peter Zatloukal (Work done while at Apple), Mohammad Rastegari (Meta (Work done while at Apple))
Accepted Papers
Contrasting Multiple Representations with the Multi-Marginal Matching Gap
Zoe Piran, Michal Klein, James Thornton, Marco Cuturi Cameto
Data-free Bootstrapping Distillation
Jiatao Gu, Shuangfei Zhai, Yizhe Zhang, Lingjie Liu (University of Pennsylvania), Josh Susskind
On a Practical Implementation of Brenier’s Polar Factorization Theorem and Its Applications to Optimization and Sampling
Marco Cuturi Cameto, Nina Vesseron (ENSAE)
On the Minimal Degree Bias in Generalization on the Unseen for non-Boolean Functions
Denys Pushkin (EPFL), Raphael Berthier (EPFL), Emmanuel Abbe (Apple/EPFL)
Scalable Pre-training of Large Autoregressive Image Models
Alaaeldin Mohamed Elnouby Ali, Michal Klein, Shuangfei Zhai, Miguel Angel Bautista Martin, Josh Susskind, Armand Joulin (Google Deepmind (Work done while at Apple))
Revealing the Utilized Rank of Subspaces of Learning in Neural Networks
Isha Garg, Eshan Verma, Daniel Ulbricht, Christian Koguchi
Superposition Prompting: Improving and Accelerating Retrieval-Augmented Generation
Thomas Merth, Qichen Fu, Mohammad Rastegari (Meta (Work done while at Apple)), Mahyar Najibi
Aligning Text-to-Image Diffusion as GFlowNets
Dinghuai Zhang (University of Montreal), Yizhe Zhang, Jiatao Gu, Ruixiang Zhang, Josh Susskind, Navdeep Jaitly, Shuangfei Zhai
Careful with that Scalpel: Improving Gradient Surgery with an EMA
Pierre Ablin, James Thornton, Eugene Ndiaye, Yu-Guan Hsieh, Michal Klein, Marco Cuturi Cameto
Executable Code Actions Elicit Better LLM Agents
Xingyao Wang (University of Illinois Urbana-Champaign), Yangyi Chen (University of Illinois Urbana-Champaign), Lifan Yuan (University of Illinois Urbana-Champaign), Yizhe Zhang, Hao Peng (University of Illinois Urbana-Champaign), Ji Heng (University of Illinois Urbana-Champaign)
How Smooth Is Attention?
Valérie Castin (ENS), Pierre Ablin, Gabriel Peyré (ENS)
Improved Modelling of Federated Datasets using Mixtures-of-Dirichlet-Multinomials
Jonny Scott (Institute of Science and Technology Austria), Aine Cahill
Knowledge Transfer from Vision Foundation Models for Efficient Training of Small Task-specific Models
Raviteja Vemulapalli, Hadi Pour Ansari, Fartash Faghri, Sachin Mehta, Mehrdad Farajtabar, Mohammad Rastegari (Meta (Work done while at Apple)), Oncel Tuzel
KV-Runahead: Scalable Causal LLM Inference by Parallel Key-Value Cache Generation
Minsik Cho, Mohammad Rastegari (Meta (Work done while at Apple)), Devang Naik
Optimization without Retraction on the Random Generalized Stiefel Manifold
Simon Vary (UC Louvain), Pierre Ablin, Bin Gao (Chinese Academy of Sciences), Pierre-Antoine Absil (UC Louvain)
Private Vector Mean Estimation in the Shuffle Model: Optimal Rates Require Many Messages
Hilal Asi, Vitaly Feldman, Jelani Nelson (University of California Berkeley), Kunal Talwar, Huy Nguyen (Northeastern University), Samson Zhou (Texas A&M University)
Projected Language Models: A Large Model Pre-Segmented Into Smaller Ones
David Grangier, Angelos Katharopoulos, Pierre Ablin, Awni Hannun
Swallowing the Bitter Pill: Simplified Scalable Conformer Generation
Yuyang Wang, Ahmed Elhag (University of Oxford), Navdeep Jaitly, Josh Susskind, Miguel Angel Bautista Martin
Whispering Experts: Neural Interventions for Toxicity Mitigation in Language Models
Xavier Suau Cuadros, Pieter Delobelle (KU Leuven), Rin Metcalf Susa, Armand Joulin (Google Deepmind (Work done while at Apple)), Nick Apostoloff, Luca Zapella, Pau Rodriguez Lopez
OpenELM: An Efficient Language Model Family with Open Training and Inference Framework
Sachin Mehta, Mohammad Sekhavat, Qingqing Cao, Max Horton, Yanzi Jin, Frank Sun, Iman Mirzadeh, Mahyar Najibi, Dmitry Belenko, Peter Zatloukal (Work done while at Apple), Mohammad Rastegari (Meta (Work done while at Apple))
Demos
MLX
We are demonstrating large model inference and training on device using MLX. MLX is a flexible array framework that is optimized for Apple silicon, and brought to you by Apple Machine Learning Research. It enables training and inference of arbitrarily complex models on Apple silicon powered devices with great brevity and flexibility.
In this demo we showcase fine-tuning of a 7B parameter LLM on an iPhone, image generation using a large diffusion model on an iPad, and text generation using a number of large language models on a M2 Ultra Mac Studio and M3 Macbook Pro.
Private Federated Learning (PFL)
This demo showcases Apple’s Private Federated Learning (PFL) technology. PFL-research is the open-source framework enabling this technology for research simulations, open sourced in March 2024 at the Apple PPML Workshop. Here we are able to show how Siri can play Music and Podcasts on iPhones, which leverages several technologies such as Siri Signals and Siri Inference, Private Federated Learning (PFL) and Differential Privacy (DP). This highlights a user-facing feature in iOS that was shipped, thanks to this framework.
Acknowledgements
Devon Hjelm, Ozan Sener, Pau Rodriguez Lopez, and Tatiana Likhomanenko are Area Chairs for ICML 2024.
Aadirupa Saha is a Co-Organizer for the Models of Human Feedback for AI Alignment workshop.
Rin Metcalf Susa is a Panelist on the Models of Human Feedback for AI Alignment workshop.
Marco Cuturi, Samy Bengio, and Vladlen Kotlun are Senior Meta Reviewers for ICML 2024.
Arno Blaas, Bailin Wang, Fartash Faghri, Gustaf Ahdritz, Hilal Asi, Junpei Zhou, Luca Zappella, Miguel Angel Bautista Martin, Miguel Sarabia del Castillo, Parikshit Gopalan, Qichen Fu, Ray Zhang, Richard Bai, Rin Metcalf Susa, Vimal Thilak, Xavier Suau Cuadros, Yu-Guan Hsieh, Yuyang Wang, and Zoe Piran are reviewers for ICML 2024.
Natalie Schluter is a Senior Workshop Chair for ICML 2024.