Apple Releases OpenELM, an Open-Source On-Device AI Model

Apple is expected to deploy LLMs across its hardware in 2024, something Tim Cook has promised

Apple has recently launched OpenELM, a new family of large language models (LLMs) designed to run on-device, eliminating the need for a cloud connection to respond to user queries. This release comes after months of speculation about Apple's efforts to develop its own foundational AI models for generative AI, following the success of ChatGPT and its integration into Microsoft's Bing search engine.

The launch of OpenELM is seen as a response to Apple being caught off guard by the rapid advancements in generative AI, with competitors like Microsoft and OpenAI showcasing the capabilities of ChatGPT 3.5 in 2023. Since then, Apple has invested billions of dollars and acquired numerous AI startups to bolster its own AI capabilities.

OpenELM consists of eight models, four pretrained and four instruction-tuned, with parameter sizes ranging from 270 million to 3 billion. While these models are smaller than some of their competitors, such as Microsoft's Phi-3mini at 3.8 billion parameters, they are specifically designed to efficiently execute text generation tasks on-device. Pre-training allows the LLM to produce coherent and helpful text, while instruction tuning enables the LLM to provide more relevant outputs to specific user requests.

Apple has released the code for OpenELM on Hugging Face, along with various training checkpoints, performance statistics, and instructions for pre-training, evaluation, instruction tuning, and parameter-efficient fine-tuning. The models are offered under a sample code license that allows for commercial usage and modification, provided that redistributions of the software include the original notice, text, and disclaimers.

This is not Apple's first foray into open-source AI models, having previously released Ferret, a multimodal large language model, in October. It also published a paper about ReLAM which is an innovative approach enables the AI to understand context in a conversation and process onscreen content, converting it into a format that can be processed by large language models. Interestingly its 80 million ReALM model rivalled the performance of OpenAI's GPT-4.

OpenELM mirrors Microsoft's recent launch of Phi-3mini, another model designed to run entirely on smartphones. The release of OpenELM is particularly surprising given Apple's reputation for secrecy and proprietary technology.

The development of OpenELM was led by Sachin Mehta, with lead contributions from Mohammad Rastegari and Peter Zatloukal. The models were pre-trained on a dataset of 1.8 trillion tokens sourced from various online platforms, including Reddit, Wikipedia, and arXiv.org. Benchmarks were run on both a high-performance workstation and a MacBook Pro with an M2 Max chipset, demonstrating the models' ability to perform well on a range of hardware.

OpenELM's performance has proven to be respectable, closely trailing Microsoft's Phi-3Mini in benchmarks, although Apple has only provided results for the 450 million parameter variant. The models utilise layer-wise scaling to assign parameters within each layer of the transformer model, enhancing accuracy while maintaining computational efficiency.

The release of OpenELM aligns with rumours of Apple's plans to introduce LLM capabilities to iPhones and iPads through iOS 18 updates at the upcoming Worldwide Developers Conference (WWDC). Additionally, Apple is said to be working on AJAX, an internal search tool that could be deployed alongside iOS 18, while also exploring licensing foundational models from Google and OpenAI for cloud-based features.

As the race to develop powerful, efficient, and user-friendly AI models intensifies, Apple's release of OpenELM demonstrates the company's commitment to staying at the forefront of the rapidly evolving AI landscape. With on-device processing capabilities and potential collaborations with other tech giants, Apple is positioning itself to deliver cutting-edge AI experiences to its users in the near future.