Apple's ReALM Model Outperforms GPT-4 Despite Smaller Size

Apple has been diligently developing an in-house large language model to compete in the rapidly evolving generative AI space. The Cupertino-based tech giant was caught off guard when Microsoft updated its Bing search engine with technology infused with OpenAI's ChatGPT, allowing the Seattle-based company to surpass Apple as the world's most valuable tech company. Despite facing challenges in iPhone sales in China due to increasing competition, Apple is now poised to respond with its latest AI advancements.

In a recently published paper, Apple researchers have detailed a new technique called Reference Resolution As Language Modelling (ReALM). This innovative approach enables the AI to understand context in a conversation and process onscreen content, converting it into a format that can be processed by large language models. ReALM is offered in four sizes: ReALM 80M, ReALM 250M, ReALM 1B, and ReALM 3B, with M and B denoting millions or billions of parameters in the models.

Remarkably, when benchmarked against ChatGPT 3.5 and GPT-4, Apple's smallest model, ReALM 80M, demonstrated performance comparable to GPT-4, OpenAI's most advanced model.

"We demonstrate large improvements over an existing system with similar functionality across different types of references, with our smallest model obtaining absolute gains of over 5% for onscreen references. We also benchmark against GPT-3.5 and GPT-4, with our smallest model achieving performance comparable to that of GPT-4, and our larger models substantially outperforming it," Apple's researchers stated.

What makes this achievement even more impressive is that Apple's models have significantly fewer parameters compared to their counterparts. GPT-4 boasts 1.75 trillion parameters, while ReALM 80M has only 80 million parameters. OpenAI's co-founder and CEO, Sam Altman, acknowledged in a podcast with Lex Friedman that the current state of GPT-4 "kind of sucks right now" and that the company plans to release a materially better model in the future. Reports suggest that OpenAI is preparing to launch GPT-5 later this year.

Apple has been aggressively acquiring GenAI startups and investing billions of dollars to catch up in the AI race. The company recently absorbed Canadian GenAI startup Darwin AI and has transferred employees from its ill-fated Project Titan, the so-called Apple Car project, to work on GenAI. The company has also held talks with OpenAI and Google to license GPT and Gemini foundational models for iOS 18, which could be showcased at WWDC 2024 on June 10.

iOS 18 is expected to feature numerous LLM-based generative AI capabilities. Apple reportedly plans to deploy its own models for on-device processing, touting that its operating system works in sync with its custom-designed silicon, which has been optimised for these AI features while preserving user privacy. For more advanced processing, Apple is in talks with Google to license Gemini as an extension of its deal to have Google Search as the default search engine on the iPhone operating system.