How Apple Silicon Powers Everything At Apple

Apple's World Wide Developers Conference (WWDC) recently showcased the company's expertise in creating harmony between their custom-built ARM processors, known as Apple Silicon, and their software platforms. From the first iPad and iPhone 4 in 2010 to the latest Mac Pro launched at WWDC. Most of its products have relied on this fundamental framework.

Its newest product, Apple Vision Pro, stands as a testament to its impressive in-house chip-making capabilities. The device features the M2 chipset from the Mac and iPad but also the R1 processor.

Many tech companies, following the success of Apple's vertically integrated approach, have tried to emulate the harmony between hardware and software. Google with the Pixel phones and Tensor chipset, Microsoft with Surface and its SQ chip, and Samsung are all noteworthy mentions. However, none have been able to master the blend as Apple has, thanks to over a decade of bespoke chip-making.

PA Semi Is The Basis For Apple Silicon 

This unique strategy emerged when Apple's founder, Steve Jobs, acquired Palo Alto Semiconductor (PA Semi) in 2008 following Intel's decision not to develop a processor for the iPhone. Apple then began to design ARM processors with Samsung, but by the time the first iPad came out in 2010, they had created their purpose-built chip, the A4 which was built by the team at PA Semi which was now part of Apple. 

PA Semi brought some prodigious chip design talent to Apple. People like Jim Kellar and Raja Koduri, both CPU and GPU design veterans led design teams at AMD and then Intel post their stints at Apple. But at Apple, the two helped design the A4 chip which would be a stepping stone for them. 

This chip also powered the iPhone 4, the first great camera Apple made on a product line which has largely displaced the point-and-shoot camera. All iPhones point towards the computational capabilities of Apple's silicon that enable the iPhone to be the world’s most popular camera. It also was responsible for the first “retina display” on an Apple product. 

In 2013, Apple developed the world’s first 64-bit ARM application processor, the A7 chipset. This processor outperformed both Qualcomm and ARM, the creator of the architecture. This improvement allowed app developers to create more powerful apps, rivalling those on the PC and Mac.

Apple's in-house chip design has influenced ARM's architectural advancements and instruction set. It's worth mentioning that Qualcomm's processors continue to lag behind the iPhone's performance by at least one generation.

The first Apple Watch was revealed in 2014 which used the S2 processor, marking Apple's entry into wearable technology. The operating system was a spin-off of the iPhone's technology, which was itself derived from macOS at its core. This approach allowed Apple to balance sufficient computing power with adequate battery life.

The same strategy was adopted for AirPods in 2016. The custom H1 chip, and its subsequent iterations in the W1 and W2, enable features like quick pairing, seamless switching between Apple devices, computational audio, spatial audio and advanced noise-cancellation techniques.

Apple even used the A-series processors in the HomePod for superior audio performance. Similarly, the Apple TV, the exceptional 4k experience and potential gaming capabilities are powered by these chipsets.

In 2018, Apple signalled a shift in macOS to ARM when it announced support for iPad apps on the Mac. This change was further reinforced with the prodigious A12X processor in the iPad Pro, boasting Xbox-rivalling graphics. Intel's production issues prompted the transition of the MacBook Air and MacBook Pro from Intel’s chips to Apple's M-series chipsets in 2020.

Apple Vision Pro, their latest product, features the M2 chip, which debuted with the MacBook Air last year and still powers devices like the new Mac Mini and iPad Pro. Working alongside the M2, the R1 chip processes information from a litany of cameras and sensors, driving the 4K resolution of the OLED screens.

NeXT Was Before Apple Silicon 

Two pivotal acquisitions have defined Apple's product range – NeXT in 1997, which was founded by Jobs after his initial departure from Apple, and PA Semi in 2008. NeXT's technologies have been foundational for the software platforms that Apple has developed. Its technology underlines everything from macOS, iOS, iPadOS, watchOS, tvOS and now even visionOS. 

NeXT's technology ran on PowerPC processors. Apple transitioned macOS to Intel in 2005, but again, it adapted the software for the iPhone's ARM architecture in 2007. In 2020, it again adapted macOS to run on ARM from Intel. This experience with adapting software platforms to different computing architectures has allowed Apple to launch new product categories whilst retaining compatibility with older apps. Intel Macs supported software from PowerPC Macs, iPads supported iPhone apps, and even recently, macOS supports and runs software written for Intel Macs. So it was appropriate visionOS supports iPhone apps which is an advantage in its favour. No mixed reality headset supports so many apps. 

Apple has woven a vertically integrated fabric of hardware and software like no other consumer-facing company. While this framework was challenging to build, it now operates at such a large scale that it is a force multiplier. 

Every new product based on these core technologies makes its ecosystem more sticky to consumers and developers. And it also enables a Lego-block-like stack of silicon-to-software amenities which is a sublime starting point further bolstered by the purpose-built invention for a particular product. 

It is not hard to see where the bits for Apple Vision Pro have come from. It incorporates bits of the Mac, iPhone, iPad and AirPods and builds further on top of them. The M2 chip is in the MacBook Air, but the A-series chips from the iPhone were the basis for it. 

The true-depth camera system first showed up on the iPhone in 2017. The LiDAR debuted on the 2020 iPad Pro and the iPhone 12 Pro. Spatial audio has been a bedrock of Apple’s audio strategy with AirPods and HomePod and is now all across its products. 

Invention For The Sake Of Features 

Some insane invention has also happened. Apple had to deliver a high-fidelity experience considering its brand and the type of screen technology it has pioneered in the last decade. It has been developing microLED screens in Taiwan for a while which are starting to surface in its products. The iPhone is expected to use this tech in the coming years as well. 

The micro LED screens on both eyes each pack 23 million pixels for a 4K plus resolution with 3D with real-time eye-tracking that is faster than human perception. There was an added challenge of doing real-time pass-throughs as there one layer of glass is projecting information from the cameras to the other layer. Users don’t see through the glass. One is seeing footage from the cameras in real-time without parallax at 4K resolution. Similarly, people around the user can see the face as it is projected from the cameras inside the headset facing the user. 

This involves fusing tons of data from the assortment of cameras and sensors. It is an impossibly difficult task for a device that is to run on a battery. All this is parsed in real-time by a new chip called the R1, which Apple created for this headset. There is AI trickery happening all over the place and Apple’s excellence in doing private on-device machine learning comes through here. 

Apple, in recent years, has acquired many companies in the VR and AR space. It has made acquisitions even in the semiconductor space when it added Intel’s modem business in 2019. But everything is purposeful. Apple acquired either talent or technologies that it can turn into features. 

So there is an iteration of existing bespoke technologies which are state of the art and operating at a ginormous scale. Then there is invention for the sake of delivering a specific experience and cost is not a factor in its development. Apple throws the kitchen sink and only releases things when they are perfected. 

The grapevine was that many outside and inside Apple weren’t convinced about Apple Vision Pro. Many reports claimed it would be the most experimental Apple product in decades. From what I can see right now is that it isn’t. It feels more developed than the initial iPhone. It feels more advanced than the first Apple Watch. Concerns around it perhaps are more related to the nature of the product category. In that sense, it is similar to the iPad. It certainly sounds like the “magical experience” Steve Jobs often touted. 

Although the Mac and the iPhone have been the epicentres of Apple's universe, the Apple Vision Pro may well displace the iPhone as the primary device one day. However, the mobility of the experience will be a determining factor. Even if Apple can't solve this problem, Apple Vision Pro is sure to be as influential as the iPad or Mac.

Also Read: The Future Is Here, The World Is Ready To Move Beyond The Smartphone