By most accounts, Apple's new smartphone, the "iPhone 8", is due out soon -- in September 2017. But I feel confident in saying the company's most groundbreaking release for the year actually happened earlier this winter, at Apple's Worldwide Developer Conference (WWDC) in San Jose, California.
At that event, Apple unveiled "ARKit", a new, free set of software tools that anyone can use to build impressive new apps and experiences on the iPhone and iPad. The "AR" in "ARKit" refers to "augmented reality", a technology that places interactive virtual objects and effects into your view of the real world, blending the virtual world with physical reality.
Today, this is usually accomplished by apps that use a smartphone's (or other device's) camera to show you a view of the real world in front of you, then layering virtual objects on top of it. For example, some of the most popularly used AR features right now are the "lenses" or face filters in Snapchat -- the effects that put animal ears on your head or warp your face in weird ways. Another one is Pokemon GO, the gaming app that made the pocket monsters look as though they were bouncing around your desk or the street. What these apps have in common is that the virtual objects inside of them appear to be inhabiting or changing real-world scenes -- "augmenting" our reality.
Thanks to Apple's restrictions, you can't get a perfect video player that handles all formats and performs with the same efficiency as Apple's built-in Videos app. Nevertheless, PlayerXtreme HD does just about everything you could ask for within the confines of iOS's walled garden.
But those apps were custom-built by pro software developers. What makes Apple's ARKit version of augmented reality so exciting is that it isn't an app -- instead, it's a free way for anybody to build new AR apps that will be able to run on many of Apple's iOS devices from the last few years. And the proof is in the pudding: Apple's ARKit has already only been publicly available for about two months, but already, independent software developers have created some incredible-looking experiences for the iPhone -- everything from virtual portals you can enter to putting virtual zombies in your living room. Even Lord of the Rings director Peter Jackson has gotten in on the fun with an ARKit short film.
I've been interested in augmented reality for years, so when Apple announced ARKit, I was thrilled. If you feel the same or are just interested in experimenting with it yourself, here's how you can get started.
Check to make sure you have a compatible iOS device
ARKit runs only Apple iOS devices containing an A9 processor (or newer). That limits it to the following devices, according to RedmondPie:
- iPhone 6s and 6s Plus
- iPhone 7 and 7 Plus
- iPhone SE
- iPad Pro (9.7, 10.5 or 12.9)
- iPad (2017)
Presumably, the new iPhone 8 lineup will also support ARKit. You'll also want to make sure the camera on your iOS device is working, as ARKit relies upon it. And you need a cable for connecting your device to your Mac computer.
Download iOS 11 on your iPhone or iPad
Once you have one of these compatible iOS devices handy, you'll need to download iOS 11 on it. That's the latest version of Apple's mobile operating system, and it's only available as a beta right now, which means it potentially contains bugs and could mess up some things on your device. If this is a major concern, buy or use another device that isn't your primary one. Or wait for the official release of iOS 11, which is also expected in September 2017.
If you don't want to wait, you can download the iOS 11 beta and get started. To do this, you'll need to enroll in Apple's Beta Software Program. This is free, but requires anywhere between 20 minutes to a couple hours to set up. Head over here and follow the instructions.
Check to make sure you have a compatible Mac computer
You'll also need a computer for coding your ARKit app. Specifically, a Mac with the free operating system mac OS Sierra 10.12.4 (or newer version) installed. That means the following machines are eligible:
- MacBook (Late 2009 or newer)
- MacBook Pro (Mid 2010 or newer)
- MacBook Air (Late 2010 or newer)
- Mac mini (Mid 2010 or newer)
- iMac (Late 2009 or newer)
- Mac Pro (Mid 2010 or newer)
Later this year, Apple will publicly release iOS 11, which includes a one-handed mode for the default keyboard. That's a great idea, which is why so many third-party keyboards already have the feature. And you don't have to jailbreak or do anything weird.
Register for an Apple developer account
You can use a free or paid developer account to get started with ARKit. If you already have a regular consumer Apple account that you use to buy apps and such on your iPhone/iPad/Mac, you can upgrade this to a free or paid developer account. Using a web browser on your Mac computer, navigate to Apple's developer account sign-in page and choose which option is right for you.
Download Xcode 9 (which contains ARKit) for your Mac for free from Apple's developer website
On your Mac, sign into your Apple developer account, then head here and click "Download" in the top right corner to get Xcode 9, Apple's program for programming (a tautology, yes!). Warning: You'll need at least 4.9 GB free of hard drive space to download this, and then Xcode itself takes up 9.76 GB of hard drive space.
Launch Xcode 9 and connect your iOS device to your computer with a cable
On your Mac, unzip your Xcode 9 download by double clicking it, then drag the Xcode app into your Applications folder and double click it to launch it. Plug in the iOS device you have with iOS 11 installed.
Create a new AR app project template in Xcode
In Xcode on your Mac, click "File", "New", and select "Project". This should pull up a grid of options. Click on "Augmented Reality App".
Fill out your new AR app's information in Xcode
Apple requires you fill out the following information for your new AR app. You can name it anything you'd like, and be sure to also select a "Team", even if you're an individual (if you don't have one yet, you'll get the option to create one). Keep everything else checked.
In the "language" option, can also choose between Swift, Apple's new programming language, and Objective-C, its older one. Both will allow you to build an AR app. If you're new to programming, Swift might be the better bet as it's what Apple plans to use for all of its tools going forward.
Finally, you can choose your "Content technology", selecting between SceneKit, SpriteKit and Metal. These are different rendering engines for displaying graphics. All three will let you build AR apps, but offer different trade-offs, some of which are described here (basically: SceneKit for 3D graphics, SpriteKit for 2D, Metal for more complicated scenes). For purposes of this demo, I selected "SceneKit".
You'll also need to put your project in a folder on your computer. Select anywhere you want, this doesn't matter much.
Select your connected iOS device to receive your new AR app
In Xcode on your Mac, select "Window" from the top menu bar, click "Devices and Simulators", and double click your iOS device. Make sure you check the box that says "Show as run destination".
You may get a pop up on your iOS device now saying "Untrusted Developer" and giving you some instructions to go to your iOS device's "Settings" page to allow this app.
If this happens, go ahead and find your "Settings" app on your iOS device, tap it, scroll down to "General" and tap it, then scroll further to "Profiles and Device Management", and you should be able to find your Apple developer email address. Tap this and it should allow you to install your ARapp.
Run your first demo AR app
Apple includes a very simple AR demo with Xcode for you to get a sense of what you can build with ARKit. This demo displays a fighter jet in the air in front of your device's camera.
In order to see it in action, first make sure your iOS device is selected in the device picker dropdown menu in the top left hand corner of Xcode.
Then, look up at the top menu bar in Xcode. Find "Product", click it, and then click "Run".
The top status indicator in Xcode should begin flickering with activity and messages, then it should launch your app on your iOS device!
To stop your demo app, go back to "Product" in Xcode and scroll down to "Stop".
That's it, you're well on your way to using ARKit! For more information on adding your own new 3D models and creating animated experiences, I'd recommend the following guides: