Microsoft’s aim with the Xbox One was to use as much Windows technology as possible, and when Windows Threshold appears, it should be theoretically possible to run a single piece of code on both platforms. Yet despite that Windows dependency and the need to run multiple apps in parallel, the Xbox One doesn’t actually use Microsoft’s Hyper-V virtualisation technology. This is why.
Microsoft partner dev lead Frank Savage gave a fascinating presentation on the development process for the operating system for the Xbox One during the Build 2014 conference in San Francisco last week. Here, in his own words from that presentation, is how virtualisation works on the Xbox One, and why it runs multiple custom versions of Windows 8.
“Games on consoles have to be predictable. Nothing is worse when you are playing a console game than having the game hitch. When it hitches, you want to throw the controller at the TV. When it hitches and you miss a shot at somebody, you want to throw the controller at the TV as fast as you possibly can. So we want to avoid that if we can. The only way to avoid that is to make it predictable.
“You get six CPU cores inside the console, four on one side of the bus and two on the other side of the bus. Those six CPU cores have to respond the same way every time. If they don’t and it runs differently every time you run it, that’s bad. And it means that there’s a lot of things Windows can’t do in the background all the time, because if it does, it will screw up that predictability. You can’t have that, and drove some operating system decisions about how we were going to do things and what made sense and what didn’t.
“That predictability is critically important. When these guys say it has to run at 30 frames per second at 1080p, or they want to run at 60 frames per second at 1080p, they’re not kidding — they really need 60 the whole time. And 16.67 milliseconds is not a lot of time to get stuff done.
“We live in this world where we need an operating system that can do two wildly different things when you think about it. The first thing it has to be able to do is be predictable and powerful, so that we can get these AAA games to run with as much of the console as you can give them, but I also need a world where I can switch away from that game really quickly to something like a Netflix or a Hulu or some other app. Having that other piece where I can switch away to means that that other piece is very unpredictable.
“I don’t know what apps are going to be running there five years from now, two years from now, six months from now. There’s no way to know. I know what we’re going to have at launch but I don’t know what comes after that. So there has to be some way to know this one can be anything it needs to be while the other one is extremely predictable and doesn’t take anything away from the game at all.
Three Operating Systems In One Console
“This turned out to be a really, really hard problem to solve with a single operating system, so in typical Microsoft fashion we built three. The first operating system that we have — the one that boots the console, the one that basically actually owns where everything and how everything works — is the host operating system (HostOS).
“The HostOS owns all the resources, it owns all the memory, it owns all the CPU and GPU, it controls all the controllers, it traffic directs all the network traffic that’s coming in or out of the box, and its job is to be the security layer as well as to control all these interfaces.
“So the HostOS is completely opaque to you as a developer. You never see it, it just works — it works really well actually which is awesome — but this is the piece where we put all of the eggs that control security and control resources management to the box.
“Part of what it is doing is it’s also hosting what were two Hyper-V virtual machines. So the shared partition or shared OS or SRA side of the Xbox One contained an operating system that is visually and in fact code indistinguishable from Windows 8. So it’s running on the shared partition right now is Windows 8. It just works, but it has a lot of other jobs to do besides just running apps. It can run apps and it runs them very effectively but it’s also in charge of the Xbox Shell.
“System Services is all the system services that games require to be able to run. The networking stack lives here, some of the audio stuff lives here, a lot of the Kinect processing happens here, and this particular shared partition when a game is running gets two CPU cores all to itself and 3GB of RAM, depending on the mode that it’s in.
“The low-level operating system is here as well, which is basically the Windows 8 and kernel pieces. So this thing is the first thing that spins up after the HostOS. Your Xbox finishes booting, you see the shell come up. The shell really isn’t anything more than an app. It can run other apps and it comes up and now you are sitting in the Xbox One.
“The exclusive partition hasn’t come up yet. The exclusive partition fires up when you play a game. You say ‘I want to play Forza Motorsport’, it says ‘fine’. It spins up that partition, and gives it all the resource it needs and Forza takes off. Everything that Forza needs below that either goes straight through to the shared partition if it’s networking or things like that.
“But there are other things that need to run way faster than that on the exclusive side in order to get the performance that we need. Those things could be like the video driver. When I’m issuing DirectX 11 commands to draw cars or to draw trees or to figure out what perimeter target I need to use, all of these commands essentially go straight through the exclusive OS to the HostOS through a series of well-known channels.
Not Your Mother’s Hyper-V
“This is why it’s not a Hyper-V virtual machine anymore and we call it a shared partition instead. These things would have run any operating system in the world that was X86 or X64 compatible. They don’t anymore. They run the shared OS and they run the exclusive OS and they don’t run anything else. They’re hand-tuned and they’re hand-coded and they run as fast as is humanly possible.
“To give you some kind of idea, when we first spun up the exclusive partition when we first started doing it, we had a bunch of games tests we would run. Some were between 5 and 10 frames per second, most of it spent sending commands back and forth to the CPU. Over the period running up to launch, we were able to get that so tight and so small that those games run over 200 or 300 frames per second today. The CPU utilisation is so small that it can run very, very quickly through those channels and do exactly what it needs to do.
“This is critically important for these guys because they need it to be powerful and they need it to be predictable. So the exclusive partition also is running Windows 8, but it’s a Windows 8 that has gone on a massive, massive diet, and in fact diet is probably the wrong word. We put it on a diet for a while but then we got out the knives and the hatchets and the chainsaws and learned a lot about how Windows is interconnected with itself that may surprise you and probably I shouldn’t talk about, but let’s just say that there were cases that didn’t make as much sense as you might have thought.
“But the good news is we were able to get rid of all of that and we have an extremely lean and mean operating system in there now that’s very tight and very small. We have all the Win32 APIs in there for games coming from Windows 7 or games coming from other more PC-type targets, but we also have the WinRT pieces so if we have a game from the phone or store that you want to put in the exclusive partition, it’s relatively easy to port that too. All of the test games that I built for this run in both Windows 8 and Xbox One even today. It has been trivially easy to do that. ”
Disclosure: Angus Kidman travelled to San Francisco as a guest of Microsoft.