All the New AI Features Coming to Google Pixel 8

There was plenty to discuss during the Made by Google event, but the company’s main announcements were the Pixel 8 and Pixel 8 Pro. While these new flagship smartphones sport new displays, camera, and other new hardware improvements, it’s the new Tensor G3 chip that might prove most impressive. With it, Google is introducing a host of new AI features you can take advantage of on Pixel 8 and Pixel 8 Pro.

Best Take

Best Take is probably going to be the talked-about feature of the new Pixel phones, for better or for worse. When snapping multiple group photos at once, the feature lets you choose the best face from each person in the photo to include in the final option. Seriously.

It’s a bizarre feature that shows how far tech has come. That group shot you post on Instagram isn’t reality: It’s mixed and matched to make sure no one is blinking, sneezing, or looking upset. Sure, it could make bad group shots a thing of the past, but it makes me feel, well, a little uneasy.

Magic Editor

You likely know Magic Eraser, the iconic Google feature that quickly removes unwanted elements from photos. Now, the company is rolling out Magic Editor, a new option that uses generative AI to improve an image however you want. You can relocate a subject from one area to another, adjust the background with preset options, and fill in missing information with AI-generated imagery.

Magic Eraser improvements (Pixel 8 Pro)

Magic Eraser on Pixel 8 Pro can now remove larger elements, and generates new pixels to fill in blank areas.

Audio Magic Eraser

Audio Magic Eraser analyzes the audio in a video clip to identify the different sounds within and sort them into layers. You can then control these layers, to reduce the levels of annoying sounds so you can hear what you want to hear in the video. For example, if you record a musician outside, you can use Audio Magic Eraser to reduce sounds like wind, people talking, and traffic.

Call Screen enhancements

Call Screen is one of the Pixel’s best features, which uses an AI voice assistant to literally screen your calls for you. Going forward, Call Screen will be able to differentiate between a spam call (say, a recording telling you that you’ve won an all-expenses-paid trip), and a legitimate call (for example, a representative from your airline calling about verifications to your upcoming trip). In the latter example, the AI voice assistant will tell the caller they’ll try to get you on the line, and will then ping you with a summary of the situation and a series of options you can take, including picking up the phone, reporting the call as spam, or taking a message.

The feature can also take care of contextual calls, such as a doctor’s office calling to confirm an appointment. The assistant will ping you with a simple summary, and you can hit “Confirm” to have it confirm on your behalf. Later this year, these updates will also work on Pixel Watch and Pixel Watch 2 when paired with a Tensor-enabled Pixel phone.

Live translations

When writing out messages using dictation, Pixel 8 and Pixel 8 Pro can translate your words in real time. If you want to share a message in Spanish, for example, speak it in English, and your phone will convert it for you in time to hit “send.”

Read aloud

This new feature lets you long-press on the power button to have your Pixel start reading a web page out loud to you.

Gboard

Gboard now uses AI to fix typos, grammar mistakes, and other errors that might occur as you type. You can simply tap “More fixes” to see what the AI thinks you missed.

If you pick up a Pixel 8 Pro, you’ll get higher quality reply suggestions in Gboard as well. That particular feature is coming in December.

Summarize

As a writer, I want you to read my articles start to finish. But if you don’t have time, Pixel 8 will summarize them for you. Whenever you open a long article, you’ll find a new “Summarize” button, which uses AI to break down the important points into an easy-to-read summary.

Video Boost

Video Boost is another feature coming later this year, but one that should prove interesting. It processes each frame of your video as you shoot it, adjusting things like colour, lighting, stabilization, and graininess. It also uses Night Sight Video to make dark scenes more clear. We won’t know how good this feature is until it drops, but it sounds promising.

Recorder

Following a Pixel Drop in December, Recorder will support on-device summarization on Pixel 8 Pro.

Zoom Enhance

People have been calling out the “enhance” trope in movies and TV for years. But Google wants to make it a reality: A new Zoom Enhance feature intelligently enhances digital zoom photos all on-device, using AI to sharpen blurry images from digital zoom. This one’s coming in a December update, too.

Assistant With Bard

Assistant With Bard is exactly what is sounds like: Google combined Google Assistant with its AI chatbot Bard to make a more powerful digital assistant deeply integrated with the Google ecosystem. With it, you should be able to ask the assistant complicated requests, and it should respond in kind. You can ask it to run through important emails you’ve missed this week, and it’ll dive into your Gmail to pull up a summary of your missed highlights.

One of the examples on stage involved asking Assistant With Bard to make a grocery list for a weekend trip for 10 people, with a ton of snacks and some smoothie recipes. The assistant came back with a detailed list of groceries as well as smoothie recipes to make with them.

Like ChatGPT, Assistant With Bard recognizes images, too. You can take a photo of something in the real world, and the assistant will be able to answer questions about it.

This feature comes out soon to initial testers, and will be an opt-in feature in coming months to Pixel 8 users.