Your Pixel 8 Pro is getting on-device AI features with December’s Pixel Feature Drop

Google is out with its quarterly Pixel Feature Drop, giving Pixel owners reasons to think their phones are new again. The spotlight for December, though, is a new on-device generative AI model called Gemini Nano that’s exclusive to the Pixel 8 Pro (for now). Here’s everything you need to know about the drop.

What is Gemini from Google?

To know what Gemini Nano is meant to do, we’ve got to explain what Gemini is first. In short, it’s Google’s newest full-fledged generative AI model that is intended to go beyond what large language models (LLMs) — the main target here is OpenAI’s ChatGPT — can do by processing not just text and code, but audio, images, and video, too.

Gemini would remove an interpretation layer between the AI model and other tools picking up text in those audiovisual contexts through methods such as optical character recognition. With the backing of its processing power, Gemini can speed up its own analysis and manipulation of the material with which it is working.

The Gemini Nano model is the most compact of the three versions on offer. Unlike larger iterations that are designed with scale and centralized servers in mind, Nano is intended to be installed onto consumer devices to help with local tasks.

The Pixel 8 Pro with Gemini Nano

That’s where we get to this Pixel Drop’s big exclusive for the Pixel 8 Pro running with its Tensor G3 processor. Some of the new features in this drop will be powered by the Gemini Nano model with all the work done on-device.

There are two big Gemini features that Google is promoting and one of them is Summarize in Recorder. Generating a transcript and labeling speakers within your audio recordings isn’t new, but what is new is that you’ll be also be able to generate a summary of the transcript (much like what Bard can do on Google Search and elsewhere) and perform all of these features offline through Gemini Nano.

Similarly, Smart Replies are already well-established across Google services, but the company says the replies Gemini Nano generates are richer and more pertinent. You can try Gemini-powered Smart Replies through a developer preview using Gboard in US English on WhatsApp. You’ll need to follow the Smart Compose tool instructions to activate and use the feature.

Other features in the drop

Yes, Google has other features that other Pixel phones will also get — it’s not like the people there have forgotten. Here’s a rundown:

• Video Boost in Google Photos works on your Pixel 8 Pro’s clips to improve stability, tune up color, turn down noise, and even bring light to the dark with Night Sight.
• A new Night Sight in Timelapse mode is available in the camera app for the Pixel 8 and 8 Pro.
• Taking a picture with the Pixel Fold? Dual Screen Preview will let your subject know what you’re seeing.
• Yet another AI boost is on the list, this one for the Pixel Tablet with Clear Calling deadening background noise and crisping up your voice.
• Pixel Tablet users can also take advantage of Spatial Audio if their earphones support it.
• All devices from the Pixel 6 onward (including the Pixel Fold and Pixel Tablet) can turn on AI-improved Portrait lighting and better Photo Unblur in Google Photos.
• Pixel 6 and later phones (sorry, Tablet owners) can become webcams when attached to a laptop or desktop that supports USB Video Class.
• Call Screen for Pixel 6 and later phones can now suggest contextual replies so that Google Assistant can answer more calls you’ll never have to pick up. It’s also available for the first time on both generations of the Pixel Watch if you take calls there.

Google

• On the Pixel 5a with 5G and later, the new Clean feature in Scan helps you remove coffee stains and smudges on receipts and important documents.
• Google Password Manager can now identify which of your accounts can support passkeys and help you add them quickly on the Pixel 5a with 5G and later.
• The Pixel Watch joins the Extend Unlock (formerly Smart Lock, details from Android Police) scheme, letting you unlock your Pixel phone (5a with 5G or later) or Pixel Tablet by unlocking your watch.
• First-gen Pixel Watch users have been begging for Do Not Disturb and Bedtime Mode status synchronization with their Pixel phones, and they’re finally getting it.
• Direct My Call and Hold for Me expands to the UK and can be used for numbers that don’t have a toll-free prefix.
• The Recorder app gains support for transcription in 28 more languages.
• Send in your device for repair? Set it to Repair Mode so that you can make sure your device is secure while in transit, in your technician’s hands, and then on the way back to you.

Will Gemini Nano make it to other phones?

In a way, yes. Google has created a new system-level service in Android 14 that it calls Android AICore (via The Verge) where it houses Gemini Nano along with other engines as well as safety checks. The company is also laying out workflows to let app developers interface with AICore (and, thus, Gemini Nano) on users’ devices.

That service is backed by the AI-related silicon of your device’s chipset, so in theory, you should be able to use Gemini Nano on an Android 14 device with sufficient AI processing power (think of flagship-level chipsets like Qualcomm’s Snapdragon 8 Gen 3 or MediaTek’s Dimensity 9300).

Still, the notion of a pint-sized generative AI model becoming part of a mobile operating system is nothing to sneeze at. We’ll just have to see how Google refines its approach to deployment as the days wear on.