You Can Try Google Lens’ Powerful AI Search Tool Right Now

Multimodal search is nothing new for Google Lens. The company introduced it back in 2022: Since then, you’ve been able to search images and text using Lens—for example, looking up a photo of a specific car, then adding “red” to specify the type of color you want to find it in.

Now Google is pushing an update to Lens that uses the company’s new AI tools to make the experience a bit more insightful. Google uses the following thought experiment to demonstrate Lens’ new abilities: Say you’re browsing a yard sale, and find an interesting board game with no box or instructions to identify it. Rather than google aimlessly about what you can see, Google suggests taking a photo of the game, then asking Lens questions like “What is this game and how is it played?” Similar to how visual search with something like ChatGPT works, Lens will then search the web based on your photo for relevant information about the game.

I imagine there are a lot of potential use cases for this tool. You could take a photo of a dish at a restaurant and ask Lens to tell you how to make it; you could take a photo of a building in a city you’re visiting and ask Lens for its history and significance; you could even snap a picture of a trading card and ask which set it’s from, and whether it’s rare or valuable. This type of tool could be used for education, nutrition, or general shopping—assuming, of course, that it works as its advertised.

How to try out Google Lens’ new AI abilities

This new Lens feature is out now: Google rolled it out Wednesday, Jan. 17. If you know how to use Lens, you already on your way. Launch the Google app on Android or iOS, then tap the Lens icon in the search bar. You can either use the search/shutter button to take a photo in Lens itself, or use the button to the left to upload an image from your phone instead. Either way, once Lens analyzes your photo, you can tap “Add to your search,” and include your expanded questions.

I just tried it out on an orchid I have next to my desk. I took a photo of it in Lens, then asked “is this toxic to dogs.” Google thought for a minute, then returned an answer with generative AI. (Long story short: No, orchids aren’t poisonous to dogs or cats.) But it’s cool that you don’t even need to specific what’s in the photo when asking about it. I don’t need to ask “is this orchid toxic?” I can just say “is this toxic,” as if I was asking someone in the room with me. That’s pretty cool. (Now, hopefully the answer is useful, correct, and on-topic, but that’s another issue.)

Image: Google