Smart Search on Android will be able to find info in PDFs and media files

Google is currently testing a new Smart Search feature that aims to enhance the process of finding information within PDF files and images. It is integrated in the Files app on Android, which is a file manager. By utilizing local machine learning, this option improves the searchability of files. The Smart Search option is able to scan the contents of files on the device automatically, resulting in more accurate search results.

This feature goes beyond traditional search capabilities, as it can locate text within images and PDFs, identify locations and objects in photos, and even recognize artist, album, and song titles from audio and video recordings.

In fact, this functionality has the potential to make the File Explorer on the device more efficient compared to various file browsers used on desktop operating systems.

For instance, you can input a query based on IKEA's slogan "design and quality" and retrieve relevant results.

Optical character recognition (OCR) is consistently reliable when it comes to searching within PDFs and images that have been stored on the device for some time. Keywords can be used to refine searches, which simplifies the process of finding older files, especially when it comes to photographs and screenshots containing text.

The search feature also allows for finding specific objects within images. To illustrate, if you want to locate images with cats, you can perform a search using the term "cats". However, it is worth noting that Smart Search feature may not be as effective as Google Photos in terms of recognizing friends and family members by their names if the user has saved them in the service. This may be changed in the future.

Google explains that Smart Search operates by scanning files in the background, prioritizing device performance and privacy. This functionality functions autonomously, ensuring that user data remains on the device and is not transmitted elsewhere. Should the need arise, the smart search option can be disabled through the application settings.

Source