Yesterday, a blog post from developer Jeff Johnson highlighted an ‘Enhanced Visual Search’ toggle added to the Apple Photos app in iOS 18. This feature gives your phone permission to send photo data to Apple in exchange for being able to identify landmarks in your photos or search for them in your picture library. What’s more, this setting is turned on by default, and until now, we weren’t aware of it. It has largely gone unmentioned by Apple or in coverage of the new iPhone OS.
What is Enhanced Visual Search?
The feature is similar to Apple’s ‘Visual Look Up,’ which uses machine learning to identify plants or animals in your photo library. To see it in action, swipe up on a picture in your library showing a building or landmark and tap ‘Look Up Landmark.’ Photos will then ideally tell you what it is.
Here are some examples from my own library:
- That’s definitely Austin’s Cathedral of Saint Mary, but the image on the right is not a Trappist monastery, but the Dubuque, Iowa city hall building.
How Does Enhanced Visual Search Work?
A description under the Enhanced Visual Search toggle says the feature will ‘privately match places in your photos,’ though it’s not clear from the description how that happens or that it means sending data from your photos to Apple. According to a company research blog, it involves your phone creating vector embeddings — of the part of a picture containing a landmark, and sending that and several ‘fake queries’ to Apple for analysis. Your phone then chooses the final match from a batch of possibilities that Apple sends back.
A Questionable Commitment to Privacy
But that’s not necessarily enough, according to Johnson, referencing Apple’s ‘What happens on your iPhone, stays on your iPhone’ billboard from CES 2019:
…if something happens entirely on my computer, then it’s private, whereas if my computer sends data to the manufacturer of the computer, then it’s not private, or at least not entirely private … You don’t even have to hypothesize lies, conspiracies, or malicious intentions on the part of Apple to be suspicious of their privacy claims. A software bug would be sufficient to make users vulnerable, and Apple can’t guarantee that their software includes no bugs.
Apple’s Response
Apple did not respond to our request for comment on Johnson’s concerns.
Finding Enhanced Visual Search
You can find the Enhanced Visual Search option on iOS / iPadOS under Settings > Apps > Photos, or under the ‘Search’ heading in the Photos > Settings on a Mac.
Correction and Update
An earlier version of this story misstated the location of the Enhanced Visual Search toggle. It is in iOS Settings under Apps > Phone and in Photos > Settings in the macOS Photos app. The headline was also made clearer.
This article has been updated for clarity.
Implications of Enhanced Visual Search
The implications of Enhanced Visual Search are far-reaching. By sending photo data to Apple, users may be compromising their privacy. While Apple’s description of the feature suggests that it will ‘privately match places in your photos,’ it’s not clear how this happens or what data is being sent.
Moreover, as Johnson pointed out, even if Apple has no malicious intentions, a software bug could still compromise user data. And with Enhanced Visual Search turned on by default, users may not be aware of the potential risks involved.
Conclusion
The introduction of Enhanced Visual Search in iOS 18 raises important questions about Apple’s commitment to privacy. While the feature may seem convenient, it also poses significant risks to users’ personal data. As we continue to rely more and more on our devices, it’s essential that we’re aware of the potential implications of these features.
By understanding how Enhanced Visual Search works and what data is being sent to Apple, users can make informed decisions about their privacy. It’s time for Apple to provide greater transparency around this feature and its impact on user data.
What Do You Think?
Do you think Enhanced Visual Search is a useful feature or a compromise of user privacy? Share your thoughts in the comments below.
