A Picture Can Be Worth A Thousand Bucks: A Brief Guide on Visual Search

By: LeVi Pham

What is Visual Search?

Have you ever been in a situation where you can visualize what you are looking for but are unable to describe it in words? If so, visual search is for you! The human brain is extremely visual - it can identify images within 13 milliseconds, and 90% of all information received by the brain are visual. Companies including Pinterest and Google are training machines to do the same. As Pinterest’s CEO Ben Silbermann says, “The future of search will be all about pictures, not keywords.”

Visual search consists of every search that uses real-world images, like photos and screenshots, as a starting point. It is powered by computer vision and machine learning - devices can see through their camera lens, learn to understand what they are seeing, and know how to answer queries with that information. You can point your camera at a sign and use visual search to translate it in real time. You can take or upload a photo of a cute puppy and identify the breed. You can take a photo of the restaurant’s menu and immediately pull up reviews and recommendations from other customers. Developers have only begun to explore the many possibilities of visual search.

Visual Search and E-Commerce

Though currently in its infancy, visual search’s popularity is on the rise. Intent Lab's 2018 study shows that 59% of consumers think visual information is more important than textual information across all consumer categories. Around 36% of respondents have conducted a visual search, while close to 60% think images are more trustworthy than text. A study from ViSenze found that 62% of Generation Z and Millennial consumers want visual search capabilities more than any other new technology. Visual search allows consumers to quickly find the items that they are inspired by and seek to purchase, due to its ability to answer queries like “I want to buy this but I don’t know what it is called,” and “Show me things similar to this.” Younger generations prefer shopping on social media platforms and Forbes deemed the rise of ‘social commerce’ will be inevitable. Major companies have begun to integrate and optimize their content for visual search, including major retailers such as Wayfair and Forever 21. Gartner predicts that by 2021, early adopter brands that have redesigned their websites to support visual and voice search will increase digital commerce revenue by 30%.

The Most Popular Visual Search Tools

  1. Pinterest


  2. Pinterest was the first to bring visual search into the limelight with their Lens app. Pinterest’s platform lets users collect images as “pins” in boards and helps them get inspired by looking at other people’s boards. In 2017, Pinterest launched Shop the Look, which was the first visual shopping tool of its kind that utilized the new Pinterest Lens technology. Through the camera, the tool identifies specific objects in a larger scene with white dots that connect the user to the checkout page, which allows them to shop directly from the app. A year after the release of Lens, users did more than 600 million visual searches every month, a 140% increase year over year. Pinterest developed this tool further with the launch of Complete the Look in June 2019. This tool can recommend visually compatible results in Fashion and Home Decor Pins by taking context such as an outfit, body type, season, furniture etc. With this new technology, brands will potentially gain more exposure on Pinterest as their Pins appear via visual search.

  3. Google Lens


  4. Google Lens was launched in the same year as Pinterest Lens and is actively pushed by Google to feature in its family of apps, including the Photos app, Google Search app, and Google Assistant. It can be used to scan and translate text, search for similar products on e-commerce sites, find restaurant reviews from menu photos, explore nearby places, and identify plants and animals. Google claims that the tool can recognize over a billion items. If you want to add integrated computer vision into your software, you can do so at Google’s Cloud Vision. You can use the two available products, AutoML Vision and Vision API, to train your platforms to recognize and categorize images.

  5. Bing Visual Search


  6. Bing Visual Search is very similar to Google Lens. It leverages on the Bing platform to deliver search results based on a photo. This is available both as a mobile app and as well as a dedicated web platform. The unique feature of Bing Visual Search is the ability for developers to set specific skills to a matched image, which can be configured through the Bing Visual Search Development Platform. The visual search can identify user’s intent and then requests different skills based on that intent. For example, a visual search for a chair can yield a “buy” skill, but also a “DIY” skill.

  7. Amazon


  8. In June 2019, Amazon revealed its own visual shopping tool called StyleSnap. Similar to Pinterest’s Shop The Look, StyleSnap allows users to use images to search for clothing that is sold on Amazon. The tool also considers factors like brand, price, and reviews to recommend similar items. Yoast notes that Amazon is also working on combining visual and voice search for their own smart assistant devices, such as Echo Show and Echo Spot. Developers can build their own integration on Amazon Web Services.


How to Optimize Your Content for Visual Search

As visual search becomes more popular, it is more important than ever to optimize your images for SEO. You can learn how to do so from our blog post: 6 Ways to Boost Your SEO Score with Image Optimization. To summarize:
  • Use relevant and original images
  • Use the highest quality format available
  • Reduce the size of your images
  • Include a concise caption with your image
  • Utilize “alt text” or “alt attribute” to maximize accessibility


In addition, it is useful to consider the following:
  • Build image search into your web inventory: use tools such as Google’s Cloud Vision or Bing Visual Search Development Platform to feature their visual search technology on your site.
  • Advertise on Pinterest: their huge market impact and well-developed visual search abilities can enhance the visibility of your products, especially for fashion or home decor, by featuring it alongside similar items.
  • Submit images to your sitemap - add additional information about important images that exist on each URL listed in your sitemap.


The Final Verdict

Visual search is among the Three V’s of Search (besides Voice search and Vertical search) that are set to change the digital marketing landscape in 2020 and beyond. It is important to start incorporating these strategies your business’ long-term plan to fully fulfill your marketing potential. Even if your business does not specialize in e-commerce, it is always helpful to present clean, professional images on your website that are also optimized to increase your visibility on all search platforms.

Speak to an UpOnline representative today to learn more about visual search!

Related Article: OK Google, How Do I Optimize My Business for Voice Search?