“A picture is worth a thousand words.” Now that we have modern picture-search technology, does that old adage still hold true? 

All indications are definitely. According to Forbes, for instance, “91% of consumers prefer interactive and visual content over traditional, text-based or static media.”

And that’s no surprise, as more than half of the human brain is dedicated to vision and visual processing.

On the Web, as everywhere, one-dimensional text will never rate against majestic landscape photos or hilarious pet snapshots. People are clamoring for pretty pictures of everything from nature to hot products, and their attachment to digital images and image-search technology is only growing. Mobile-device-toting young folks (like millennials and especially Generation Z) have simply become too addicted to the joy of locating cool or relevant images in a matter of seconds. All of which has led to the ecommerce game-changer, visual shopping.

Those not yet tuned in to the advancements in image search can still search by entering a word or phrase in their web browser on their Mac or PC. But the results are the same regardless of the operating system: you can pull up hundreds or thousands of pictures in various formats (e.g., JPG, GIF, PNG) and different sizes, with image retrieval subcategories for deep diving into every variation. You can even click the camera icon to find out where an image URL originated on the Web.

“J-Lo green dress”

It may be hard to believe, but only a couple of decades ago, the concept of image search, so ingrained today, was no more than a glint in Google’s eye. 

In 2001, a bunch of ravenous Jennifer Lopez fans were entering text queries in Google’s search box with the hope of laying eyes on a unique green dress the singer had worn at the Grammys in 2000. Google execs noticed the frenzy, and the (green) light went on. They realized that they must immediately launch an image-focused version of their budding text-based search tool. 

The rest is history: now, people searching for photos of stars in jungle dresses or anything else have the luxury of using image-finder search engines to scour databases for just the right picture. The search-engine options range from big-name text search engines that also accommodate searching for images to image-only niche engines that “reverse search” (which we’ll talk about in a bit) to niche image-finding and image-oriented apps.

And because people have been uploading images to the Web in droves for years (300 million photos are uploaded to the Web every day), there’s no shortage (bazillions, to be exact) to choose from. Some photo libraries are continually updated with new content, while others are “complete” collections of images available for the searching, but in any case, the odds of finding the perfect image are nothing short of awesome. 

What’s a picture search engine?

Let’s start with the basics: a visual search engine is one that lets you look specifically for and download images on the Web to use for any number of purposes, such as illustrating a blog post, livening up a home page, conveying an idea in an overly texty report, or locating a fashion or home-decor item you want to buy.

People can search for images in various ways: 

• Type in a keyword or phrase describing the image they want
• Select an image topic on a menu
• Paste a JPEG or other type of image file in the search box as the query (to find similar images)

What are images?

Images may look complicated, but they’re actually just collections of pixels that are either illuminated or not illuminated. If you’ve ever looked at a TV up close, you’ve seen something like this pattern, which is illumination of the primary-colored pixels red, green, and blue (RGB):

How does an image search engine work?

The short answer: An image search engine works in the same way a text search engine does to give you the most relevant results: it pulls up a bunch of images based on a keyword or image. As when searching by text, an image search considers patterns and then points you to web sites based on matches.

The text associated with an image, such as its file name, can also play a role in the image search and discovery process. A search engine confirms that an image is related to the keyword, which involves checking out the data on the web page on which it appears.

In many cases, you can find an image based just on its file name and the context you want. Depending on the broadness of your search query, an image search engine may also provide a cluster of images that have matching content, and you can then identify the specific context in order to drill down and find the exact-right photo, drawing, painting, or other type of image you need.

From pretty pictures to mathematical formulas…

What’s going on beneath that instantly and seemingly effortless picture-finder task?

Visual search basically feeds the grid/matrix of an image into a neural network that’s trained to interpret the input. Neural networks are incredibly powerful machine-learning tools for analyzing and extracting visual information from images. 

Finding similar images is done by training the network to recognize similar visual patterns. This process can also be used to identify specific objects, such as faces. 

If you ask the Google image search feature, for instance, to find something similar to the image you’re providing, it does what’s called content based image retrieval (CBIR); it analyzes the image to collect details such as color and texture. Then it creates a query to match with other (billions of) images to present you with the most similar matches.

In order to use text input for visual search, the text is transformed into mathematics (vectors). Text, image alt text, and associated images are represented mathematically together for training and search optimization. The concepts of the text are encoded in a vector space and, with training, the model learns their relation to the images. In other words, it figures out what the images are and becomes able to deliver similar item results.

…with a little help from AI

A visual search also relies on advanced algorithms to steer it in the right direction. The ability of artificial intelligence (AI) models to describe what’s happening in images — along with classifying them in distinct groups to deliver visual search results — is unparalleled.

This type of search is available from Bing, Google Images, and Google Search, as well as DuckDuckGo (through Bing). Google has used it for both web and Google photo image search.

For example, this Google Images tab shows what’s been extracted from possibly millions of possible matches and their contexts in articles and blog posts across the Internet:

The best news? The models used by public picture search engines are available for anyone and any company to build on.

Online visual search APIs such as the Google Vision API show what can be achieved:

Advantages of extracting structured data

There are several positive reasons to pull structured data out of images:

• Image data can be combined with other structured information (such as text)
• It works with existing text search interfaces
• It works with voice-based queries

The first of these advantages is huge. Converting images to regular text and structured data stays nicely in the bucket of existing search technology. That means it can easily be extended to filters, facets, and more. And regarding the second bullet, a search interface people are familiar with is a major plus.

In contrast to using an image as a full query, there are distinct advantages to extracting structured data, while also offering your users a way to filter the data. For example if someone wants to search for a “blue polka dot dress under $50”, the image analysis would be fantastic for identifying matches in the right color and pattern, but it wouldn’t be able to provide the price. So in this case, extracting the information from the image would make more sense, and the data could then be combined with the price field (thereby offering even stronger relevance by filtering on price). 

The third advantage, compatibility with voice queries, is also significant. You may think the concept of voice search seems strange in a blog post about visual search, but voice recognition and image analysis are a powerful combination to be reckoned with. The ascent of Siri, Alexa, and other voice-driven programs has thrust voice search squarely into the mainstream. And when you combine it with AI-based visual analysis, you can, for example, search for things like “a house surrounded by redwood trees with a view.”  

Types of image search engines

An image search engine is one type of search engine, and there are also various subtypes of image search engines. The main image search engine types include (and sometimes overlap):

General image search

This is what it sounds like: the big-name search engine you’d visit to start browsing for a straightforward, likely available-in-many-variations image, like a photo of a bouquet of flowers or a famous athlete. A general image search engine is likely to present you with a wide variety of options.

Stock image search

A stock image is one that’s been photographed (or created in some other way, such as illustrated), edited (possibly retouched), and then made available in a website database to be downloaded for corporate or personal use. Stock images are mostly copyrighted, but they can be licensed (you make royalty payments), royalty free (you pay a one-time fee), free but with the requirement that you credit the photographer, or completely free (and possibly free to even adapt and use in your own projects), with no attribution required.

Reverse image search

If you have an image and want to locate others like it, or learn who’s using a photo you snapped or artwork you created (with or without your permission), or decipher where an image originated on the Web, or do any number of other interesting-to-know things, reverse photo search is your friend. You just paste the image file in as the search query and get search results pointing you to the image’s origin or items related to it.

Impressive photo-finder engines

Specifically, have you ever used Google Lens or Pinterest Lens (“Shop the look”) to take a picture of something with your mobile phone and then used reverse image lookup to identify it by searching for an online image? It’s amazing technology, and right at your fingertips.

When it comes to image search engines, you have many choices. It’s all based on your image needs and preferences and how you like to use features to search (e.g., browse lots of pages of general categories or use filters to immediately get specific).

The best image search engines aren’t typically ranked by reviewers, as they’re all slightly different and unique in terms of their offerings, interface functionality, approach toward fees and royalty freedom, and advantages. However, if you ask an amateur photographer, graphic designer, or web content creator, they will probably tell you there’s no single “best” image search engine, but they’ll then go on to mention the search option they prefer, and why.

And by conventional standards, there are some pretty clear front runners. Here are a few image search engines that are widely viewed as winners:

Google Images

No surprise here. Just like regular-old text-based Google, Google Images is the cream of the crop when it comes to finding a ton of photos and other types of images online: the hands-down-best image search experience with the most comprehensive collection of possibilities, plus the most outstanding ways (a powerful interface and loads of filters) to let you track them down. 

If you’re on a budget, you can sort your found-Google-image treasure according to usage rights. 

Google Images also has an advanced (via settings) option for helping you get specific. And it provides great reverse photo lookup functionality: you can simply paste in an image from your phone’s camera roll, a Dropbox folder, or your Google drive and search similar images.

Bing

Want to skip over the front runner and give the competition a try? Visit the home page for Microsoft Bing’s image search engine, which houses the next-largest database of images. Its pluses include a super colorful interface (whereas Google shows you just the search bar to start), large thumbnails, prominent filtering options, trending items, excellent video search functionality, and the ability to watch videos slightly more quickly without being forced onto YouTube. 

Some people think that for various reasons, Bing Images is superior to Google Images. Of course you can’t Google an image with Bing, but if the search experience and image results are just as good or better, who cares?

Yahoo

Why would you want to use the image search tool on Yahoo instead of the one on Google or Bing? Well, Yahoo owns the photography sharing and networking site Flickr (more on that below), so its database is full of cool photographer-provided shots. And the graceful Yahoo image interface lets you narrow your image search results using advanced filters for parameters such as type of license, file format, and image size.

Flickr

Photos by both amateur and professional photographers make Flickr, which boasts tens of billions of photos, an amazing visual experience to behold, not to mention use as a jumping-off point for finding the right related images (some of which are royalty free). Since Flickr is a social-media type of site, you can “follow” photographers; it also has a staggering 2 million groups of users, which it values as “the connective force of our community, bringing members with common interests together.” Well! This site could seriously be “The One.”

Pinterest

Pinterest subscribers create “pins” — bookmarks for saving content — on this image-search site that bills itself as a “visual discovery engine for finding ideas like recipes, home and style inspiration….” One cool feature: you can crop and search on a portion of an image. If you’re the type of person who gets addicted to decorating and happy homemaking, Pinterest could easily become your haven (if it’s not already).

Stock photo search engines

Oriented toward the needs of businesses, stock photo sites supply pictures, many of which have themes that appeal across subjects, as an economical alternative to having a photographer do a custom photo shoot. Here are a few of the main stock-photo sites out there for the exploring:

Shutterstock

One of the largest and best-known resources among graphic designers, marketing agencies, and other businesspeople, Shutterstock is the one that came up with the concept of buying a subscription in order to download stock photos. This site supplies not only royalty-free photos but videos and music.

Getty Images

Getty Images offers 350 million high-quality, archival, and sometimes exclusive images targeted to deep-pocketed commercial licensing. Its images are typically higher priced and have usage limitations (such as requiring an image be displayed only in a particular time frame). Some of its practices have been controversial.

iStock

If you’re not a corporate giant or you’re on a budget, then iStock, the Getty Images “microstock” subsite (royalty-free images), could be your happy photo place.

Reverse image search

TinEye

Toronto-based TinEye’s claim to fame is that it was the very first reverse image search tool, as well as the first to use image identification techniques instead of keywords. This happened three years before Google added its reverse image search-engine functionality. TinEye also thinks its matching is superior to Google’s and that it wins at finding cropped and edited images. 

One bummer: the maximum size of an image you can upload to TinEye for reverse searching is only 20 megabytes, which means you may have to go through the hassle of first saving a photo in a lower resolution. 

Google Reverse Image Search

Google is of course a major player in the subcategory of reverse image search. There’s no limit on the image size you can upload, which means you can quickly reverse search no matter how many megabytes you’re dealing with. 

Pinterest Lens

The reverse picture search feature on Pinterest, Lens, is handy for letting people snap photos of things they see out in the world (for instance, a fashionable piece of clothing worn by someone in a different country or must-have home decor at a friend’s apartment) and then look for similar photos of items online. In 2020, Pinterest added a dedicated shopping tab to Lens, which links item images to companies’ ecommerce check-out pages.

Apps

Google has a “lens” feature too, an image-recognition app (downloadable from the iOS and Android stores). Google Lens does visual analysis and provides related information; it can translate text, identify animals, pull up similar images, and lots more.

There are also some apps out there that are dedicated to reverse searching, including:

• Photo Sherlock
• Reversee
• Veracity

How to find free images

Some images cost big bucks to license, and while cash-flush companies will have no trouble paying high fees, if you’re just looking for something colorful to jazz up a dull newsletter, it can be demoralizing to find the perfect image and then discover that it has a hefty price tag and you can’t or don’t want to pay the fee.

Fortunately, there are some image search engines dedicated to providing not just royalty-free images but actually free ones, some with no strings attached. For that, you can thank a nonprofit called Creative Commons (renamed Openverse), which is committed to fostering creativity and sharing on the Internet.

The Openverse interface permits searching of an impressive collection of images. Image reprint permissions range from narrow, with a requirement that the original source of an image be credited, to very liberal — being allowed to use and even adapt the artist’s work in new ways for your own creative projects without citing the image source.

While you can search for Openverse-licensed images on general sites, you may be able to zero in on a wider array on sites that specialize in royalty-free images, many of which don’t require fees. These include:

EveryPixel

EveryPixel’s AI-powered search engine indexes what it finds on 50 image sites in a giant database that’s easily searchable using a variety of types of filters.

Librestock

This organization’s multisite search engine mines public-domain-only (usable without attribution) photos from a number of other free stock photo sites and makes them all accessible from its interface. Unfortunately, Librestock provides fewer photos than other sites (roughly 5,400 at the time of this writing), but hey, how could you complain if they’re free?

Pixabay

This image site supplies searchers with more than 2.5 million high-quality images (including illustrations and vector graphics), along with videos, plus music.

Ecommerce image-search apps

Visual search is a powerful alternative to text- and voice-based search functionality, especially for use cases like ecommerce search.

Image search engines also include AI-powered apps used by online stores to help their customers more quickly find and buy desired items. When a shopper enters an image of something such as an outfit, the search engine looks for an exact match and similar items (possibly from other brands, too). It may also make image suggestions based on what it knows about the customer’s preferences or items that are often purchased along with the target item.

This image-search technology is promising: according to Invesp, 74% of online shoppers believe text-based search is ineffective when it comes to finding the right product, and 72% say they “regularly or always search for visual content before making a purchase.”

And this strong consumer demand is backed up by statistics like this from Predictly: the forecast for the global visual search market is for almost $15 million by 2023. Plus, Gartner projected that companies adapting fast in 2021 and redesigning their websites to support visual search would increase their digital revenue by 30%.

One study (National Research Group) found that consumers are 50% more likely to be influenced to buy from seeing visual search results. For retailers, that’s a rather compelling reason to invest in visual search. Ecommerce and home decor are arguably the best use cases.

Entire visual-search-engine companies have also sprung up to offer AI-powered “computer vision,” particularly for ecommerce use cases.

And visual search is not just being used for selling products. Augmented reality (AR) is expected to generate billions in the next few years for commerce-related sales, particularly among Millennials and Gen Z, and visual search will account for a big chunk of that. Even Snapchat has gotten into the game with visual search optimized for AR. 

Where visual search is headed: everywhere

Outside of public search engines and big ecommerce, the ability to use image search has still been fairly limited. However, the world according to image search is changing fast. You can expect to see more visual search features in the next few years as the technology finds its way into smaller-sized businesses.

Two examples:

• Facebook updates used to be mostly text, then they became much more visual
• Instagram and Pinterest have both taken off as visual social platforms

Putting visual search into practice

So how can you put visual search into practice for your ecommerce site or app? Here are a few ideas for inspiration: 

• Amazon.com added a visual search feature called StyleSnap in 2019. It lets shoppers look for fashion and home-decor items by using an AI-powered image (or screenshot) search. Among other things, this tool fuels influencers who love to use their iPhones or Android phones to post their fashion finds on social media.
• At Algolia, we used the Google Vision API to add visual search capabilities to an ecommerce database. With the API, we can automatically extract color and other metadata from images as they’re being indexed, which can then be used to design search filters and facets. Now, anytime new products are added to the site, the API automatically extracts the color data for use in filters.

Here’s what it looks like:

• Algolia also provides a reverse image search app for ecommerce: companies can employ a third-party API or platform to use images as search queries.

Other image-related applications

In addition, there are some applications of image search technology that don’t involve doing an image search or even bothering with a search bar. For example, Algolia has an app that incorporates image search technology to quickly identify recipients of delivered packages so items can be expediently passed along. Optical character recognition (OCR) is used to extract the text from label images, but the resulting content may contain typos, so integrated search technology takes images into account. A search engine that has robust, adaptable relevance can match OCR unstructured text against a structured set of data and return accurate results.

A beautiful big picture

There’s no question that image-search websites and applications have been taking the Internet by storm. For now, image search engines aren’t about to displace text-based search engines, but they’re a strong and growing presence in the online search world. The future of searching won’t revolve around only visual, voice, or text search alone. It’s going to incorporate all of them — and they’re only the beginning (just wait ’til we have brain-powered search).

Consumers have seemingly limitless, excellent options for digging up the perfect photo or other visual element. Companies are busy optimizing for visual search in order to attract and retain customers. The corporate world is increasingly benefiting from the rollout of image search technology.

Want to optimize your ecommerce site search before your competitors see the light behind the  pixels? Connect with our team today and let’s envision all your possibilities.