the home of online investigations

You can support the work of Bellingcat by donating through the following link:

Guide To Using Reverse Image Search For Investigations

December 26, 2019

By Aric Toler

Reverse image search is one of the most well-known and easiest digital investigative techniques, with two-click functionality of choosing “Search Google for image” in many web browsers. This method has also seen widespread use in popular culture, perhaps most notably in the MTV show Catfish, which exposes people in online relationships who use stolen photographs on their social media.

However, if you only use Google for reverse image searching, you will be disappointed more often than not. Limiting your search process to uploading a photograph in its original form to just images.google.com may give you useful results for the most obviously stolen or popular images, but for most any sophisticated research project, you need additional sites at your disposal — along with a lot of creativity.

This guide will walk through detailed strategies to use reverse image search in digital investigations, with an eye towards identifying people and locations, along with determining an image’s progeny. After detailing the core differences between the search engines, Yandex, Bing, and Google are tested on five test images showing different objects and from various regions of the world.

Beyond Google

The first and most important piece of advice on this topic cannot be stressed enough: Google reverse image search isn’t very good.

As of this guide’s publication date, the undisputed leader of reverse image search is the Russian site Yandex. After Yandex, the runners-up are Microsoft’s Bing and Google. A fourth service that could also be used in investigations is TinEye, but this site specializes in intellectual property violations and looks for exact duplicates of images.

Yandex

Yandex is by far the best reverse image search engine, with a scary-powerful ability to recognize faces, landscapes, and objects. This Russian site draws heavily upon user-generated content, such as tourist review sites (e.g. FourSquare and TripAdvisor) and social networks (e.g. dating sites), for remarkably accurate results with facial and landscape recognition queries.

Its strengths lie in photographs taken in a European or former-Soviet context. While photographs from North America, Africa, and other places may still return useful results on Yandex, you may find yourself frustrated by scrolling through results mostly from Russia, Ukraine, and eastern Europe rather than the country of your target images.

To use Yandex, go to images.yandex.com, then choose the camera icon on the right.

From there, you can either upload a saved image or type in the URL of one hosted online.

If you get stuck with the Russian user interface, look out for Выберите файл (Choose file), Введите адрес картинки (Enter image address), and Найти (Search). After searching, look out for Похожие картинки (Similar images), and Ещё похожие (More similar).

The facial recognition algorithms used by Yandex are shockingly good. Not only will Yandex look for photographs that look similar to the one that has a face in it, but it will also look for other photographs of the same person (determined through matching facial similarities) with completely different lighting, background colors, and positions. While Google and Bing may just look for other photographs showing a person with similar clothes and general facial features, Yandex will search for those matches, and also other photographs of a facial match. Below, you can see how the three services searched the face of Sergey Dubinsky, a Russian suspect in the downing of MH17. Yandex found numerous photographs of Dubinsky from various sources (only two of the top results had unrelated people), with the result differing from the original image but showing the same person. Google had no luck at all, while Bing had a single result (fifth image, second row) that also showed Dubinsky.

Yandex is, obviously, a Russian service, and there are worries and suspicions of its ties (or potential future ties) to the Kremlin. While we at Bellingcat constantly use Yandex for its search capabilities, you may be a bit more paranoid than us. Use Yandex at your own risk, especially if you are also worried about using VK and other Russian services. If you aren’t particularly paranoid, try searching an un-indexed photograph of yourself or someone you know in Yandex, and see if it can find yourself or your doppelganger online.

Bing

Over the past few years, Bing has caught up to Google in its reverse image search capabilities, but is still limited. Bing’s “Visual Search”, found at images.bing.com, is very easy to use, and offers a few interesting features not found elsewhere.

Within an image search, Bing allows you to crop a photograph (button below the source image) to focus on a specific element in said photograph, as seen below. The results with the cropped image will exclude the extraneous elements, focusing on the user-defined box. However, if the selected portion of the image is small, it is worth it to manually crop the photograph yourself and increase the resolution — low-resolution images (below 200×200) bring back poor results.

Below, a Google Street View image of a man walking a couple of pugs was cropped to focus on just the pooches, leading to Bing to suggest the breed of dog visible in the photograph (the “Looks like” feature), along with visually similar results. These results mostly included pairs of dogs being walked, matching the source image, but did not always only include pugs, as French bulldogs, English bulldogs, mastiffs, and others are mixed in.

Google

By far the most popular reverse image search engine, at images.google.com, Google is fine for most rudimentary reverse image searches. Some of these relatively simple queries include identifying well-known people in photographs, finding the source of images that have been shared quite a bit online, determining the name and creator of a piece of art, and so on. However, if you want to locate images that are not close to an exact copy of the one you are researching, you may be disappointed.

For example, when searching for the face of a man who tried to attack a BBC journalist at a Trump rally, Google can find the source of the cropped image, but cannot find any additional images of him, or even someone who bears a passing resemblance to him.

While Google was not very strong in finding other instances of this man’s face or similar-looking people, it still found the original, un-cropped version of the photograph the screenshot was taken from, showing some utility.

Five Test Cases

For testing out different reverse image search techniques and engines, a handful of images representing different types of investigations are used, including both original photographs (not previously uploaded online) and recycled ones. Due to the fact that these photographs are included in this guide, it is likely that these test cases will not work as intended in the future, as search engines will index these photographs and integrate them into their results. Thus, screenshots of the results as they appeared when this guide was being written are included.

These test photographs include a number of different geographic regions to test the strength of search engines for source material in western Europe, eastern Europe, South America, southeast Asia, and the United States. With each of these photographs, I have also highlighted discrete objects within the image to test out the strengths and weaknesses for each search engine.

Feel free to download these photographs (every image in this guide is hyperlinked directly to a JPEG file) and run them through search engines yourself to test out your skills.

Olisov Palace In Nizhny Novgord, Russia (Original, not previously uploaded online)

Isolated: White SUV in Nizhny Novgorod

Isolated: Trailer in Nizhny Novgorod

Cityscape In Cebu, Philippines (Original, not previously uploaded online)

Isolated: Condominium complex, “The Padgett Place

Isolated: “Waterfront Hotel

Students From Bloomberg 2020 Ad (Screenshot from video)

Isolated: Student

Av. do Café In São Paulo, Brazil (Screenshot from Google Street View)

Isolated: Toca do Açaí

Isolated: Estacionamento (Parking)

Amsterdam Canal (Original, not previously uploaded online)

Isolated: Grey Heron

Isolated: Dutch Flag (also rotated 90 degrees clockwise)

Results

Each of these photographs were chosen in order to demonstrate the capabilities and limitations of the three search engines. While Yandex in particular may seem like it is working digital black magic at times, it is far from infallible and can struggle with some types of searches. For some ways to possibly overcome these limitations, I’ve detailed some creative search strategies at the end of this guide.

Novgorod’s Olisov Palace

Predictably, Yandex had no trouble identifying this Russian building. Along with photographs from a similar angle to our source photograph, Yandex also found images from other perspectives, including 90 degrees counter-clockwise (see the first two images in the third row) from the vantage point of the source image.

Yandex also had no trouble identifying the white SUV in the foreground of the photograph as a Nissan Juke.

Lastly, in the most challenging isolated search for this image, Yandex was unsuccessful in identifying the non-descript grey trailer in front of the building. A number of the results look like the one from the source image, but none are an actual match.

Bing had no success in identifying this structure. Nearly all of its results were from the United States and western Europe, showing houses with white/grey masonry or siding and brown roofs.

Likewise, Bing could not determine that the white SUV was a Nissan Juke, instead focusing on an array of other white SUVs and cars.

Lastly, Bing failed in identifying the grey trailer, focusing more on RVs and larger, grey campers.

Google‘s results for the full photograph are comically bad, looking to the House television show and images with very little visual similarity.

Google successfully identified the white SUV as a Nissan Juke, even noting it in the text field search. As seen with Yandex, feeding the search engine an image from a similar perspective as popular reference materials — a side view of a car that resembles that of most advertisements — will best allow reverse image algorithms to work their magic.

Lastly, Google recognized what the grey trailer was (travel trailer / camper), but its “visually similar images” were far from it.

Scorecard: Yandex 2/3; Bing 0/3; Google 1/3

Cebu

Yandex was technically able to identify the cityscape as that of Cebu in the Philippines, but perhaps only by accident. The fourth result in the first row and the fourth result in the second row are of Cebu, but only the second photograph shows any of the same buildings as in the source image. Many of the results were also from southeast Asia (especially Thailand, which is a popular destination for Russian tourists), noting similar architectural styles, but none are from the same perspective as the source.

Of the two buildings isolated from the search (the Padgett Palace and Waterfront Hotel), Yandex was able to identify the latter, but not the former. The Padgett Palace building is a relatively unremarkable high-rise building filled with condos, while the Waterfront Hotel also has a casino inside, leading to an array of tourist photographs showing its more distinct architecture.

Bing did not have any results that were even in southeast Asia when searching for the Cebu cityscape, showing a severe geographic limitation to its indexed results.

Like Yandex, Bing was unable to identify the building on the left part of the source image.

Bing was unable to find the Waterfront Hotel, both when using Bing’s cropping function (bringing back only low-resolution photographs) and manually cropping and increasing the resolution of the building from the source image. It is worth noting that the results from these two versions of the image, which were identical outside of the resolution, brought back dramatically different results.

As with Yandex, Google brought back a photograph of Cebu in its results, but without a strong resemblance to the source image. While Cebu was not in the thumbnails for the initial results, following through to “Visually similar images” will fetch an image of Cebu’s skyline as the eleventh result (third image in the second row below).

As with Yandex and Bing, Google was unable to identify the high-rise condo building on the left part of the source image. Google also had no success with the Waterfront Hotel image.

Scorecard: Yandex 4/6; Bing 0/6; Google 2/6

Bloomberg 2020 Student

Yandex found the source image from this Bloomberg campaign advertisement — a Getty Images stock photo. Along with this, Yandex also found versions of the photograph with filters applied (second result, first row) and additional photographs from the same stock photo series. Also, for some reason, porn, as seen in the blurred results below.

When isolating just the face of the stock photo model, Yandex brought back a handful of other shots of the same guy (see last image in first row), plus images of the same stock photo set in the classroom (see the fourth image in the first row).

Bing had an interesting search result: it found the exact match of the stock photograph, and then brought back “Similar images” of other men in blue shirts. The “Pages with this” tab of the result provides a handy list of duplicate versions of this same image across the web.

Focusing on just the face of the stock photo model does not bring back any useful results, or provide the source image that it was taken from.

Google recognizes that the image used by the Bloomberg campaign is a stock photo, bringing back an exact result. Google will also provide other stock photos of people in blue shirts in class.

In isolating the student, Google will again return the source of the stock photo, but its visually similar images do not show the stock photo model, rather an array of other men with similar facial hair. We’ll count this as a half-win in finding the original image, but not showing any information on the specific model, as Yandex did.

Scorecard: Yandex 6/8; Bing 1/8; Google 3.5/8

Brazilian Street View

Yandex could not figure out that this image was snapped in Brazil, instead focusing on urban landscapes in Russia.

For Toca do Açaí, for some reason, Yandex mostly brought back porn as results. These images were blurred, and you can click here to see the results. However, despite the blurred smut, two of the results did correctly identify the logo.

For the parking sign [Estacionamento], Yandex did not even come close.

Bing did not know that this street view image was taken in Brazil.

…nor did Bing recognize the parking sign

…or the Toca do Açaí logo.

Despite the fact that the image was directly taken from Google’s Street View, Google reverse image search did not recognize a photograph uploaded onto its own service.

Just as Bing and Yandex, Google could not recognize the Portuguese parking sign.

Lastly, Google did not come close to identifying the Toca do Açaí logo, instead focusing on various types of wooden panels, showing how it focused on the backdrop of the image rather than the logo and words.

Scorecard: Yandex 7/11; Bing 1/11; Google 3.5/11

Amsterdam Canal

Yandex knew exactly where this photograph was taken in Amsterdam, finding other photographs taken in central Amsterdam, and even including ones with various types of birds in the frame.

Yandex correctly identified bird in the foreground of the photograph as a grey heron (серая цапля), also bringing back an array of images of grey herons in a similar position and posture as the source image.

However, Yandex flunked the test of identifying the Dutch flag hanging in the background of the photograph. When rotating the image 90 degrees clockwise to present the flag in its normal pattern, Yandex was able to figure out that it was a flag, but did not return any Dutch flags in its results.

 

Bing only recognized that this image shows an urban landscape with water, with no results from Amsterdam.

Though Bing struggled with identifying an urban landscape, it correctly identified the bird as a grey heron, including a specialized “Looks like” result going to a page describing the bird.

However, like with Yandex, the Dutch flag was too confusing for Bing, both in its original and rotated forms.

Google noted that there was a reflection in the canal of the image, but went no further than this, focusing on various paved paths in cities and nothing from Amsterdam.

Google was close in the bird identification exercise, but just barely missed it — it is a grey, not great blue, heron.

Google was also unable to identify the Dutch flag. Though Yandex seemed to recognize that the image is a flag, Google’s algorithm focused on the windowsill framing the image and misidentified the flag as curtains.

Final Scorecard: Yandex 9/14; Bing 2/14; Google 3.5/14

Creative Searching

Even with the shortcomings described in this guide, there are a handful of methods to maximize your search process and game the search algorithms.

Specialized Sites

For one, you could use some other, more specialized search engines outside of the three detailed in this guide. The Cornell Lab’s Merlin Bird ID app, for example, is extremely accurate in identifying the type of birds in a photograph, or giving possible options. Additionally, though it isn’t an app and doesn’t let you reverse search a photograph, FlagID.org will let you manually enter information about a flag to figure out where it comes from. For example, with the Dutch flag that even Yandex struggled with, FlagID has no problem. After choosing a horizontal tricolor flag, we put in the colors visible in the image, then receive a series of options that include the Netherlands (along with other, similar-looking flags, such as the flag of Luxembourg).

Language Recognition

If you are looking at a foreign language with an orthography you don’t recognize, try using some OCR or Google Translate to make your life easier. You can use Google Translate’s handwriting tool to detect the language* of a letter that you hand-write, or choose a language (if you know it already) and then write it out yourself for the word. Below, the name of a cafe (“Hedgehog in the Fog“) is written out with Google Translate’s handwriting tool, giving the typed-out version of the word (Ёжик) that can be searched.

*Be warned that Google Translate is not very good at recognizing letters if you do not already know the language, though if you scroll through enough results, you can find your handwritten letter eventually.

Pixelation And Blurring

As detailed in a brief Twitter thread, you can pixelate or blur elements of a photograph in order to trick the search engine to focus squarely on the background. In this photograph of Rudy Giuliani’s spokeswoman, uploading the exact image will not bring back results showing where it was taken.

However, if we blur out/pixelate the woman in the middle of the image, it will allow Yandex (and other search engines) to work their magic in matching up all of the other elements of the image: the chairs, paintings, chandelier, rug and wall patterns, and so on.

After this pixelation is carried out, Yandex knows exactly where the image was taken: a popular hotel in Vienna.

Conclusion

Reverse image search engines have progressed dramatically over the past decade, with no end in sight. Along with the ever-growing amount of indexed material, a number of search giants have enticed their users to sign up for image hosting services, such as Google Photos, giving these search algorithms an endless amount of material for machine learning. On top of this, facial recognition AI is entering the consumer space with products like FindClone and may already be used in some search algorithms, namely with Yandex. There are no publicly available facial recognition programs that use any Western social network, such as Facebook or Instagram, but perhaps it is only a matter of time until something like this emerges, dealing a major blow to online privacy while also (at that great cost) increasing digital research functionality.

If you skipped most of the article and are just looking for the bottom line, here are some easy-to-digest tips for reverse image searching:

  • Use Yandex first, second, and third, and then try Bing and Google if you still can’t find your desired result.
  • If you are working with source imagery that is not from a Western or former Soviet country, then you may not have much luck. These search engines are hyper-focused on these areas, and struggle for photographs taken in South America, Central America/Caribbean, Africa, and much of Asia.
  • Increase the resolution of your source image, even if it just means doubling or tripling the resolution until it’s a pixelated mess. None of these search engines can do much with an image that is under 200×200.
  • Try cropping out elements of the image, or pixelating them if it trips up your results. Most of these search engines will focus on people and their faces like a heat-seeking missile, so pixelate them to focus on the background elements.
  • If all else fails, get really creative: mirror your image horizontally, add some color filters, or use the clone tool on your image editor to fill in elements on your image that are disrupting searches.
Aric Toler

Aric Toler started volunteering for Bellingcat in 2014 and has been on staff since 2015. He currently heads up Bellingcat's training efforts and its Eastern Europe/Eurasia research.

Join the Bellingcat Mailing List:

Enter your email address to receive a weekly digest of Bellingcat posts, links to open source research articles, and more.

30 Comments

  1. mvario

    Agreed, Yandex does have the best reverse image search. It also has an English language interface so one should never have to figure out the Russian if English is one’s first language.

    Reply
    • Aric Toler

      (Sorry for late reply)
      True! But sometimes when I navigate the site, I’ll randomly get thrown into the Russian UI (or if I’m already in the Russian UI, it’ll throw me into the English one).

      Reply
  2. Nick

    Hi,
    Thanks again for a great and very useful article.
    Sorry for being picky though. Found this:
    “You can use Google Translate’s handwriting took to detect the language* of a letter that you hand-write, or choose a language (if you know it already) and then write it out yourself for the word. ”

    I can be wrong but, I would replace the word “took” to “tool”.

    Cheers,
    Nick

    Reply
    • Jack Schofield

      Yandex Image Search often pulls up results from Twitter and vk.

      I don’t remember seeing them on other image search engines but it’s a couple of years since I’ve used anything except Yandex, TinEye and Karma Decay.

      TinEye has great image search algorithms but its database is small and often very out of date. You can sort hits by “oldest”, which is helpful when you’re looking for original images rather than doctored ones.

      Karma Decay only finds matches in Reddit. However, news photos often appear on Reddit, often with useful comments about what they show and sometimes their origins. It has limited uses but it’s worth trying with the right sort of photo.

      Reply
      • Aric Toler

        I’ll try out Karma Decay, thanks! May be worth trying out a bit more.

        And yes, Yandex is really good about pulling from social networks. Google will pull from Twitter, but it’s in a pretty nonsensical way – link to an account’s page rather than the tweet where a photo appears – thus restricting the results to indexed account pages rather than for each individual tweet.

        Reply
  3. George

    Hi Aric, great article. I have similar observations that currently Yandex is on the top of the game for reverse image search. We have made a tool to save some time and search by image on all major searches (link here https://hostingchecker.com/tools/reverse-image-search/)

    I see that you are interested in online forensics so you can give a go to the other tools (esp the reverse ip lookup, we have a db for about 250mln active domains and its free to use.) And if you find an ip with more than a 1000 results drop me an email and i can make a report for you.

    George

    Reply
    • Scampolice Group

      Thanks for the comment George. Excellent article from Bellincat as usual. I hate the word “Catfishing” however. I regard it as just plain identity theft and often, blatant 419 romance scams. It is all about criminal activity and high value theft and fraud and the sooner the term “catfishing” is forgotten the better. The more tools potential victims have the better off they are, financially and emotionally. I support Bellingcat and the better we become at investigating all crimes inevitably makes the world safer

      Reply
  4. Ariella

    Something I’ve discovered from similar experiments is that Google Images’ priorities seem to be 1) semantic information and 2) colour profile.

    If you screenshot a well known black and white portrait (I used a much copied image of Canadian Prime Minister Wilfrid Laurier from circa 1900) and save it with a title that does not provide any useful metadata, Tineye will find 190 examples of that exact portrait–sometimes with modest cropping–but Google will show you one instance of that particular portrait and dozens of other pictures of Laurier, as if you had entered “Wilfrid Laurier” in the text search box. Importantly, a portrait of an actor playing Laurier in a biopic ranks quite highly, so facial recognition may not be a factor in the search at all.

    Farther down the results ranking, around image #45, Google will also show you portraits of completely different people from pages that mention Wilfrid Laurier’s name. However, the results will not show you any of the schools and other buildings named after Wilfrid Laurier, so the algorithm can at least distinguish a face from other kinds of images.

    If you use image editing software to turn the classic portrait of Willy pink, Tineye will find slightly fewer examples of that exact photo, but Google Images will start finding portraits of Wilfrid Laurier that contain a lot of red pixels. From the tenth result onward, the results are portraits of other people that contain a lot of red pixels and come from pages that mention Wilfrid Laurier’s name on them somewhere. Changing the colour also changes the Yandex results, though not as dramatically.

    The tl;dr is that if you’re having trouble finding matches for a face, try turning the picture black and white, or give a black and white photo some kind of flesh-toned colour filter.

    Reply
  5. Deshandre Watkins

    Can Yandex or Bing defeat tumblr’s invisible watermark? An unfortunate, lasting legacy of tumblr is that that if an image was ever saved to tumblr it is completely and utterly unsearchable in google image search except to find other instances of the tumblr version of that exact image and that exact image only.

    The water mark seems to encompass the entire image and survives recompression with JPEG and even doing an AI upscale with an app like Gigapixel AI.

    Reply
  6. Bal Krishna

    Thanks a lot for this very insightful article Aric. Makes for a great reading. Can I please request you for two things?
    One, can you please elaborate on this “Increase the resolution of your source image, even if it just means doubling or tripling the resolution until it’s a pixelated mess. None of these search engines can do much with an image that is under 200×200.”. Does that mean we should take more area during cropping or we should pick up smaller and precise area? I think that taking more area will pixelate it less.
    Second, which is better – downloading the image and uploading it to search engine for search or just putting the url into the search bar? Or, both will have the same result. I’ve seen a couple of times that downloading and uploading the image – gave better results than submitting url. Is that the case?

    Reply
    • cpc2

      I think they meant either increasing the size of the image directly or upscaling it. For example, here’s an image that none of the search engines can find: https://i.imgur.com/uVnSudY.png
      However, if we upscale it https://i.imgur.com/XQysNmg.png the results will be different. In this case Bing can find it somehow (https://i.imgur.com/n0xvT0q.png), even though it’s just the same image but upscaled x2.

      I don’t think downloading vs using URL will give better or worse results, but it can give different results, so it’s worth a try. There are many ways to force different outputs for really similar images.

      Reply
    • Aric Toler

      (Sorry for the late reply)
      Hi Bal,

      By increasing the resolution, I mean using an image editor and manually increasing the resolution by stretching it out. I think you can do this on MS Paint, and you can definitely do it on most non-barebones image editing tools. For example, if I have an image that’s 150×100, I can stretch it (by keeping the aspect ratio) to 300×200 on most all image editors.

      I download them, but I don’t think it makes a different. Some sites, like Twitter, use strange URL structures for their images by not direct-linking to the JPEGs (you get things like =orig, =large, etc at the end of Twitter photo URLs), so I either directly download the image or take a large screenshot of it in high-resolution.

      Reply
  7. cpc2

    Hi, this article was pretty nice! I use reverse image search almost daily (for a Reddit game about finding locations, objects, etc.) and out of these Yandex is usually the most impressive. It’s a shame because it could be even better if it could also index sites like Instagram. For Yandex sometimes cropping in different ways until you get a result can actually work. For example, for that cropped building it couldn’t crop, if you crop it like this (https://i.imgur.com/yRuq73y.png) instead it will find it: https://i.imgur.com/zHclM8V.png Not sure why it finds it with one crop and not the other, but it’s always worth trying.

    Also, comparing results to other users, Yandex, Bing and Google can sometimes show results for some people that don’t appear for others, we don’t know why it happens. Might be based on location or cookies. If you really want to find something it might be worth it to try it in different browsers with different IPs. Also, sometimes it’s useful to scroll down on Yandex’s results because in some cases it can somehow find it buried deep in the results.

    Good luck on your research!

    Reply

Leave a Reply

  • (will not be published)

You can support the work of Bellingcat by donating through the following link:

TRUST IN JOURNALISM - IMPRESS