Using AI to Beat TimeGuessr

I am currently working on a project which requires me to identify the locations depicted in works of art (more about this, hopefully, very soon). In order to narrow down the exact locations shown in the paintings I have begun to use two AI image identification tools GeoSpy and Bard.

The sketch shown at the top of this post is the 'Tour de Montelban, Amsterdam' by Maxime Lalanne. I downloaded the image from the Cleveland Museum of Art. The location of this scene is easy to find thanks to the painting's title and because the tower still stands in Amsterdam. Because we know the exact location of this scene it can be used to test how well GeoSpy and Bard are able to calculate the locations shown in the painting.

I uploaded the Tour de Montleban into both GeoSpy and Bard. Almost instantaneously GeoSpy responded that the 'photo was taken in Amsterdam, Netherlands'. The actual longitude and latitude given was a little off the exact location but GeoSpy was impressively able to determine the correct city (if not the correct medium). In its response, 

'The architecture of the buildings and the canal are typical of the city. The vegetation is also consistent with the climate of the Netherlands. The soil is likely to be sandy, as this is the most common type of soil in the Netherlands. ',

we can also get a sense of some of the clues GeoSpy uses to identify locations from images (architecture, climate and nature).

Bard actually identified the name of the painting and the artist. I think this reveals that Google uses its extensive knowledge of the internet to do a comparative image search of the web. However from my previous attempts at using Bard to geolocate photos, I know that Bard also uses visual clues in the image to help determine its answer.

In this case Bard responded that,

'The etching shows the harbor of Amsterdam in the Netherlands, with the Montalban Tower in the background.'

While very impressive Bard has also not identified the exact location of the image. However as Bard is a chat AI it is possible to ask for more information. I therefore asked Bard to give me the longitude and latitude and it responded by showing me a Google Map with a marker exactly on the Montalban Tower in Amsterdam.

It was now time to get serious!  I therefore decided to test Geospy against TimeGuessr. TimeGuessr is a fun game which requires you to identify the year and location depicted in old photographs. It is a non-scientific but hugely diverting way to test the accuracy of GeoSpy's location detection skills. 

Round 1.

Round one of today's TimeGuessr Daily Challenge is a photograph of the Beatles on a sled in front of a building. GeoSpy identified the location as being Obergurgl, Austria. While GeoSpy got the correct country it was  actually 196.4 km off identifying the correct town (Obertauern). 

GeoSpy scored 4304 for the location of round 1. 

Round 2. 

The photo of round two is of two cars in front of a statue of what looks like a native American. GeoSpy identified the location as "the Golden Gate International Exposition on Treasure Island in San Francisco Bay". Handily for TimeGuessr it also said that the 'photo was taken in 1939'. Placing my pin in the middle of Treasure Island got me 552.1 metres from the correct location. The year was actually 1940 so I was one year out. 

GeoSpy scored 4989 for the location.

Round 3. 

This photo shows Kate Moss in a cafe. Through the cafe window some blurred buildings are visible but this looks to be a tough challenge for GeoSpy. However undeterred GeoSpy tells me that this is Paris. Placing my map pin on the Louvre I get within 772.0 metres of the correct location.

GeoSpy scored 4985 for the location of Round 3.

Round 4.

Round 4 is a black and white photo of what looks like a market square or plaza. GeoSpy is a little stumped and only responds that this 'photo was taken in what is now the Czech Republic'. Placing my pin in roughly the middle of the Czech Republic places me 161.2 km from the correct location.

GeoSpy scored 4339 points in this round.

Round 5. 

Round 5 involves two people staring over what I think is the Berlin Wall. GeoSpy agrees and tells me it was taken from the East German side of the wall. It gives me coordinates for the Unter den Linden. This was only 1.4 km from the correct location.

GeoSpy scored 4958 points.

Overall GeoSpy scored an impressive 23,602 points from a possible 25,000 points. 

I would have loved to have compared GeoSpy's results to those of Google's Bard on TimeGuessr. However Google's entirely reasonable privacy measures mean that you cannot use Bard to identify the locations of any photographs which include people. This means that it refused to identify the locations in any of the images in today's TimeGuessr Daily Challenge.