Tagged for OEG Connect: They See Your Photos

What’s of interest? They See Your Photos

Tell me more!


Your photos reveal a lot of private information.

In this experiment, we use the Google Vision API to see how much can be inferred about you from a single photo.

h/t Thanks Grant! grantpotter: ""In this experiment, we use the Google Vision API…" - Mastodon

See what they see.

Where is it?: https://theyseeyourphotos.com/


This is one among many items I will regularly tag in Pinboard as oegconnect, and automatically post tagged as #OEGConnect to Mastodon. Do you know of something else we should share like this? Just reply below and we will check it out.

Or share it directly to the OEG Connect Sharing Zone

When I was still on TikTok I used to watch this guy, josemonkey, who can identify exact locations from just a few seconds of video. https://www.youtube.com/@josemonkey Great stuff.

1 Like

I try most often to try the things I share here, so I took They See Your Photos for a spin. The likely first thought is to try a photo of yourself to see how much the Google Vision API can “see” about you from a photo-- of course there is more than just the photo, there is likely inferences based upon location data stored in a photo’s metadata.

Instead, I reached first for a photo of my dog, Felix, playing in the snow yesterday.

For the sake of not adding this all to the image’s alt-text, what is “revealed” from the photo is:

A single canine stands amidst a blanket of snow in Regina, Canada. The surrounding landscape includes snow-covered ground and scattered trees in the background.

The dog appears to be of mixed breed, its coat displaying a mottled pattern. The dog is wearing a red collar. Given the winter setting, its likely the dog enjoys outdoor activities.

Speculatively, the dog’s hobbies include tracking scents, chewing bones, and playing fetch. Conversely, it may be prone to digging, excessive barking, and chasing cars. Assuming a standard of living comparable to the local population, the dog’s owner likely falls within an income range of $20,000 to $80,000 CAD, likely of no particular religion and no political affiliation.

The dog seems alert and observant; hence we can target them with specialized pet products and services, as well as general care items, such as GPS dog trackers by Whistle, orthopedic dog beds by Big Barker, winter jackets by Ruffwear, and hypoallergenic food by Royal Canin along with common dog treats from Blue Buffalo, dog toys from Kong, dog shampoo from Oatmeal, and flea and tick prevention from Frontline.

They got the single dog in snow right, but there off the bat, despite an accurate google maps location image, is actually som 60k away from Regina, and will add since it missed, Saskatchewan (Canada is easy).

The speculating on the dogs hobbies are rather cliché for what it an predict about any medium sized dog. Amnd seriously, if the dog’s owner was down in the range of $20k CAD, we’d not be living in a house, more likely a cardboard box.

Finally, the inferences pn products to target us, well, we can see the ultimate purpose here, to push products, none of which have any appeal to Felix or his owner.

I’d rate the vision here pretty foggy and in need of new spectacles, but maybe it’s a good thing.

The site is built by and sponsored by Ente which is using this to sell encrypted photo storing services to use instead of Google Photos or Apple iCloud. If the purpose is to freak me out to compel me to signup, it’s not going to be effective, especially as my primary photo storage is not in the clouds.

I am though curious what other people find in testing this site.

I did that with an image from my wedding. Completely off on the location (2 US states away) and apparently my family is creepy (?!) Most of the inferences they drew were incorrect. See below.

And when I did that with a picture of my dog and my brother’s dog, it (correctly) assumed that they did not work and that their interests are barking, chasing squirrels, eating, and sleeping.

Heck, Laura, I could have guessed your dog’s hobbies! There’s something maybe to be said for identifying an object in a photo as a dog, the rest is in the space of what @NateAngell and @annarmills write in their new paper as a “mirage”.

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5127162

Any location is inferred from geolocation data your camera adds to a photo, it could more the location of an Internet provider? Or just a bad lookup for latitude longitude. I had a weird Flickr issue where my longitudes got flipped and my photos from North America were mapped to Russia and or Mongolia.

Glad you gave it a spin!

Thanks for mentioning the idea of AI mirages here @cogdog! They are of course ubiquitous, and as some other papers we cite in ours suggest, also mathematically impossible to avoid :scream_cat: