The science of optical shoe recognition

This week’s episode of The Big Bang Theory, The Bus Pants Utilization, revolved around the guys coming up with an iPhone app.  It sounded cool for certain people in certain situations — it basically amounted to Wolfram Alpha with optical character recognition, so you didn’t have to type in the equations to get them solved.  A side plot involved Sheldon being banished to help Penny with her idea for an iPhone app — one that can perform shoe recognition.  Something about that sounded vaguely familiar, and by the end of the episode, I remembered that Amazon has an app that allows you to take a photo of something which then gets processed by their Mechanical Turk engine for possible matches.  My understanding is that it works really well for things with text or that can otherwise be solidly identified (DVD box, movie poster, book cover) and not so well for clothing.

I thought I’d give Amazon’s product recognition a try to see how close the world was to Sheldon and Penny’s iPhone app.  I started with a picture of some very unique shoes.  They are a bronze-colored Steve Madden tennis shoe with rhinestones along the sides.  Kim like shoes, really likes these particular shoes, and I figured they were unique enough to prevent confusion and make a positive match.  I took a photo similar to the following.  (I actually had to re-take this photo after everything was said and done because the Amazon app only saved a thumbnail upon completion, not the full-size picture.)  I deliberately made sure the inside Steve Madden label was not visible from the camera angle.

Maybe the bronze/gold coloring does not show as well as it could in the photo, but the overall style and rhinestones are certainly visible.  The Amazon app uploaded this to the site and I went about my business.  Twenty minutes later, I got notification of a match:

Yep, there is still a ways to go with optical shoe recognition.

UPDATE: As thebruce asked about in the comments, I tried again with Google Goggles, which resulted in a whole bunch of nothing:

Posted in: Dear Diary iPhone Television

Published by

Brian Enigma

Brian Enigma is a Portlander, manipulator of atoms & bits, minor-league blogger, and all-around great guy. He typically writes about the interesting “maker” projects he's working on, but sometimes veers off into puzzles, software, games, local news, and current events.

2 thoughts on “The science of optical shoe recognition”

Leave a Reply

Your email address will not be published. Required fields are marked *