When I was asked what my favorite recent website was for the last blog, I immediately thought of Tin Eye (http://www.tineye.com), a reverse photo lookup site, where you upload or link to a picture and it finds matches or near matches, regardless of whether they’ve been altered, cropped, or resized. If you’ll indulge me, I’d like to talk about image searching on the web a little more today.
There have been efforts to make good photo lookups since the early 90s – the metadata tags that most image search engines rely on are not well suited for multimedia. And most engines use them – Google Images (http://www.google.com/imghp?hl=en&tab=wi) and PicSearch (http://www.picsearch.com/), two commonly used examples, both search text about the pictures rather than the image itself. And since that data is entered by people, it often doesn’t include everything that might be relevant. If you’re looking for a picture of a celebrity at an awards show, and they were wearing a gold dress, it’s not likely that typing ‘gold dress’ and the name into Google is going to get you what you want – even though that’s the part that stuck in your memory.
Tin Eye looks at visual data, and how the pixels in the image are related. That doesn’t mean it’s perfect – Tin Eye lacks any kind of facial recognition, for example, that would let you search for a similar pose of the same event – but it’s a big step forward.
Querying by visual example holds a lot of promise. As an artist, I have watched the halting development of such engines with interest. And it’s been building for a while – the ‘Art Museum’ database by Kyoji Hirata and Toshikazu Kato, pioneers in the world of content-based image retrieval, was started in 1992.
One fun example on the web is Retrievr – you can try it at http://labs.systemone.at/retrievr/ – but not much has yet come of the ‘draw your search’ idea. Still, I think this is the eventual wave of the future, and not just for images. One day, I think we’ll be able to hum a few lines into a microphone to search for songs, or mime an action in front of a webcam to search for video. Text-based searching will never go away, because it’s perfect for text. But multimedia? Engines like TinEye show us how much more they can be.
