Seattle, Washington - A new personalized image search engine synthesizes versions of an input photo (left) with internet queries such as “curly hair” (top row), in “India” (2nd row), and in “1930” (3rd row).Ira Kemelmacher-Shlizerman, University of Washington.
When we go to the hair stylist, we can browse magazines with pictures of models and point to a photo we’d like to try. Actors change appearances all the time to fit a role. Missing people are often disguised by changing their hair color and style.
But how can we predict if an appearance change will look good without physically trying it? Or explore what missing children might look like if their appearance is changed?
A new personalized image search engine developed by a University of Washington computer vision researcher called Dreambit lets a person imagine how they would look a with different a hairstyle or color, or in a different time period, age, country or anything else that can be queried in an image search engine.
After uploading an input photo, you type in a search term - such as “curly hair,” “India” or “1930s.” The software’s algorithms mine Internet photo collections for similar images in that category and seamlessly map the person’s face onto the results.
Initial results will be presented July 25 at SIGGRAPH 2016, the world’s largest annual conference on computer graphics and interactive techniques. Plans are underway to make the system publicly available later this year.
Dreambit draws on previous research conducted at the UW and elsewhere in facial processing, recognition, three-dimensional reconstruction and age progression, combining those algorithms in a unique way to create the blended images.
The new software can also help show what a missing child or person evading the law might look like if their appearance has been purposefully disguised, or even how they would look at an advanced age if years have passed.
Developer Ira Kemelmacher-Shlizerman, UW assistant professor of computer science and engineering, and her team previously developed automated age progression software that focused only on a person’s face. The new system adds varied hairstyle options and other contextual elements.
These new features enable one to imagine what a child might look five or 10 years into the future under different circumstances — with red hair, curly hair, black hair or even a shaved head.
“It’s hard to recognize someone by just looking at a face, because we as humans are so biased towards hairstyles and hair colors,” said Kemelmacher-Shlizerman. “With missing children, people often dye their hair or change the style so age-progressing just their face isn’t enough. This is a first step in trying to imagine how a missing person’s appearance might change over time.”
Another potential application is to envision how a certain actor or actress might appear in a role. For example, the system can marry internet photographs of the actress Cate Blanchett and Bob Dylan to predict how she would appear playing the Dylan role in the movie “I’m Not There.”
“This is a way to try on different looks or personas without actually changing your physical appearance,” said Kemelmacher-Shlizerman, who co-leads the UW Graphics and Imaging Laboratory (GRAIL). “While imagining what you’d look like with a new hairstyle is mind blowing, it also lets you experiment with creative imaginative scenarios.”
The software system analyzes the input photo and searches for a subset of internet photographs that fall into the desired category but also match the original photo’s face shape, pose and expression.
Its ability to accurately and automatically synthesize two photographs stems from the combination of algorithms that Kemelmacher-Shlizerman assembled, as well as the sheer volume of photos available on the internet.
“The key idea is to find a doppelgänger set — people who look similar enough to you that you can copy certain elements of their appearance,” said Kemelmacher-Shlizerman. “And because the system has hundreds of thousands of photos to choose from, the matching results are spellbinding.”
