Polar Rose Blog – A blog about sharing photos & other stuff that matters to us
Computing 3D portraits from 2D photos
At Polar Rose we use computer vision to analyze the content of online photos and recognize the people in them.
Recognizing people in photos is a great challenge because the computer must interpret the photos to extract information above a pure pixel-level, something we humans do easily every day. The main problems when working with general photos are changes in lighting and pose which have great effect on the appearance of a face in a photo.
To cope with this our software creates a virtual 3D portrait of a person so that we can factor out and compensate for these effects. The technology is based on computer vision research for computing 3D data using ordinary photos. Statistical relationships between what can be observed in the 2D photos and how this relates to the possible 3D shape variations gives us a way to get 3D using a single picture. Even if this 3D portrait is just an approximation and not a measurement it is a very good one which adds significant boost to recognition.
Here’s a screenshot of our good friend Ola Ahlvarsson as his 2D picture is converted into a 3D portrait. View the entire screen recording here.
Hey Guys,
We have been waiting for this for a long time, wishing you much sucess in the future…..............Peace!
God Bless
Michael
Excellent, I want to plug this into Second Life!
This is exciting stuff. I wish you great success in your venture. About 10-12 years ago while working for a National Lab (US DOE-owned), I worked on various technologies to identify people, e.g. several biometrics (including human scent—I named it “Scent-sor”). The intended use was primarily for national security. One idea was to take facial recognition algorithms and enhance them for use with 3-D photography. We built a 3D camera based on LADAR (which can “see” in total darkness) but replaced the laser light (which would have damaged eyesight) with LEDs. My brilliant research partner was able to get the resolution down to 2 cm, which allowed for sensing depth of eye sockets, unique configurations of ear shapes, nose length, etc. Unfortunately funding for that project was cut and I left before the work was completed. I have since heard that two Israeli (twin) brothers did something similar, though I never heard of any subsequent commercialization of the product or process. Good luck!
Is this component of your software available as a stand alone app? Thanks.
Unrelated, but interesting: Polarrose mentioned in Michael Arrington’s TechCrunch blog.
Lee: Not at the moment. We might consider the idea further ahead though…
All: Thanks for your comments and encouragement.
-Jan Erik
You guys made the news two months ago – see here.
Glad to see someone is finally tackling this challenge. It’s always been a fascinating subject, though rather difficult. Good luck and hopefully you’ll develop quite a following.
My friend is working on a similar kind of project. Is there a forum or discussion group that he can use to get help from you guys.
Thanks
George
This software is of interest to me because I’m curious to see what it can “tell” about the images of John W. Hillmon, Frederick A. Walters and photos of an unidentified corpse which was variously claimed to be the remains of either Hillmon (Hillman) or Walters.
This case went to the US Supreme Court and was responsible for the establishing of a rule of evidence which is still in effect today in 2006; it is the subject of a forthcoming book by law professor and National Public Radio legal commentator Marianne Wesson.
This hyperlink is to a collection of photos of both men and the corpse:
http://www.rootsweb.com/~ksbarber/hillmon_photos.html
Contemporary news articles about the case can be found on the following page:
http://www.rootsweb.com/~ksbarber/hillman_jw.html
The remains were exhumed in 2006 to attempt to establish the identity of the corpse by DNA tests, but insufficient DNA was recovered from the few remaining bone fragments to permit doing so, so photo analysis is now being used by forensic anthropologist Dennis Van Gerven to identify the corpse.
while excited at the development of your tool. I can’t help but wonder what the social implications would be, particularly with the litigious nature of our world. Wouild people search for images of themselves and claim for misuse oof their image or use without permission?
Other than that I look forward to the plugins release.
interesting; lets be era in Image tech…. I wish you great success in your venture
Useful for historic archival photographic databases. We have 120,000 images online, 10 million in non-scanned.
great work and fantastic features. Keep it up.
Guys
KUDOS and “Major Props” for developing this! One of the future applications could be in trying to find missing persons and long lost relatives who might have their photos posted on “social service sites” like MySpace and Utube. Can’t wait to see this (pun intended)!!
[...] Polar Rose is doing what it can to identify people in photos on the Web. Unlike the previous generation of face recognition software, it turns a photo’s 2D array of dots into what it hopes is an accurate 3D rendering of the face depicted. This gets past the problem of not recognizing yor Aunt Sally’s profile because it only knows what she looks from the front, and maybe of confusing your Aunt Sally with Adolph Hitler because the sun is above her, casting a shadow across her upper lip. [...]
[...] Polar Rose is doing what it can to identify people in photos on the Web. Unlike the previous generation of face recognition software, it turns a photo’s 2D array of dots into what it hopes is an accurate 3D rendering of the face depicted. This gets past the problem of not recognizing yor Aunt Sally’s profile because it only knows what she looks from the front, and maybe of confusing your Aunt Sally with Adolph Hitler because the sun is above her, casting a shadow across her upper lip. [...]
[...] Seems construction of 3d models of faces from 2d pictures is close to becoming mainstream. Polar Rose is doing a similar thing but apparently for face recognition. [...]
Seems like this would be useful for phtographers with digital photos… I would love to scan my PC for photos and then find the photos of my daughter, friend, etc…
[...] “The technology is based on computer vision research for computing 3D data using ordinary photos,” according to the Polar Rose blog. “Statistical relationships between what can be observed in the 2D photos and how this relates to the possible 3D shape variations gives us a way to get 3D using a single picture. Even if this 3D portrait is just an approximation and not a measurement it is a very good one which adds significant boost to recognition.” [...]
bagus!!!, bs minta software nya?
Will this be restriction to only faces?
[...] “The technology is based on computer vision research for computing 3D data using ordinary photos,” according to the Polar Rose blog. “Statistical relationships between what can be observed in the 2D photos and how this relates to the possible 3D shape variations gives us a way to get 3D using a single picture. Even if this 3D portrait is just an approximation and not a measurement it is a very good one which adds significant boost to recognition.” [...]
that’s great…keren banget!! suatu terobosan hebat dalam dunia IT
How can we be a tester???
I cann’t wait too long to enjoy it!
Just wondering how this technology deals with images that have been mirror reversed. Many images on the internet have been reflected does this affect the visual signiture of the face? If so how does the system compensate for this?
I’d like to have a flat photo, in a coppier, print out a 3-D rendition of it.
Can this be done ?
If so, how do I find out about it ?
If not, why not ?
Please respond to twbbs@bellsouth.net
Any news about this software. I would like to test a demo version it is possible.
Thank GB
I’m a professional medical photographer. Any waay this looks to transfer to body parts? I’d like to do some portraits with something like this, too.
[...] how does it work? There’s a layman’s explanation on their blog which is pretty illuminating, and if it all adds up, it (and any other planned sources like it) [...]