SIFT is the core of PhotoSynth

PhotoSynth underlying "magic" is using SIFT (Scale Invariant Feature Transform) http://en.wikipedia.org/wiki/Scale-invariant_feature_transform

quoting Seitz
"We use a feature-matching technique called SIFT, developed by David Lowe at the University of British Columbia, that handles very significant differences in lighting, shading, weather, scale, and so forth,"
http://www.spokesmanreview.com/blogs/txt/archive/?postID=1454

David Lowe's Autostitch project.
http://www.cs.ubc.ca/~lowe/home.html
http://www.cs.ubc.ca/~mbrown/autostitch/autostitch.html

more SIFT implementations and source code