Slip me some skin, Bro!

Feng-GUI free attention heatmap service now includes skin detection.
Skin and Face detection have an important role in human visual attention.
It was already demonstrated in eye tracking sessions that face and skin are the first and the second spots that people tends to look at.

Checking “skin detection” will employ a combination of skin models and statistics to identify skin hue color in your image.

Do not use it unless you know that there is skin in the image.


original image

attention heatmap with no skin detection

attention heatmap with skin detection

Feng-GUI attention heatmap accuracy

We’ve been asked several times about the accuracy of Feng-GUI attention heatmap and why is it not reaching 100%.
Feng-GUI attention heatmap algorithm, which we call the ViewFinder , is a composition of algorithms aggregating over 30 years of research. Research from the fields of neuro-science studies, visual attention, statistics of eye-tracking sessions, perception and cognition of humans.
We improve the ViewFinder algorithm in order to achieve the highest accuracy, but is it possible to get to 100% ?

Let us answer by asking a question:
How accurate is the "Magic Wand" in Photoshop? (also referred as the “Fuzzy select” in Gimp)



The Magic Wand tool is designed to select areas of the image by finding edges of image segments.



Part of the answer to accuracy can be found at the "image segmentation and boundary detection" benchmark conducted by Berkley.
It’s a 1000 photos test suite for algorithms that try to find the boundaries of elements inside pictures.
In the test suite results you can see how real people and computer algorithms, marked the boundaries of segments inside photos and how accurate they are.
The current score is:
Real people: 79%
Machines algorithms: 70%


original image

edge detection algorithm

edge detection by human

http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/
results:
http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/bench/html/algorithms.html

You can see that edge detection algorithms are not 100% accurate and will never be considering the hardware used be personal computers.
To solve this limitation, GIMP supports six different selection tools. In order of appearance in the tool box they are: rectangle, ellipse, free, fuzzy, bezier, and intelligent.
Each tool has its advantages and disadvantages--no one tool is clearly superior to the others. In general, a combination of tools proves to be the optimal strategy for complex selection tasks.

We conducted a comparison benchmark between Feng-GUI results and eye-tracking results of 40 people, 90 images, 15 seconds per image.
http://www.feng-gui.com/faq.htm#quality
Comparing the regions of interests (ROI) in the images, results with more than 70% of similarities.
If you compare Feng-GUI results to random groups of 10 people (out of the 40 people), for 5 seconds, the similarities go over 80%.

To enable designers and advertisers integrate Feng-GUI’s attention heatmap into their work process, we keep on improving the ViewFinder algorithm while maintaining two important requirements: Accuracy as well as Speed.

happy new year!

API documentation preview

We recently posted a preliminary and small version of documentation for Feng-GUI Services API.
We are very eager to receive feedback from members of the Beta community as they access this documentation over the next several weeks so we can use that feedback to improve our first API version to be released in October.

The API documentation can be found at
http://service.feng-gui.com/api.htm

The API is implemented as XML over HTTP (XML-RPC).
Send an XML Request and receives an XML response.
The XML-RPC Server Endpoint URL is:
http://service.feng-gui.com/xmlrpc/api.ashx
The Server Endpoint also contains automatic documentation of the API.

The guest credentials are:
username:guest
password:guest

Right now, the service API methods are partially-functional and your images appears at Feng-GUI.com MRU list.

Brain Scanner Can Tell What You're Looking At

Scientists have developed a computer model that predicts the brain patterns elicited by looking at different images -- a possible first step on the path to mind reading.

Image: University of California at Berkeley
read more at http://www.wired.com/science/discoveries/news/2008/03/mri_vision

DisrupTV convention

Feng-GUI for Videos will be demonstrate at DisrupTV convention (23 May). We will show how to predict people's attention in videos, commercials, and games.

Disrupt.TV is an un-conference intents to bring together Israeli and global Technology, content and creative talents that are taking part in creating the future of TV and media for sharing knowledge, exploring the opportunities and to spark new ideas about how TV and Video will change us and we will change TV.
It is aimed for people who are in the conjunction of Media, Content, Creative and Technology.

DLD 2008 - Digital gets Physical and Social

Our made man, Ohad Eder Pressman reached to Speaker level at DLD* 2008, lecturing about connecting the Physical and Digital worlds.

Digital gets Physical session include Kati London, Julian Bleecker (NearFuture), Neri Oxman, Ohad Eder-Pressman and hosted by Kevin Slavin



watch the full video at sevenload

http://de.sevenload.com/videos/kHp4S78/DLD08-Day-3-Digital-gets-physical/sliderPage/2



DLD conference is a blast of sharing ideas, thoughts and vision.
It was a great opportunity to meet GarageGeeks friends from all over the world:

Pabls Holman the international hacker, Digital Dan Dubno , check out their great presentation at DLD*

http://en.sevenload.com/videos/bAoG482/DLD08-Day-2-Educate-experiment/sliderPage/2

JB Jean-Baptiste Labrune who creates our next generation of interfaces, Régine Debatty editor of the art/tech blog http://www.we-make-money-not-art.com



Michael Reinboth from Compost records, Munich, kindly had a meeting with us and we discussed about various ways to integrate the GarageGeeks activities with the music industry in Germany.

Music Workshop - DIY Toy Music Controllers

At this music workshop, we are creating modified toys music controllers that send their MIDI or Audio information to central audio and visual stations.

Reflections from this workshop

photos
http://www.flickr.com/photos/rafael_mizrahi/sets/72157603505744930/

videos
http://www.youtube.com/watch?v=E4z5LKpeLZU
http://www.youtube.com/watch?v=ri4vpqTnP9A
http://www.youtube.com/watch?v=Gd1H2T5uUqQ
http://www.youtube.com/watch?v=5raa-XiAFAo
http://www.youtube.com/watch?v=S42jX6-n9K4
http://www.youtube.com/watch?v=Gd1H2T5uUqQ

Date: 3 January 2008
Hosts: Rafael Mizrahi, Eyal ‘Person’ Sachar and Shira Miasnik

Schedule19:00 Garage Open: Prepare your "Basta".
20:00 Workshop agenda: Enough with Baby Einstein, its Baby Clubber time.
20:30 workshop starts



Who should come?
* Experienced music technicians and interactive designer.
* The event is open for everyone. You can join others (see /write in the wiki http://wiki.garagegeeks.org) by helping take apart and stuff the musical toys with gadgets.

The Garage will be wired with MIDI, and Audio cables. Each station can choose from several methods of communication to the central stations Hub. Your Toy output can be connected to the central stations or to your Laptop, and from there, to the central stations.



Ingredients: bring one or more of the followings:

1. A Toy (or some device that you wish to convert into a controller)


2. Sensors http://www.sensorwiki.org/index.php/Sensors


3. Sensor interface board http://www.sensorwiki.org/index.php/Sensor_interfaces


4. Parts - Assorted switches, potentiometers and other parts
   http://www.anti-theory.com/soundart/circuitbend/cb04.html


5. Laptop


6. Cables:MIDI, LAN, Audio and converter accessories.


7. Tools: soldering iron, glue, etc’
   http://www.anti-theory.com/soundart/circuitbend/cb03.html


8. USB Keyboard to take apart.

How to participate with low budget:
Here is a list of ingredients to build a Music Toy Controller for anyone who wish to participate but is not so skilled or equipped. A Toy wired with Keys from a USB keyboard sending keystrokes to a utility (MouseTrap) which translates them into MIDI messages, mapped by MIDI-OX into Ableton Live.
* Toy
* USB Keyboard to take apart (also a mouse can be used).
* Laptop
* Ableton Live Demo http://www.ableton.com
* MIDI utilities:
** Keyboard/Mouse to MIDI: MouseTrap http://www.humatic.de/htools/MouseTrap.htm
** MIDI virtual device: MIDI-OX http://www.midiox.com/?http://www.midiox.com/moxdown.htm
* switches, knobs, sliders to hook on the Toy
* wires, solder, glue gun.
* anything else you wish to do with your lovely musical Toy.



Examples of participation (there can be more variations):

1. Toy+MIDI board->central station.


2. Toy+USB Keyboard->laptop->MIDI device or Audio->central station.


3. Toy+phidgets->laptop->Audio->central station.


4. Toy->laptop->MAX/MSP with MIDI module->MIDI device->central station.


5. Toy+ Arduino with MIDI module->laptop or central station.

Recommended boards:

1. MIDISense (MIDI) http://www.ladyada.net/make/midisense


2. MIDITron (MIDI) http://eroktronix.com


3. Arduino (Multi Purpose board USB) http://www.arduino.cc


4. Phidgets (Sensors to USB) http://www.phidgets.com/index.php?module=pncommerce&func=itemview&IID=85


5. Pocket Electronic (MIDI) http://www.doepfer.de/pe.htm


6. Make Controller Kit (USB and OSC) http://www.makingthings.com/products/KIT-MAKE-CTRL

notes: Don’t buy a board without understanding what you are buying and how to operate it.
The board kits are pretty easy to build, but prepare them at home because it takes several hours to assemble a 50$ kit.



Home Preparations:

1. Get a Toy, disassemble it at home and look for ways to attach the sensors or potentiometers onto it.


2. Decide how the toy will function? (MIDI, USB, Phidget, OSC, etc')


3. Prepare your board and test it with the sensors or potentiometers.


4. A handful of parts (knobs, slides, etc') that you can hook on the toy, bring extra, maybe others can use them.
   tour some electronic shops for such parts, or just take them from electronic junk you still didn't throw away.

Links

1. List of electronic shops which also sell sensors http://www.garagegeeks.org/blog/?p=78


2. List of music shops that sell music instruments http://www.act.co.il/index.php?action=shops

Reading material

MIDI
Essential MIDI toolbox
Use to monitor midi messages, send midi from your keyboard, map midi devices and more.
MIDI-OX
http://www.midiox.com/?http://www.midiox.com/moxdown.htm
MIDI-Yoke
http://www.midiox.com/index.htm?http://www.midiox.com/myoke.htm

OSC – Open Sound Control
OSC is meant to supersede the MIDI While MIDI requires a MIDI USB device, OSC communicate via a standard home or studio network (TCP/IP, Ethernet) or via the internet.

http://opensoundcontrol.org
http://www.cnmat.berkeley.edu/OpenSoundControl
http://en.wikipedia.org/wiki/OpenSound_Control

OSC to MIDI utilities
OCCAM (MAC OS) http://www.mat.ucsb.edu/~c.ramakr/illposed/occam.html
OSCulator (MAC OS) http://www.osculator.net/wiki

USB keyboard or Mouse to MIDI or OSC

MouseTrap - MouseTrap lets mouse and keyboard act as general purpose Midi / OSC control hardware.
http://www.humatic.de/htools/MouseTrap.htm

More Links for Muse

http://www.act.co.il
http://createdigitalmusic.com
http://createdigitalmotion.com

GuitarHeroNoid at LeWeb3 France

Kathy brooks from Six Apart and Loic Le Meur invited the GuitarHeroNoid to perform at LeWeb3 conference France.
Our man Tal Chalozin, the heronoid Puppeteer, went there and presented the garageGeeks activities.


Here is a video of the show

GuitarHeroNoid v2 at VON 2007 Boston

Mr Jeff Pulver invited the human-size robot playing guitar hero aka the GuitarHeroNoid to preform in VON Boston 2007 (Video Over the Net) conference in Boston.

Tal Chalozin and Yuval Tal have improved the robot version 2, adding some more features like the Controller-Controller and the Penis Guitar Holder.



Pictures from VON

SIFT is the core of PhotoSynth

PhotoSynth underlying "magic" is using SIFT (Scale Invariant Feature Transform) http://en.wikipedia.org/wiki/Scale-invariant_feature_transform

quoting Seitz
"We use a feature-matching technique called SIFT, developed by David Lowe at the University of British Columbia, that handles very significant differences in lighting, shading, weather, scale, and so forth,"
http://www.spokesmanreview.com/blogs/txt/archive/?postID=1454

David Lowe's Autostitch project.
http://www.cs.ubc.ca/~lowe/home.html
http://www.cs.ubc.ca/~mbrown/autostitch/autostitch.html

more SIFT implementations and source code

Google Images Search for faces

Google didn't buy riya and bought neven vision adding a face recognition to the Images Search service.
For example you can search for Paris in general
or search for Paris Hilton
Just add the imgtype=face to the search query.

Google eye-counting video camera

Google unveiled an eye-counting video camera that could enable the company to extend its highly successful online business model to brick-and-mortar advertisers.

The Eyebox was developed by Xuuk Inc.(Kingston, Ontario).

Using its PageRank technology, Google (Mountain View, Calif.) has been able to collect revenue from advertisers based on the number of ads on which people are clicking.

Now with the Eyebox, Google can determine which billboards or products people are looking at (32 feet range) in mall corridors or on store shelves, and count them in the same manner that Google counts clicks for online ads.

tobii new generation eye tracking

Tobii announced its launch of a new generation eye tracking hardware and analysis software.

Fundamental technology advances and new tools facilitate use of eye tracking and add substantial new values to usability and user experience studies. Making up a complete lab solution, the new products will be presented at CHI in San Jose CA, April 29.

SpikeNet human visual system

SpikeNet uses processing algorithms that are directly inspired by the strategies used by the human visual system which outperforms even the most sophisticated machine vision systems. Indeed, the human visual system is able to analyse a complex scene in a fraction of a second.
 

Foveon X3 Technology

A digital camera should see color the way the human eye does.

"It's easy to have a complicated idea," Carver Mead used to tell his students at Caltech. "It's very, very hard to have a simple idea."

The genius of Carver Mead is that over the past 40 years, he has had many simple ideas. More than 50 of them have been granted patents, and many involved him in the start-up of at least 20 companies, including Intel. Without the special transistors he invented, cell phones, fiber-optic networks, and satellite communications would not be ubiquitous. Last year, high-tech high priest George Gilder called him "the most important practical scientist of the late 20th century."
"Nobody," Bill Gates once said, "ignores Carver Mead."



X3 is the latest and most innovative product from Foveon Inc., the Silicon Valley digital-imaging company that Mead, 68, founded in 1997. Named for the fovea centralis—the part of the human retina where vision is sharpest and most color perception is located Foveon took as its mission another radically simple idea Mead loves: "Use all the light."

Biologically Inspired Vision Systems

Neuroscientists at MIT have developed a computer model that mimics the human vision system to accurately detect and recognize objects in a busy street scene, such as cars and motorcycles.

"Maybe we shouldn't be surprised," says David Lowe, a computer vision and object recognition expert at the University of British Colombia in Vancouver. "Human vision is vastly better at recognition than any of our current computer systems, so any hints of how to proceed from biology are likely to be very useful."

The article:
http://www.technologyreview.com/Infotech/18210/

The lab:
http://web.mit.edu/bcs/research/

face - before recognize, detect

prior step to face recognition, the accurate face detection of human faces in arbitrary scenes, is the most important process involved.
http://www.facedetection.com

Intel OpenCV face detection license

Intel OpenCV is a great free image processing library.(take a look at its license)

but regards face detection, it might be not that free.
looking into $OpenCV\data\ which contains the HAAR cascades, you may find a readme.txt file which goes like this:

This folder contains various data that is used by cv libraries and/or demo applications.
----------------------------------------------------------------------------------------
haarcascades - the folder contains trained classifiers for detecting objects
of a particular type, e.g. faces (frontal, profile), pedestrians etc.
Some of the classifiers have a special license - please,
look into the files for details.

Now lets have some background
(thanks to Sébastien Marcel at idiap)

The face detection method implemented in OpenCV by Rainer Lienhart is very similar to the one published and patented by Paul Viola and Michael Jones (patent 1 and patent 2), namely called Viola-Jones face detection method.

More precisely, Rainer Lienhart proposed an extension of Viola and Jones work which consists essentially in additional haar-like features (center, tilted) and the use of a tree-based classifiers instead of stump-classifiers in the cascade. Indeed, even if the excellent implementation of Lienhart is available in OpenCV, it seems that the classifiers (check carefully the face detection XML models) in the cascade are not trees but stumps and that they don't contain any "tilted" haar-like features. As a consequence, those "default" models implement a solution very close from Viola and Jones.

Therefore, to differ really from the Viola-Jones patent, you will need to re-train yourself a cascade with tree-based classifiers, and possibly with tilted features.

Google Portrait

Google Portrait is a demonstration system of IDIAP and Torchvision face detection technology. It is for personal and non-commercial use only. We acknowledge Google for providing the image indexing and retrieval service and we garantee that we don't perform any automated querying. Indeed, the query made by a user is equivalent to a query made directly on Google Image and the face detection processing is done "on-the-fly". Please note also that we don't make use of the page ranking information, but only the url of images indexed and retrieved by Google.

Watch out Riya! :)

check it out at: http://www.idiap.ch/googleportrait/

for example: A search for Michael Arrington
http://www.idiap.ch/googleportrait/index.cgi?query=Michael+Arrington

Object Detection mantra by HCK

found this nice object detection mantra from a user called HCK:

Poor evaluation results + good training results + small number of weak classifiers + much training data = too uniform data.

Poor evaluation results + better training results + large number of weak classifiers + much training data = data has too much variation.

Poor evaluation results + poor training results + large number of weak classifiers + much training data = weak classifiers are too weak.

Mom, Do I have a big Nouse?

Nouse™ (Nose as Mouse) Interfaces. Operate your computer with your nose as with a mouse (or a pen) or a joystick (or a pointer).
comming up next, Doubleblink™ event, to detect a facial expression event, such as an intentional eye blink.
http://www.cv.iit.nrc.ca/research/Nouse/
http://synapse.vit.iit.nrc.ca/Nouse/index2.html

Image-Based Face Recognition Algorithms

PCA, ICA, LDA, EP, EBGM, Kernel Methods, Trace Transform,
AAM, 3-D Morphable Model, 3-D Face Recognition,
Bayesian Framework, SVM, HMM, Boosting & Ensemble,
Algorithms Comparisons

you name it. you got it all here.
http://www.face-rec.org/algorithms/

and ... also at Intel OpenCV
http://opencvlibrary.sourceforge.net/FaceRecognition

you wanna know, which one of them is the best?
check out this comparison
http://www.cs.colostate.edu/evalfacerec/

panoramic viewfinder and much more

patrick baudisch's is a research scientist in the field of human-computer interaction at the Adaptive Systems and Interaction Research Group at Microsoft Research.
He has been working on some projects that will definitely will inspire your mind.

http://www.patrickbaudisch.com/projects/panoramicviewfinder/index.html
http://www.patrickbaudisch.com/projects/index.html

Music and Neuroimaging Laboratory

Reveal the perceptual and cognitive aspects of music at the Music and Neuroimaging Laboratory
http://www.musicianbrain.com/

Can't dance? Test to see if you're rhythmdeaf!
http://tonometric.com/rhythmdeaf/

Test your musical skills in 6 minutes!
http://jakemandell.com/tonedeaf/

face recognition algorithms

• Eigenfaces or Principal Component Analysis method (PCA)
• Fisherfaces or Linear Discriminant Analysis method
• Kernel Methods
• 3D face recognition methods
• Gabor Wavelets method
• Hidden Markov Models
  

Sample face Databases

Yale Face Database
Yale Face Database B
FERET database

Text Information Extraction algorithms

Author -Year -Approach -Features

• Ohya et al. -1994 -Adaptive thresholding and relaxation operations -Color, scene text (train, signboard,skew and curved), localization and recognition '
• Lee and Kankanhalli -1995 -Coarse search using edge information, followedby connected component (CC) generation -Scene text (cargo container),localization and recognition
• Smith and Kanade -1995 -3×3 filter seeking vertical edges -Caption text, localization
• Zhong et al. -1995 -CC-based method after color reduction, local spatialvariance-based method, and hybrid method -Scene text (CD covers), localization
• Yeo and Liu -1996 -Localization based on large inter-frame difference in MPEG compressed image -Caption text, localization
• Shim et al. -1998 -Gray level difference between pairs of pixels -Caption text, localization
• Jain and Yu -1998 -CC-based method after multi-valued color image decomposition -Color (book cover, Web image, video frame), localization
• Sato et al. -1998 -Smith and Kanade’s localization method and recognition-based character extraction -Recognition
• Chun et al. -1999 -Filtering using neural network after FFT -Caption text, localization
• Antani et al. -1999 -Multiple algorithms in functional parallelism -Scene text, recognition
• Messelodi and Modena -1999 -CC generation, followed by text line selection usingdivisive hierarchical clustering procedure -Scene images (book covers, slanted)localization
• Wu et al. -1999 -Localization based on multi-scale texture segmentation-Video and scene images (newspaper, advertisement) recognition
• Hasan and Karam -2000 -Morphological approach -Scene text, localization
• Li et al. -2000 -Wavelet-based feature extraction and neural networkfor texture analysis -Scene text (slanted), localization, enhancement, and tracking
• Lim et al. -2000 -Text detection and localization using DCT coefficient and macroblock type information -Caption text,MPEG compressed video, localization
• Zhong et al. -2000 -Texture analysis in DCT compressed domain -Caption text, JPEG and I-frames of MPEG, localization
• Jung -2001 -Gabor filter-like multi-layer perceptron for texture analysis -Color, caption text, localization
• Chen et al. -2001 -Text detection in edge-enhanced image -Caption text, localization and recognition
• Strouthopoulos et al. -2002 -Page layout analysis after adaptive color reduction -Color document image, localization

2D image to 3D

Automatically constructs simple "pop-up" 3D models from 2D images.
http://www.cs.cmu.edu/~dhoiem/projects/popup/index.html

Six powerful segmentation algorithms

Meanshift
meanShift Segmentation
MsRobustApproach.pdf


J Segmentation (JSEG)
http://vision.ece.ucsb.edu/segmentation/jseg/
01PAMIJseg.pdf


Efficient Graph-Based Image Segmentation (EGBIS)
http://people.cs.uchicago.edu/~pff/segment/
http://www.cs.cornell.edu/~dph/papers/seg-ijcv.pdf


Color Structure Code (CSC)
http://www.uni-koblenz.de/~lb/lb_downloads/
csc.pdf


Statistical Merging Segmentation (SRM)
http://www.charlesriver.com/visualcomputing/
http://dx.doi.org/10.1109/TPAMI.2004.110


Region Growing
documentation