Interactive Camera-Projector System
 |
 |
Contents |
[edit] Camera-Projector System
[edit] Application
mando (or kmando) is a human computer interface using a camera and a projector. It was developed by Juan Roldan (prototype), Ushakiran Soutapalli (user interface), Julien Faucher (calibration), Jan Wedekind (technical supervision), and Claudia Amico (artwork) at the Mobile Machines and Vision Laboratory (MMVL) at Sheffield Hallam University, UK.
The software makes use of a Video for Linux version 1 (V4L) camera such as the Logitech Quickcam Pro 4000 640x480 webcam (see exploits.org/v4l and qbik.ch for compatible USB devices). The camera is calibrated against a standard projector screen and then used to determine the position of physical pointer (e.g. a pen) which is then used to virtually move the X11 pointer. Point-and-click functionality has also been implemented. The software now also supports IIDC/DCAM compatible firewire digital cameras such as the Unibrain Fire-i (see Damien Douxchamps list for cameras with IIDC=yes/1.30/1.31). Note that Video for Linux version 2 (V4L2) currently is not supported by mando.
The required parts of the Mimas real-time computer vision toolkit are included in the package.
[edit] How it works
[edit] Detection
The detection is a simple colour-segmentation using a thresholded colour-histogram. Therefore you need to use a marker pen or some other object with a distinct colour. The colour should not appear on the screen itself. If the colour of the pen will change due to different illumination conditions, you need to capture these colours when taking the reference image.
[edit] Mouse clicks
The standard deviation of a few recent mouse-positions is used as a criterion, wether the mouse is stationary or moving. If the mouse is stationary, a mouse click is issued.
[edit] Design
 |
 |
[edit] Changelog
2008-02-07 version 1.6
- Added more confusing array operator methods to keep up with the GCC freaks.
2007-03-27 version 1.5
- Adding support for firewire digital camera
- image_v4linput.cc: Accepting YUV422 (assuming it is same as UYVY).
- Wizard has new combo-box for selecting video mode (PAL/NTSC/SECAM/AUTO)
- image_v4linput.cc: Not preferring RGB support of camera. Prefer UYVY or YUV420p instead.
2007-04-14 version 1.4
- Added spin-box for selecting camera channel.
- GUI now has sliders for adjusting brightness, contrast, and others.
2007-03-27 version 1.3
- Added some explanation about capturing reference image.
- Distorted e-mails because of spam.
2007-02-12 version 1.2
- configure.ac: Configure script must not use Qt3-moc.
2007-02-11 version 1.1
- Makefile.am: Added moc-, qrc-, and uic-files to clean-rule.
- configure.ac: Call "make clean" to delete built sources.
2007-02-02 version 1.0
- Applications seems to run stable now -> releasing version 1.0
2006-10-20 version 0.6
- Minor fixes on configure-script
2006-10-20 version 0.5
- Fixed the check for gfortran (using AC_PROG_FC now). Thanks to Steven G. Johnson!
# On the older Mandriva version you may have to configure using ./configure FLIBS="-lgfortran"
# On openSuSE 10.0 libgfortranbegin.a was compiled without "-fPIC". You need to configure using ./configure F77=gfortran FLIBS="-lgfortran"
2006-10-17 version 0.4
- Added support for UYVY-cameras.
2006-10-16 version 0.3
- Video display can be zoomed using the mouse-wheel.
- Display error-messages when accessing camera fails.
2006-10-13 version 0.2
- Removed dependency on ImageMagick.
[edit] Downloads
Download the latest release mando-*.tar.bz2 from the mando download section at Sourceforge. For your convenience the required parts of Mimas are included in the package. To build and run the package do
tar xjf mando-*.tar.bz2 cd mando-* ./configure make ./mando
Donnie Bhayangkara has built a package for openSuSE 10.3 which can be downloaded here http://software.opensuse.org/search?baseproject=openSUSE%3A10.3&p=1&q=mando
[edit] Git Repository
The git repository can be accessed as follows
git clone git://mando.git.sourceforge.net/gitroot/mando mando
[edit] Gallery
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
[edit] Possible Future Work
- Port application to Ruby using HornetsEye and Qt4-Ruby
- Port application to OpenMoko+Webcam
- Support for libdc1394 version 2
- Support for V4L version 2
- Test with cheap IR-camera
- Test with 90 fps camera
- Gray code calibration
- use histogram inversion instead of colour histograms
- allow multi-touch (detect multiple pointers) by using local minima of boundary curve
You can help by donating some of your spare time and expertise ;)
[edit] See Also
[edit] External Links
- Interative Camera-Projector System
- References
- Other projects
- Edward Tse - Interactions Inventor
- Multi-Touch Interaction Research
- Magic board project by IIHM
- IBM Research on interactive projector
- Finger tracking with laser scanner
- LaserGo allows playing Go using a laser-pointer
- E-Chalk project
- MIT iCampus: Magic Paper
- Occlusion-free projection with multiple projectors
- Detect and neutralise digital cameras
- Video of Microsoft Surface at popularmechanics.com and at at youtube.com
- Calibration for a multiple projector mosaic
- Andy Wilson's human computer interface (video)
- Navisis USB touch screen extension (technique)
- projection keyboards
- Johnny Chung Lee - Wiimote whiteboard (also see Sourceforge project). The Wiimote has a 90 fps infrared camera and a microcontroller for maxima detection.
- HandVu hand gesture recognition
- Multi-Touch Display by Pascal Schmitt
- Virtual Laser Keyboard for mobile phones!
- Compaq self-calibrating projector
- L.A.S.E.R. Graffiti (video)
- Publicity
