Revision as of 15:50, 19 December 2007

Developed by Juan Roldan, ERASMUS exchange student from Cordoba, Spain	Ushakiran Soutapalli, MSc student at Sheffield Hallam worked on the graphical user-interface
Playing Reversi against the computer (available as Google-Video or 5.54 MByte DivX-video)

Camera-Projector System

Application

A software for camera-projector interaction was developed by Juan Roldan (prototype), Ushakiran Soutapalli (user interface), Julien Faucher (calibration), Jan Wedekind (technical supervision), and Claudia Amico (artwork) at the Microsystem & Machine Vision Laboratory (MMVL) at Sheffield Hallam University, UK.

The software makes use of a Logitech Quickcam Pro 4000 webcam with a resolution of 640x480. The camera is calibrated against a standard projector screen and then used to determine the position of physical pointer (e.g. a pen) which is then used to virtually move the X11 pointer. Point-and-click functionality has also been implemented. Note that video for linux version 2 (V4L2) is not yet supported by kmando.

The vision system uses the Mimas real-time computer vision toolkit developed by the MMVL.

How it works

Detection

The detection is a simple colour-segmentation using a thresholded colour-histogram. Therefore you need to use a marker pen or some other object with a distinct colour. The colour should not appear on the screen itself. If the colour of the pen will change due to different illumination conditions, you need to capture these colours when taking the reference image.

Mouse clicks

The standard deviation of a few recent mouse-positions is used as a criterion, wether the mouse is stationary or moving. If the mouse is stationary, a mouse click is issued.

Design

UML-diagram of static structure

UML-diagram showing collaboration of classes

Changelog

2007-03-27 Jan Wedekind <jan at wedesoft.de> 1.3

• Added some explanation about capturing reference image.
• Distorted e-mails because of spam.

2007-02-12 Jan Wedekind 1.2

• configure.ac: Configure script must not use Qt3-moc.

2007-02-11 Jan Wedekind <jan at wedesoft.de> 1.1

• Makefile.am: Added moc-, qrc-, and uic-files to clean-rule.
• configure.ac: Call "make clean" to delete built sources.

2007-02-02 Jan Wedekind <jan at wedesoft.de> 1.0

• Applications seems to run stable now -> releasing version 1.0

2006-10-20 Jan Wedekind 0.6

• Minor fixes on configure-script

2006-10-20 Jan Wedekind 0.5

• Fixed the check for gfortran (using AC_PROG_FC now). Thanks to Steven G. Johnson!

# On the older Mandriva version you may have to configure using
./configure FLIBS="-lgfortran"

# On openSuSE 10.0 libgfortranbegin.a was compiled without "-fPIC". You need to configure using
./configure F77=gfortran FLIBS="-lgfortran"

2006-10-17 Jan Wedekind 0.4

• Added support for UYVY-cameras.

2006-10-16 Jan Wedekind 0.3

• Video display can be zoomed using the mouse-wheel.
• Display error-messages when accessing camera fails.

2006-10-13 Jan Wedekind 0.2

• Removed dependency on ImageMagick.

Downloads

Download mando-1.4.tar.bz2 (334 kByte). For your convenience the required parts of Mimas are included in the package. To build and run the package do

tar xjf mando-1.4.tar.bz2
cd mando-1.4
./configure
make
./mando

Note that thanks to Víctor Fernández there is an RPM package of kmando-1.0 for openSuSE 10.2!

Thanks to Donnie S. Bhayangkara there is an RPM package of kmando-1.4 for openSuSE 10.2 at http://repos.opensuse.org/KDE:/Community/openSUSE_10.2/

Treviño Treviño supplied a Ubuntu package of mando-1.4 for Ubuntu Edgy at http://3v1n0.tuxfamily.org/dists/edgy/3v1n0/

Gallery

Accessing the KDE-menu using the system (2.72 MByte video)	Artist's impression of the system	Interactive camera-projector system	Long demonstration of the prototype-system (11.4 MByte video)	Short interactive slides presentation with the system (6.3 MByte video)
Browsing the file-system and trying to play minigolf (14.4 MByte video)	Calibration of the screen using projected patterns	Prototype of early user-interface (13.7 MByte video)	Screenshot of the wizard for configuring the recognition algorithm	Playing Reversi against the computer (5.54 MByte video)

Future Work

• Port application to Ruby using HornetsEye and Qt4-Ruby
• Develop port for MS Windows
• Add support for firewire cameras (libdc1394, libdc1394 version 2)
• Support for V4L version 2

You can help by donating some of your spare time and expertise ;)

See Also

• Mimas Camera Calibration
• Student Projects Available

External Links

• Interative Camera-Projector System
  • Project page at KDE-apps
• References
  • A Flexible New Technique for Camera Calibration: Zhengyou Zhang
  • Creating input events in Linux/X11 using the X11 test interface
  • Report by Ushakiran Soutapalli
  • Report by Juan Roldan
• Other projects
  • Edward Tse - Interactions Inventor
  • Multi-Touch Interaction Research
  • Magic board project by IIHM
  • IBM Research on interactive projector
  • Finger tracking with laser scanner
  • LaserGo allows playing Go using a laser-pointer
  • E-Chalk project
  • MIT iCampus: Magic Paper
  • Occlusion-free projection with multiple projectors
  • Detect and neutralise digital cameras
  • Video of Microsoft Surface at popularmechanics.com and at at youtube.com
  • Calibration for a multiple projector mosaic
  • Andy Wilson's human computer interface (video)
  • Navisis USB touch screen extension (technique)
  • projection keyboards
  • Johnny Chung Lee - Wiimote whiteboard (also see Sourceforge project)
• Publicity
  • kmando on digg.com
  • kmando on barrapunto.com
  • kmando on es.theinquirer.net
  • kmando on vocescuola.it
  • kmando on ossblog.it
  • kmando on linuzeros.org
  • kmando on somgnu.org
  • kmando on linmagazine.co.il