Interactive Camera-Projector System

From MMVLWiki
(Difference between revisions)
Jump to: navigation, search
m (→Application)
m
 
(16 intermediate revisions by one user not shown)
Line 2: Line 2:
 
|-
 
|-
 
|[[Image:Juan.jpg|thumb|160px|Developed by ''Juan Roldan'', ERASMUS exchange student from Cordoba, Spain]]||[[Image:Soutapalli.jpg|thumb|160px|''Ushakiran Soutapalli'', MSc student at Sheffield Hallam worked on the graphical user-interface]]
 
|[[Image:Juan.jpg|thumb|160px|Developed by ''Juan Roldan'', ERASMUS exchange student from Cordoba, Spain]]||[[Image:Soutapalli.jpg|thumb|160px|''Ushakiran Soutapalli'', MSc student at Sheffield Hallam worked on the graphical user-interface]]
−
|-
 
−
| colspan="2" align="center"| <html><div class="thumb tright"><div style="width:360px;"><embed style="width:360px; height:314px;" id="VideoPlayback" type="application/x-shockwave-flash" src="http://video.google.com/googleplayer.swf?docId=4201589281466176805" pluginspage="http://www.macromedia.com/go/getflashplayer"/>
 
−
<div class="thumbcaption">Playing Reversi against the computer (available as <a href="http://video.google.co.uk/videoplay?docid=4201589281466176805">Google-Video</a> or <a href="http://vision.eng.shu.ac.uk/jan/reversi.avi">5.54 MByte DivX-video</a>)</div></div></div></html>
 
 
|-
 
|-
 
|}
 
|}
Line 10: Line 7:
 
=Camera-Projector System=
 
=Camera-Projector System=
 
==Application==
 
==Application==
−
'''kmando''' is a '''human computer interface''' using a camera and a projector. It was developed by ''Juan Roldan'' (prototype), ''Ushakiran Soutapalli'' (user interface), ''Julien Faucher'' (calibration), ''Jan Wedekind'' (technical supervision), and ''Claudia Amico'' (artwork) at the [[MMVL|Microsystem & Machine Vision Laboratory (MMVL)]] at [http://www.shu.ac.uk Sheffield Hallam University, UK].
+
'''[http://mando.sourceforge.net/ mando]''' (or '''kmando''') is a '''human computer interface''' using a camera and a projector. It was developed by ''Juan Roldan'' (prototype), ''Ushakiran Soutapalli'' (user interface), ''Julien Faucher'' (calibration), ''Jan Wedekind'' (technical supervision), and ''Claudia Amico'' (artwork) at the [[MMVL|Mobile Machines and Vision Laboratory (MMVL)]] at [http://www.shu.ac.uk Sheffield Hallam University, UK].
  
 
The software makes use of a Video for Linux version 1 (V4L) camera such as the [http://www.qbik.ch/usb/devices/showdev.php?id=3332 Logitech Quickcam Pro 4000] 640x480 webcam (see [http://www.exploits.org/v4l/ exploits.org/v4l] and [http://www.qbik.ch/usb/devices/ qbik.ch] for compatible USB devices). The camera is [[Mimas Camera Calibration|calibrated]] against a standard projector screen and then used to determine the position of physical pointer (e.g. a pen) which is then used to virtually move the X11 pointer. Point-and-click functionality has also been implemented. The software now also supports IIDC/DCAM compatible firewire digital cameras such as the [http://damien.douxchamps.net/ieee1394/cameras/ Unibrain Fire-i]
 
The software makes use of a Video for Linux version 1 (V4L) camera such as the [http://www.qbik.ch/usb/devices/showdev.php?id=3332 Logitech Quickcam Pro 4000] 640x480 webcam (see [http://www.exploits.org/v4l/ exploits.org/v4l] and [http://www.qbik.ch/usb/devices/ qbik.ch] for compatible USB devices). The camera is [[Mimas Camera Calibration|calibrated]] against a standard projector screen and then used to determine the position of physical pointer (e.g. a pen) which is then used to virtually move the X11 pointer. Point-and-click functionality has also been implemented. The software now also supports IIDC/DCAM compatible firewire digital cameras such as the [http://damien.douxchamps.net/ieee1394/cameras/ Unibrain Fire-i]
−
(see [http://damien.douxchamps.net/ieee1394/cameras/ Damien Douxchamps list] for cameras with IIDC=yes/1.30/1.31). Note that Video for Linux
+
(see [http://damien.douxchamps.net/ieee1394/cameras/ Damien Douxchamps list] for cameras with IIDC=yes/1.30/1.31). Note that Video for Linux version 2 (V4L2) currently is not supported by mando.
−
version 2 (V4L2) currently is not supported by kmando.
+
  
 
The required parts of the [[Mimas|Mimas real-time computer vision toolkit]] are included in the package.
 
The required parts of the [[Mimas|Mimas real-time computer vision toolkit]] are included in the package.
Line 83: Line 79:
  
 
==Downloads==
 
==Downloads==
−
Download [http://vision.eng.shu.ac.uk/jan/mando-1.6.tar.bz2 mando-1.6.tar.bz2] (366 kByte). For your convenience the required parts of [[Mimas]] are included in the package. To build and run the package do
+
Download the latest release ''mando-*.tar.bz2'' from the [http://sourceforge.net/project/platformdownload.php?group_id=220882 mando download section at Sourceforge]. For your convenience the required parts of [[Mimas]] are included in the package. To build and run the package do
−
  tar xjf mando-1.6.tar.bz2
+
  tar xjf mando-*.tar.bz2
−
  cd mando-1.6
+
  cd mando-*
 
  ./configure
 
  ./configure
 
  make
 
  make
 
  ./mando
 
  ./mando
  
−
[http://3v1n0.tuxfamily.org/dists/edgy/3v1n0/ Treviño Treviño] supplied a Ubuntu package of mando-1.4 for Ubuntu Edgy at http://3v1n0.tuxfamily.org/dists/edgy/3v1n0/
+
<!-- [http://3v1n0.tuxfamily.org/dists/edgy/3v1n0/ Treviño Treviño] supplied a Ubuntu package of mando-1.4 for Ubuntu Edgy at http://3v1n0.tuxfamily.org/dists/edgy/3v1n0/ -->
 +
 
 +
[http://kde-apps.org/usermanager/search.php?username=pupil Donnie Bhayangkara] has built a package for openSuSE 10.3 which can be downloaded here http://software.opensuse.org/search?baseproject=openSUSE%3A10.3&amp;p=1&amp;q=mando
 +
 
 +
==Git Repository==
 +
The git repository can be accessed as follows
  
−
[http://kde-apps.org/usermanager/search.php?username=pupil Donnie Bhayangkara] has built a package for openSuSE 10.3 which can be downloaded here
+
git clone git://mando.git.sourceforge.net/gitroot/mando mando
−
http://software.opensuse.org/search?baseproject=openSUSE%3A10.3&amp;p=1&amp;q=mando
+
  
 
==Gallery==
 
==Gallery==
Line 118: Line 118:
 
* [[Mimas Camera Calibration]]
 
* [[Mimas Camera Calibration]]
 
* [[Interactive Presentation Software]]
 
* [[Interactive Presentation Software]]
−
* [[Student Projects Available]]
+
* [[Available Student Projects]]
  
 
=External Links=
 
=External Links=
 
* Interative Camera-Projector System
 
* Interative Camera-Projector System
 +
** '''[http://mando.sourceforge.net/ Mando project page at Sourceforge]'''
 
** [http://kde-apps.org/content/show.php?content=47105 Project page at KDE-apps]
 
** [http://kde-apps.org/content/show.php?content=47105 Project page at KDE-apps]
−
** [[Image:New.gif]] [http://sourceforge.net/projects/mando/ Mando project page at Sourceforge]
 
 
* References
 
* References
 
** [http://research.microsoft.com/users/zhang/ Zhengyou Zhang]
 
** [http://research.microsoft.com/users/zhang/ Zhengyou Zhang]
Line 130: Line 130:
 
** [http://www.bedroomlan.org/~alexios/coding_evrouter.html Creating input events in Linux/X11] using the [http://www.xfree86.org/current/xtestlib.html X11 test interface]
 
** [http://www.bedroomlan.org/~alexios/coding_evrouter.html Creating input events in Linux/X11] using the [http://www.xfree86.org/current/xtestlib.html X11 test interface]
 
** [http://vision.eng.shu.ac.uk/jan/report_soutapalli.pdf Report by Ushakiran Soutapalli]
 
** [http://vision.eng.shu.ac.uk/jan/report_soutapalli.pdf Report by Ushakiran Soutapalli]
−
** [http://vision.eng.shu.ac.uk/jan/report_roldan.pdf Report by Juan Roldan]
+
** '''[http://vision.eng.shu.ac.uk/jan/report_roldan.pdf Report by Juan Roldan]'''
 
* Other projects
 
* Other projects
 
** [http://edwardtse.com/ Edward Tse - Interactions Inventor]
 
** [http://edwardtse.com/ Edward Tse - Interactions Inventor]
Line 152: Line 152:
 
** [http://www.virtual-laser-keyboard.com/ Virtual Laser Keyboard] for mobile phones!
 
** [http://www.virtual-laser-keyboard.com/ Virtual Laser Keyboard] for mobile phones!
 
** [http://citeseer.ist.psu.edu/454741.html Compaq self-calibrating projector]
 
** [http://citeseer.ist.psu.edu/454741.html Compaq self-calibrating projector]
 +
** [http://graffitiresearchlab.com/?page_id=76 L.A.S.E.R. Graffiti] ([http://graffitiresearchlab.com/rotterdam/laser_tag_WEB.mov video])
 
* Publicity
 
* Publicity
−
** [http://digg.com/linux_unix/Want_a_touch_screen_without_having_to_buy_expensive_stuff kmando on digg.com]
+
** [http://digg.com/linux_unix/Want_a_touch_screen_without_having_to_buy_expensive_stuff mando on digg.com]
−
** [http://barrapunto.com/article.pl?sid=07/03/27/1127200 kmando on barrapunto.com]
+
** [http://barrapunto.com/article.pl?sid=07/03/27/1127200 mando on barrapunto.com]
−
** [http://es.theinquirer.net/2007/03/27/pantallas_tactiles_a_golpe_de.html kmando on es.theinquirer.net]
+
** [http://es.theinquirer.net/2007/03/27/pantallas_tactiles_a_golpe_de.html mando on es.theinquirer.net]
−
** [http://www.vocescuola.it/2007/03/28/kmando/ kmando on vocescuola.it]
+
** [http://www.vocescuola.it/2007/03/28/kmando/ mando on vocescuola.it]
−
** [http://www.ossblog.it/post/2266/kmando-interazione-avanzata-con-kde kmando on ossblog.it]
+
** [http://www.ossblog.it/post/2266/kmando-interazione-avanzata-con-kde mando on ossblog.it]
−
** [http://www.linuzeros.org/index.php?option=com_content&task=view&id=99&Itemid=1 kmando on linuzeros.org]
+
** [http://www.linuzeros.org/index.php?option=com_content&task=view&id=99&Itemid=1 mando on linuzeros.org]
−
** [http://www.somgnu.org/?p=66 kmando on somgnu.org]
+
** [http://www.somgnu.org/?p=66 mando on somgnu.org]
−
** [http://linmagazine.co.il/hacking/kmando-camera-projector-system-37794 kmando on linmagazine.co.il]
+
** [http://linmagazine.co.il/hacking/kmando-camera-projector-system-37794 mando on linmagazine.co.il]
−
** [http://www.opennewsshow.org/node/14 kmando on opennewsshow.org]
+
** [http://www.opennewsshow.org/node/14 mando on opennewsshow.org]
  
−
<!-- Has iSightSee taken of now? [http://mydreamapp.com/forums/viewtopic.php?id=960 Mike Gaboury about his project iSightSee] -->
+
{{Addthis}}
  
 
[[Category:Projects]]
 
[[Category:Projects]]
 
[[Category:Fun-Projects]]
 
[[Category:Fun-Projects]]

Latest revision as of 20:09, 16 April 2011

Developed by Juan Roldan, ERASMUS exchange student from Cordoba, Spain
Ushakiran Soutapalli, MSc student at Sheffield Hallam worked on the graphical user-interface

Contents

[edit] Camera-Projector System

[edit] Application

mando (or kmando) is a human computer interface using a camera and a projector. It was developed by Juan Roldan (prototype), Ushakiran Soutapalli (user interface), Julien Faucher (calibration), Jan Wedekind (technical supervision), and Claudia Amico (artwork) at the Mobile Machines and Vision Laboratory (MMVL) at Sheffield Hallam University, UK.

The software makes use of a Video for Linux version 1 (V4L) camera such as the Logitech Quickcam Pro 4000 640x480 webcam (see exploits.org/v4l and qbik.ch for compatible USB devices). The camera is calibrated against a standard projector screen and then used to determine the position of physical pointer (e.g. a pen) which is then used to virtually move the X11 pointer. Point-and-click functionality has also been implemented. The software now also supports IIDC/DCAM compatible firewire digital cameras such as the Unibrain Fire-i (see Damien Douxchamps list for cameras with IIDC=yes/1.30/1.31). Note that Video for Linux version 2 (V4L2) currently is not supported by mando.

The required parts of the Mimas real-time computer vision toolkit are included in the package.

[edit] How it works

[edit] Detection

The detection is a simple colour-segmentation using a thresholded colour-histogram. Therefore you need to use a marker pen or some other object with a distinct colour. The colour should not appear on the screen itself. If the colour of the pen will change due to different illumination conditions, you need to capture these colours when taking the reference image.

[edit] Mouse clicks

The standard deviation of a few recent mouse-positions is used as a criterion, wether the mouse is stationary or moving. If the mouse is stationary, a mouse click is issued.

[edit] Design

UML-diagram of static structure
UML-diagram showing collaboration of classes

[edit] Changelog

2008-02-07 version 1.6

  • Added more confusing array operator methods to keep up with the GCC freaks.

2007-03-27 version 1.5

  • Adding support for firewire digital camera
  • image_v4linput.cc: Accepting YUV422 (assuming it is same as UYVY).
  • Wizard has new combo-box for selecting video mode (PAL/NTSC/SECAM/AUTO)
  • image_v4linput.cc: Not preferring RGB support of camera. Prefer UYVY or YUV420p instead.

2007-04-14 version 1.4

  • Added spin-box for selecting camera channel.
  • GUI now has sliders for adjusting brightness, contrast, and others.

2007-03-27 version 1.3

  • Added some explanation about capturing reference image.
  • Distorted e-mails because of spam.

2007-02-12 version 1.2

  • configure.ac: Configure script must not use Qt3-moc.

2007-02-11 version 1.1

  • Makefile.am: Added moc-, qrc-, and uic-files to clean-rule.
  • configure.ac: Call "make clean" to delete built sources.

2007-02-02 version 1.0

  • Applications seems to run stable now -> releasing version 1.0

2006-10-20 version 0.6

  • Minor fixes on configure-script

2006-10-20 version 0.5

  • Fixed the check for gfortran (using AC_PROG_FC now). Thanks to Steven G. Johnson! 
# On the older Mandriva version you may have to configure using
./configure FLIBS="-lgfortran"

# On openSuSE 10.0 libgfortranbegin.a was compiled without "-fPIC". You need to configure using
./configure F77=gfortran FLIBS="-lgfortran"

2006-10-17 version 0.4

  • Added support for UYVY-cameras.

2006-10-16 version 0.3

  • Video display can be zoomed using the mouse-wheel.
  • Display error-messages when accessing camera fails.

2006-10-13 version 0.2

  • Removed dependency on ImageMagick.

[edit] Downloads

Download the latest release mando-*.tar.bz2 from the mando download section at Sourceforge. For your convenience the required parts of Mimas are included in the package. To build and run the package do 

tar xjf mando-*.tar.bz2
cd mando-*
./configure
make
./mando


Donnie Bhayangkara has built a package for openSuSE 10.3 which can be downloaded here http://software.opensuse.org/search?baseproject=openSUSE%3A10.3&p=1&q=mando

[edit] Git Repository

The git repository can be accessed as follows 

git clone git://mando.git.sourceforge.net/gitroot/mando mando

[edit] Gallery

Accessing the KDE-menu using the system (2.72 MByte video)
Artist's impression of the system
Interactive camera-projector system
Long demonstration of the prototype-system (11.4 MByte video)
Short interactive slides presentation with the system (6.3 MByte video)
Browsing the file-system and trying to play minigolf (14.4 MByte video)
Calibration of the screen using projected patterns
Prototype of early user-interface (13.7 MByte video)
Screenshot of the wizard for configuring the recognition algorithm
Playing Reversi against the computer (5.54 MByte video)

[edit] Possible Future Work

  • Port application to Ruby using HornetsEye and Qt4-Ruby
  • Port application to OpenMoko+Webcam
  • Support for libdc1394 version 2
  • Support for V4L version 2
  • Test with cheap IR-camera
  • Test with 90 fps camera
  • Gray code calibration
  • use histogram inversion instead of colour histograms
  • allow multi-touch (detect multiple pointers) by using local minima of boundary curve

You can help by donating some of your spare time and expertise ;)

[edit] See Also

[edit] External Links

Bookmark and Share

Retrieved from "https://vision.eng.shu.ac.uk/mmvl/mmvlwiki/index.php?title=Interactive_Camera-Projector_System&oldid=5736"
Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox