TEM vision software
From MMVLWiki
(Difference between revisions)
m |
m |
||
| (15 intermediate revisions by one user not shown) | |||
| Line 1: | Line 1: | ||
| − | {|align=right | + | {|align="right" |
|- | |- | ||
| − | | | + | |[[Image:Vision_GUI5.gif|240px|thumb|Prototype using [http://www.ruby-doc.org/core/classes/DRb.html Distributed Ruby] for vision-based closed-loop control]] |
| − | + | |- | |
| − | + | |[[Image:Nano-Telemanip.jpg|thumb|240px|Moving the tip using "drag-and-drop" without vision feedback. The circle marks the initial position of the mouse-cursor]] | |
| − | + | ||
| − | + | ||
| − | + | ||
|- | |- | ||
| − | |[[Image: | + | |[[Image:Nano-Closed.jpg|thumb|240px|Here vision-based closed-loop control is used to control the position of the tip. The cross-and-circle marks the last known position of the nano-indenter. The cross marks the current nominal position]] |
|- | |- | ||
|} | |} | ||
| − | As part of the [[Nanorobotics]] | + | As part of the '''[[Nanorobotics|Nanorobotics project]]''' a [[TEM vision software]] was developed. The software makes use of a [http://www.jeolusa.com/PRODUCTS/ElectronOptics/TransmissionElectronMicroscopesTEM/300kV/JEM3010/tabid/213/Default.aspx JEOL 3010] transmission electron microscope with a [http://tvips.com/Prod_F114.php TVIPS FastScan-F114 camera] which is an IIDC/DCAM-compatible firewire camera. The nano-indenter is controlled by a [http://web.nanomagnetics-inst.com/product_detail.php?product_link=SPM Nanomagnetics SPM controller] (the old version of the controller can be accessed with a [http://www.mccdaq.com/pci-data-acquisition/PCI-DIO24.aspx PCI-DIO24 card]). |
The software runs under GNU/Linux and it makes use of [http://damien.douxchamps.net/ieee1394/libdc1394/ Damien Douxchamps' libdc1394] to access the camera and [ftp://lx10.tx.ncsu.edu/pub/Linux/drivers/ Warren Jasper's PCI-DIO24 driver] to access the PCI-card which interfaces with the SPM controller. | The software runs under GNU/Linux and it makes use of [http://damien.douxchamps.net/ieee1394/libdc1394/ Damien Douxchamps' libdc1394] to access the camera and [ftp://lx10.tx.ncsu.edu/pub/Linux/drivers/ Warren Jasper's PCI-DIO24 driver] to access the PCI-card which interfaces with the SPM controller. | ||
| Line 20: | Line 17: | ||
==Demonstration== | ==Demonstration== | ||
| + | ===Videos=== | ||
{|align="center" | {|align="center" | ||
|- | |- | ||
| − | | | + | |<html> |
| + | <div class="thumb tright"><div style="width:480px;"> | ||
| + | <object type="application/x-shockwave-flash" data="http://vision.eng.shu.ac.uk/jan/flv/flvplayer.swf" width="480" height="500"> | ||
| + | <param name="flashvars" | ||
| + | value="file=http://vision.eng.shu.ac.uk/jan/flv/tem.xml&shuffle=false&searchbar=false&displayheight=418" /> | ||
| + | <param name="movie" value="http://vision.eng.shu.ac.uk/jan/flv/flvplayer.swf" /> | ||
| + | <param name="allowfullscreen" value="true" /> | ||
| + | </object> | ||
| + | <div class="thumbcaption">Demonstration of <b>TEM vision software</b> including telemanipulation as well as closed-loop control using machine-vision feedback (also available as DivX3 videos <a href="http://vision.eng.shu.ac.uk/jan/configuration.avi">configuration.avi (64 MByte)</a>, <a href="http://vision.eng.shu.ac.uk/jan/closed-loop.avi">closed-loop.avi (44 MByte)</a>, and <a href="http://vision.eng.shu.ac.uk/jan/interaction.avi">interaction.avi (19 MByte)</a>)</div></div></div></html> | ||
|- | |- | ||
|} | |} | ||
| + | ===Setup procedure=== | ||
{|align="center" | {|align="center" | ||
|- | |- | ||
| − | |[[Image:Nano- | + | |[[Image:Nano-Configure.jpg|thumb|160px|'''1.''' The vision algorithms are configured]]||[[Image:Nano-Calibration.jpg|thumb|160px|'''2.''' The SPM axes are calibrated against the camera image. A fourth step compensates for linear drift occurring during calibration]]||[[Image:Nano-Control.jpg|thumb|160px|'''3.''' Using closed-loop control the nano-indenter is moved along a linear path]] |
|- | |- | ||
|} | |} | ||
| + | |||
| + | ==Download== | ||
| + | The software can be downloaded here: '''[http://vision.eng.shu.ac.uk/jan/visiongui-0.2.tar.bz2 visiongui-0.2.tar.bz2]''' | ||
| + | |||
| + | The software is implemented in [http://www.ruby-lang.org/ Ruby]. It uses [[HornetsEye]] (current development of version 0.32) and [http://rubyforge.org/projects/korundum/ Qt4-QtRuby]. | ||
==Future Work== | ==Future Work== | ||
| Line 36: | Line 48: | ||
* port to Ruby-1.9 which has native threads | * port to Ruby-1.9 which has native threads | ||
* integrate serial-port interface of JEOL TEM | * integrate serial-port interface of JEOL TEM | ||
| + | * access USB-controls for shutter and gain of the TVIPS camera | ||
* feature-based recognition and tracking (less sensitive to brightness changes) | * feature-based recognition and tracking (less sensitive to brightness changes) | ||
* offset- and gain-compensation for camera image | * offset- and gain-compensation for camera image | ||
| Line 42: | Line 55: | ||
* [[Nanorobotics]] | * [[Nanorobotics]] | ||
* [[HornetsEye]] | * [[HornetsEye]] | ||
| + | * [[Microscope Control Software]] | ||
| + | * [[Microscope Vision Software]] | ||
=External Links= | =External Links= | ||
| Line 57: | Line 72: | ||
** J. P. Lewis: [http://www.idiom.com/~zilla/Work/nvisionInterface/nip.html Fast Normalized Cross-Correlation], Industrial Light & Magic | ** J. P. Lewis: [http://www.idiom.com/~zilla/Work/nvisionInterface/nip.html Fast Normalized Cross-Correlation], Industrial Light & Magic | ||
** S. Baker, I. Matthews: [http://www.ri.cmu.edu/pub_files/pub3/baker_simon_2004_1/baker_simon_2004_1.pdf Lucas-Kanade 20 Years On: A Unifying Framework], International Journal of Computer Vision, Vol. 56, No. 3, March, 2004, pp. 221-255. | ** S. Baker, I. Matthews: [http://www.ri.cmu.edu/pub_files/pub3/baker_simon_2004_1/baker_simon_2004_1.pdf Lucas-Kanade 20 Years On: A Unifying Framework], International Journal of Computer Vision, Vol. 56, No. 3, March, 2004, pp. 221-255. | ||
| + | |||
| + | {{Addthis}} | ||
[[Category:Projects]] | [[Category:Projects]] | ||
[[Category:Nanorobotics|*Nanorobotics]] | [[Category:Nanorobotics|*Nanorobotics]] | ||
Latest revision as of 14:08, 10 June 2010
 |
 |
 |
As part of the Nanorobotics project a TEM vision software was developed. The software makes use of a JEOL 3010 transmission electron microscope with a TVIPS FastScan-F114 camera which is an IIDC/DCAM-compatible firewire camera. The nano-indenter is controlled by a Nanomagnetics SPM controller (the old version of the controller can be accessed with a PCI-DIO24 card).
The software runs under GNU/Linux and it makes use of Damien Douxchamps' libdc1394 to access the camera and Warren Jasper's PCI-DIO24 driver to access the PCI-card which interfaces with the SPM controller.
The software was implemented in Ruby using Qt4-QtRuby, HornetsEye, libJIT, and a custom Ruby-extension to access the SPM controller via the PCI-DIO24 card. Distributed Ruby and multiple processes were used to work around the problem that Ruby-1.8 does not offer native threads.
The vision algorithms are configured using a separate program and the configuration is saved in a file using Ruby marshalling. A plugin-based architecture, which accepts plugins for recognition and tracking, was implemented which allows one to select and configure Normalised Cross-Correlation, Lucas-Kanade tracking, or Connected Component Analysis.
Contents |
[edit] Demonstration
[edit] Videos
|
Demonstration of TEM vision software including telemanipulation as well as closed-loop control using machine-vision feedback (also available as DivX3 videos configuration.avi (64 MByte), closed-loop.avi (44 MByte), and interaction.avi (19 MByte)) |
[edit] Setup procedure
 |
 |
 |
[edit] Download
The software can be downloaded here: visiongui-0.2.tar.bz2
The software is implemented in Ruby. It uses HornetsEye (current development of version 0.32) and Qt4-QtRuby.
[edit] Future Work
Possible future work is
- port to Ruby-1.9 which has native threads
- integrate serial-port interface of JEOL TEM
- access USB-controls for shutter and gain of the TVIPS camera
- feature-based recognition and tracking (less sensitive to brightness changes)
- offset- and gain-compensation for camera image
[edit] See Also
[edit] External Links
- Hardware
- Software
- Related publications
- Jung-Me Park, C. G. Looney, Hui-Chuan Chen: Fast connected component labeling algorithm using a divide and conquer technique, 15th International Conference on Computers and their Applications, March, 2000, pp. 373-6
- J. P. Lewis: Fast Normalized Cross-Correlation, Industrial Light & Magic
- S. Baker, I. Matthews: Lucas-Kanade 20 Years On: A Unifying Framework, International Journal of Computer Vision, Vol. 56, No. 3, March, 2004, pp. 221-255.
