TEM vision software

Demonstration of TEM vision software including telemanipulation as well as closed-loop control using machine-vision feedback (also available as DivX3 videos configuration.avi (64 MByte), closed-loop.avi (44 MByte), and interaction.avi (19 MByte))

Prototype using Distributed Ruby for vision-based closed-loop control

As part of the Nanorobotics project a TEM vision software was developed. The software makes use of a JEOL 3010 transmission microscope with a TVIPS FastScan-F114 camera which is an IIDC/DCAM-compatible firewire camera. The nano-indenter is controlled by a Nanomagnetics SPM controller (the old version of the controller can be accessed with a PCI-DIO24 card).

The software runs under GNU/Linux and it makes use of Damien Douxchamps' libdc1394 to access the camera and Warren Jasper's PCI-DIO24 driver to access the PCI-card which interfaces with the SPM controller.

The software was implemented in Ruby using Qt4-QtRuby, HornetsEye, and a custom Ruby-extension to access the SPM controller via the PCI-DIO24 card. Distributed Ruby and multiple processes were used to work around the problem that Ruby-1.8 does not offer native threads.

The vision algorithms are configured using a separate program and the configuration is saved in a file using Ruby marshalling. A plugin-based architecture, which accepts plugins for recognition and tracking, was implemented which allows one to select and configure Normalised Cross-Correlation, Lucas-Kanade tracking, or Connected Component Analysis.

Future Work

Possible future work is

• port to Ruby-1.9 which has native threads
• integrate serial-port interface of JEOL TEM

See Also

• Nanorobotics
• HornetsEye

External Links

• Hardware
  • JEOL 3010
  • TVIPS FastScan-F114 IIDC/DCAM-compatible firewire camera
  • Nanomagnetics SPM controller
  • PCI-DIO24 card
• Software
  • Ruby
  • Warren Jasper's PCI-DIO24 driver
  • Qt4-QtRuby
• Related publications
  • Jung-Me Park, C. G. Looney, Hui-Chuan Chen: Fast connected component labeling algorithm using a divide and conquer technique, 15th International Conference on Computers and their Applications, March, 2000, pp. 373-6
  • J. P. Lewis: Fast Normalized Cross-Correlation, Industrial Light & Magic
  • S. Baker, I. Matthews: Lucas-Kanade 20 Years On: A Unifying Framework, International Journal of Computer Vision, Vol. 56, No. 3, March, 2004, pp. 221-255.