Revision as of 14:54, 27 June 2007

Poster for nanomanipulation workshop

Introduction

HornetsEye is a Ruby-extension for real-time computer vision under GNU/Linux offering interfaces to do image- and video-I/O with RMagick, Xine, IIDC/DCAM-compatible firewire digital camera (DC1394) and video for Linux (V4L).

HornetsEye also is an attempt to use the Mimas library and create a minimalistic and consistent real-time computer vision library.

• minimalistic: The library is focused on real-time computer vision. Existing libraries are being made used of.
• consistent:: A non-redundant set of data-types is used. Also the library tries to stay consistent with existing libraries.

The logo was created using GIMP and it is based on a nice photo published by Olivander. A hornet is capable of navigating and detecting objects with the limited resolution of its compound eyes.

Examples

See Hornetseye homepage for more examples.

Simple Webcam Application

The webcam application uses HornetsEye-1.5, RMagick, and qt4-ruby.

To install qt4-qtruby-1.4.7 I had to change the script ./smoke/qt/qtguess.pl.in. Otherwise on gets an error message like this:

problem with QListWidget missing parent at ../../kalyptus/kalyptusCxxToSmoke.pm line 2207.

Also qt4-qtruby-1.4.7 seems to have a memory leak in Qt::ByteArray. Thanks to Richard Dale the problem was solved and the bugfix will be included in the next release of qt4-qtruby. The modified code already is available via the KDE source repository.

#!/usr/bin/ruby
require 'hornetseye'
require 'Qt'
app=Qt::Application.new(ARGV)
class VideoWidget < Qt::Label
  def initialize( parent = nil )
    super
    @input = Hornetseye::V4LInput.new
    startTimer( 0 )
  end
  def timerEvent( e )
    str = @input.read.to_magick.to_blob { self.format = "PPM"; self.depth = 8 }
    pix = Qt::Pixmap.new
    pix.loadFromData( Qt::ByteArray.fromRawData( str, str.size ) )
    setPixmap( pix )
    resize( pix.width, pix.height )
    update
  end
end
win = VideoWidget.new
win.show
app.exec

Phase Correlation

This is an implementation of the phase correlation for aligning images. The code depends on HornetsEye-1.5 and NArray-fftw3.

#!/usr/bin/ruby
require 'hornetseye'
require 'fftw3'
include Hornetseye
syntax = <<END_OF_STRING
Shift estimation
Syntax: registration.rb <image1> <image2>
Example: registration.rb astronaut.jpg apollo-16.jpg
Example: registration.rb apollo_left.jpg apollo_right.jpg
END_OF_STRING
if ARGV.size != 2
  puts syntax
  raise "Wrong number of command-line arguments."
end
image = (0...2).collect { |i| NArray.load_grey8( ARGV[i] ) }
# TODO: Apply windowing function?
# Force images to have same size. Make image twice as big to avoid cyclical
# correlation.
maxwidth  = [ image[0].shape[0], image[1].shape[0] ].max * 2
maxheight = [ image[0].shape[1], image[1].shape[1] ].max * 2
limage = image.collect { |img|
  nimg = NArray.new( NArray::BYTE, maxwidth, maxheight )
  nimg[ 0...img.shape[0], 0...img.shape[1] ] = img
  nimg
}
fimage = limage.collect { |img|
  FFTW3.dft( img.to_type( NArray::DCOMPLEX ), +1 )
}
limage = nil
fshift = ( fimage[0] * fimage[1].conj ) / ( fimage[0].abs * fimage[1].abs )
fimage = nil
# TODO: Replace with native implementation for higher performance.
fshift = fshift.collect { |value|
  if value.real.nan? or value.imag.nan?
    0
  else
    value
  end
}
shift = FFTW3.dft( fshift, -1 )
width  = shift.shape[0]
height = shift.shape[1]
shiftx = nil
shifty = nil
maxvalue = 0
for i in 0...width
  for j in 0...height
    if shift[i,j] > maxvalue
      shiftx = i
      shifty = j
      maxvalue = shift[i,j]
    end
  end
end
shiftx = shiftx - width  if shiftx > width  / 2
shifty = shifty - height if shifty > height / 2
shift = nil
puts "shift-x = #{shiftx}"
puts "shift-y = #{shifty}"
minx = [ 0, shiftx ].min
miny = [ 0, shifty ].min
maxx = [ image[0].shape[0], image[1].shape[0] + shiftx ].max - 1
maxy = [ image[0].shape[1], image[1].shape[1] + shifty ].max - 1
offsetx = -minx
offsety = -miny
resultwidth  = maxx + 1 - minx
resultheight = maxy + 1 - miny
result1 = NArray.new( NArray::BYTE, resultwidth, resultheight )
result1[ offsetx...( offsetx + image[0].shape[0] ),
         offsety...( offsety + image[0].shape[1] ) ] = image[0] / 2
result2 = NArray.new( NArray::BYTE, resultwidth, resultheight )
result2[ ( shiftx + offsetx )...( shiftx + offsetx + image[1].shape[0] ),
         ( shifty + offsety )...( shifty + offsety + image[1].shape[1] ) ] = image[1] / 2
result = result1 + result2
result.display

Downloads

Hornetseye-0.15

• Download HornetsEye-0.15 released on Jun 4th 2007

Release Notes

See HornetsEye homepage for installation instructions.

Change log

• Renamed loading and saving methods to 'save_grey8', 'load_grey8', 'save_rgb24', and 'load_rgb24'.
• Renamed "XineInput::seek" to "XineInput::pos=" and implemented "XineInput::pos".
• Test for 'ruby/narray' if 'narray' is not found.
• hornetseye/ruby/hornetseye.cc: Bug! Red and blue channel where swapped in numerical array.
• hornetseye/base/colourspace.cc: Using slightly different colourspace conversions more suitable for YUV <-> RGB (ITU-R BT.709 standard). This allows YUV to grey conversion by simply stripping of the chroma components.
• hornetseye/ruby/hornetseye_ext.rb: Added methods for clipping grey values. Renamed method 'threshold' to 'binarise'.
• Implemented colour conversions RGB24-to-YV12 and Grey8-to-YV12.

Hornetseye-0.14

• Download HornetsEye-0.14 released on May 7th 2007

Release Notes

See HornetsEye homepage for installation instructions.

Change log

• Added IRB example.
• hornetseye/ruby/hornetseye.cc: Added typechecking for arguments.
• hornetseye/io/dc1394input.cc: More detailed error-message for DMA transfer does not work (after testing with Unibrain Fire-i camera).
• configure.ac: Linking with glut not required at the moment.

Hornetseye-0.13

• Download HornetsEye-0.13 released on April 5th 2007

Release Notes

See HornetsEye homepage for installation instructions.

Change log

• Porting documentation from RDoc to NaturalDocs.
• hornetseye/io/openglimagepainter.cc: Not including GL/glext.h any more because it leads to problems on some NVidia installations.
• Added methods to unmap and map X11-windows.
• Bug! Dimensions of NArray were swapped. All conversion to and from Hornetseye::Image as well as the filters had to be fixed.

HornetsEye-0.12

• Download HornetsEye-0.12 released on March 20th 2007

Release Notes

See HornetsEye homepage for installation instructions.

Change log

• Added OpenGLOutput and XVideoOutput for displaying images and videos. Added corresponding examples in directory ./samples/display.

HornetsEye-0.11

• Download HornetsEye-0.11 released on March 11th 2007

Release Notes

See HornetsEye homepage for installation instructions.

Change log

• hornetseye/io/dc1394input.cc: Do not prefer RGB24 in mode selection.
• hornetseye/io/v4linput.cc: Do not prefer RGB24 in mode selection, because it is very slow. Fixed bug in colourspace selection code. Improved speed by implementing background capture.
• Now also links with versions of libdc1394 older than 1.1

HornetsEye-0.10

• Download HornetsEye-0.10 released on February 1st 2007

Release Notes

See HornetsEye homepage for installation instructions.

Change log

• Made display method accept more element-types.
• Normalisation also works on blank image.

Older releases

See Hornetseye page at Rubyforge for older releases.

See Also

• Mimas

External Links

• Hornetseye homepage
• Hornetseye at Rubyforge
• Ruby programming language
• QtRuby, Korundum
• SWIG (Simplified Wrapper and Interface Generator)
• Moments in image processing