Revision as of 17:34, 20 March 2007

Introduction

HornetsEye is a Ruby-extension for real-time computer vision under GNU/Linux offering interfaces to do image- and video-I/O with RMagick, Xine, firewire digital camera (DC1394) and video for Linux (V4L).

HornetsEye also is an attempt to use the Mimas library and create a minimalistic and consistent real-time computer vision library.

• minimalistic: The library is focused on real-time computer vision. Existing libraries are being made used of.
• consistent:: A non-redundant set of data-types is used. Also the library tries to stay consistent with existing libraries.

The logo was created using GIMP and it is based on a nice photo published by Olivander. A hornet is capable of navigating and detecting objects with the limited resolution of its compound eyes.

Example

Simple Object Recognition Example

The example program performs two-dimensional object recognition with three degrees of freedom. This is a customised algorithm which only works on images showing a single object which can be detected using colour-segmentation. In a controlled environment however this algorithm can be very useful as it is easy to implement. It is also possible to optimise it for real-time applications.

#!/usr/bin/ruby
# Detect location and rotation of an object using color-segmentation and
# principal component analysis on resulting binary image.
require 'hornetseye'
require 'matrix'
raise "Syntax: pcarecognition.rb [media resource location]" if ARGV.size != 1
input = Hornetseye::XineInput.new( ARGV[0] )
# Object is black.
dominant = 0 & 0xE0
frame = 0
old_eigenvector = Vector[ 1, 0 ]
while input.status?
  # Read image.
  img = input.read_grey
  # Detect center and rotation of object using principal component analysis.
  # Assuming object has a principal axis (otherwise this approach fails).
  c = 0
  n = 0
  sum = Vector[ 0, 0 ]
  squares = Matrix[ [ 0, 0 ], [ 0, 0 ] ]
  img.each do |v|
    if v & 0xE0 == dominant
      point = Vector[ c % img.shape[1], c / img.shape[1] ]
      sum += point
      squares += point.covector.transpose * point.covector
      n += 1
    end
    c += 1
  end
  center = sum * ( 1.0 / n )
  covariance = ( n * squares -
                 sum.covector.transpose * sum.covector ) / ( n ** 2 ).to_f
  # "abs" is needed to deal with numerical errors.
  discriminant = ( covariance.trace ** 2 - 4 * covariance.determinant ).abs
  # Take largest eigenvalue. Eigenvalues are "0.5 * ( covariance.trace +- Math.sqrt( discriminant ) )" 
  lambda1 = 0.5 * ( covariance.trace + Math.sqrt( discriminant ) )
  eigenspace = covariance - lambda1 * Matrix.unit( 2 )
  # Compute eigenvector by projecting basis-vectors.
  projected1 = eigenspace * Vector[1,0]
  projected2 = eigenspace * Vector[0,1]
  if projected1.r >= projected2.r
    projected = projected1 * ( 1.0 / projected1.r )
  else
    projected = projected2 * ( 1.0 / projected2.r )
  end
  eigenvector = Vector[ -projected[ 1 ], projected[ 0 ] ]
  # Resolve ambiguity by comparing with previous eigenvector.
  if old_eigenvector.inner_product( eigenvector ) < 0
    eigenvector = Vector[ projected[ 1 ], -projected[ 0 ] ]
  end
  old_eigenvector = eigenvector
  gc=Magick::Draw.new
  pointer=center+eigenvector*30
  gc.fill_opacity(0)
  gc.stroke('red').stroke_width(3)
  gc.circle(center[0],center[1],pointer[0],pointer[1])
  gc.line(center[0],center[1],pointer[0],pointer[1])
  result=img.to_magick
  gc.draw(result)
  result.to_hornetseye.save( ( "%08d" % frame ) + ".jpg" )
  frame += 1
end

Thanks to Prof. Dr.-Ing. Christoph Stiller for pointing out this algorithm.

Simple Webcam Application

#!/usr/bin/ruby
require 'hornetseye'
require 'Qt'
app=Qt::Application.new(ARGV)
class VideoWidget < Qt::Label
  def initialize( parent = nil )
    super
    @input = Hornetseye::V4LInput.new
    startTimer( 0 )
  end
  def timerEvent( e )
    str = @input.read.to_magick.to_blob { self.format = "PPM"; self.depth = 8 }
    pix = Qt::Pixmap.new
    pix.loadFromData( Qt::ByteArray.fromRawData( str, str.size ) )
    setPixmap( pix )
    resize( pix.width, pix.height )
    update
  end
end
win = VideoWidget.new
win.show
app.exec

The webcam application uses HornetsEye, RMagick, and qt4-ruby.

qt4-qtruby-1.4.7 seems to have a memory leak in Qt::ByteArray. Thanks to Richard Dale the problem was solved and the bugfix will be included in the next release of qt4-qtruby. The modified code already is available via the KDE source repository.

Downloads

HornetsEye-0.12

• 48px Download HornetsEye-0.12 released on March 20th 2007

Release Notes

See HornetsEye homepage for installation instructions.

Change log

• Added OpenGLOutput and XVideoOutput for displaying images and videos. Added corresponding examples in directory ./samples/display.

HornetsEye-0.11

• 48px Download HornetsEye-0.11 released on March 11th 2007

Release Notes

See HornetsEye homepage for installation instructions.

Change log

• hornetseye/io/dc1394input.cc: Do not prefer RGB24 in mode selection.
• hornetseye/io/v4linput.cc: Do not prefer RGB24 in mode selection, because it is very slow. Fixed bug in colourspace selection code. Improved speed by implementing background capture.
• Now also links with versions of libdc1394 older than 1.1

HornetsEye-0.10

• 48px Download HornetsEye-0.10 released on February 1st 2007

Release Notes

See HornetsEye homepage for installation instructions.

Change log

• Made display method accept more element-types.
• Normalisation also works on blank image.

Older releases

See Hornetseye page at Rubyforge for older releases.

See Also

• Mimas

External Links

• Hornetseye homepage
• Hornetseye at Rubyforge
• Ruby programming language
• QtRuby, Korundum
• SWIG (Simplified Wrapper and Interface Generator)
• Moments in image processing