|
|
| (90 intermediate revisions by one user not shown) |
| Line 1: |
Line 1: |
| | {|align="right" | | {|align="right" |
| | |- | | |- |
| − | |[[Image:Nanoworkshop.jpg|200px]]||[[Image:Hornetseyerf.png|420px]] | + | |[[Image:AVA-Bristol-2008.jpg|thumb|240px|[http://vision.eng.shu.ac.uk/jan/ava-bristol-2008.pdf Poster] for the 2008 [http://hlsweb.dmu.ac.uk/ava/meetings/bristol2008.html AVA meeting] in Bristol]] |
| | + | |- |
| | + | |[[Image:Oscon08foils.jpg|240px|thumb|Conference presentation [http://vision.eng.shu.ac.uk/jan/oscon08-foils.pdf Real-time Computer Vision With Ruby] presented at [http://en.oreilly.com/oscon2008/ OSCON 2008]]] |
| | |- | | |- |
| | |} | | |} |
| | =Introduction= | | =Introduction= |
| − | [[Image:Hornetseye.png|right|120px]]
| + | '''[http://www.wedesoft.demon.co.uk/hornetseye-api/ HornetsEye]''' is a Ruby-extension for developing video processing and real-time computer vision software under GNU/Linux offering interfaces to do image- and video-I/O with RMagick, Xine, firewire digital camera, and video for Linux. A new class of unprecedented solutions and a new way of working becomes conceivable when applying a dynamically typed, object-oriented language like Ruby to computer vision. |
| − | '''[http://www.wedesoft.demon.co.uk/hornetseye-api/ HornetsEye]''' is a [http://www.rubyist.net/~nobu/ruby/Ruby_Extension_Manual.html Ruby-extension] for real-time computer vision under GNU/Linux offering interfaces to do image- and video-I/O with [http://rmagick.rubyforge.org/ RMagick], [http://www.xinehq.de/ Xine], IIDC/DCAM-compatible firewire digital camera ([http://damien.douxchamps.net/ieee1394/libdc1394/ DC1394]) and video for Linux ([http://www.exploits.org/v4l/ V4L]). | + | |
| − | | + | |
| − | '''[http://www.wedesoft.demon.co.uk/hornetseye-api/ HornetsEye]''' also is an attempt to use the Mimas library and create a ''minimalistic'' and ''consistent'' real-time computer vision library.
| + | |
| − | * '''minimalistic''': The library is focused on real-time computer vision. Existing libraries are being made used of.
| + | |
| − | * '''consistent''':: A non-redundant set of data-types is used. Also the library tries to stay consistent with existing libraries.
| + | |
| − | | + | |
| − | The logo was created using [http://gimp.org/ GIMP] and it shows a honeycomb structure as you would find on an insect's compound eye. A hornet is capable of navigating and detecting objects with the limited resolution of its [http://en.wikipedia.org/wiki/Compound_eye compound eyes].
| + | |
| − | | + | |
| − | =Examples=
| + | |
| − | See [http://www.wedesoft.demon.co.uk/hornetseye-api/files/inputgrey-txt.html Hornetseye homepage] for more examples.
| + | |
| − | ==Simple Webcam Application==
| + | |
| − | [[Image:Rubywebcam.jpg|thumb|240px|right|Screenshot of webcam application written in Ruby]]
| + | |
| − | The webcam application uses [[Hornetseye|HornetsEye-1.5]], [http://rubyforge.org/projects/rmagick/ RMagick], and [http://rubyforge.org/projects/korundum/ qt4-ruby].
| + | |
| − | | + | |
| − | You need to install qt4-qtruby-1.4.8 or later because there was a memory leak in Qt::ByteArray of qt4-qtruby-1.4.7. Many thanks to [http://rubyforge.org/users/rdale/ Richard Dale] for fixing the problem!
| + | |
| − | | + | |
| − | <!-- To install [http://rubyforge.org/frs/shownotes.php?release_id=7630 qt4-qtruby-1.4.7] I had to [http://rubyforge.org/forum/message.php?msg_id=19132 change the script ./smoke/qt/qtguess.pl.in]. Otherwise on gets an error message like this:
| + | |
| − | problem with QListWidget missing parent at ../../kalyptus/kalyptusCxxToSmoke.pm line 2207.
| + | |
| − | | + | |
| − | Also [http://rubyforge.org/frs/shownotes.php?release_id=7630 qt4-qtruby-1.4.7] seems to have a [http://rubyforge.org/forum/message.php?msg_id=19077 memory leak in Qt::ByteArray]. Thanks to [http://rubyforge.org/users/rdale/ Richard Dale] the problem was solved and the bugfix will be included in the next release of qt4-qtruby. The modified code already is [http://websvn.kde.org/trunk/KDE/kdebindings/qtruby/ available via the KDE source repository]. -->
| + | |
| − | | + | |
| − | Once the required software is installed, one can develop sophisticated applications within a very short time.
| + | |
| − | As an example here is the source code of the webcam application. Note that you need the user-interface description file as well (which was created with [http://trolltech.com/products/qt/features/designer Qt4 designer]). You need to compile the user-interface file with '''rbuic4''' to get ''ui_webcamwindow.rb''.
| + | |
| − | <pre>
| + | |
| − | #!/usr/bin/ruby
| + | |
| − | # Qt webcam (requires Qt4-Ruby)
| + | |
| − | require 'Qt'
| + | |
| − | require 'hornetseye'
| + | |
| − | require 'ui_webcamwindow'
| + | |
| − | app=Qt::Application.new(ARGV)
| + | |
| − | class WebcamWindow < Qt::Dialog
| + | |
| − | slots 'open_camera()'
| + | |
| − | slots 'set_value(int)'
| + | |
| − | def initialize( parent = nil )
| + | |
| − | super
| + | |
| − | @ui = Ui::WebcamWindow.new
| + | |
| − | @ui.setupUi( self )
| + | |
| − | Qt::Object.connect( @ui.reconnectButton, SIGNAL('clicked()'),
| + | |
| − | self, SLOT('open_camera()'))
| + | |
| − | Qt::Object.connect( @ui.brightnessSlider, SIGNAL('valueChanged(int)'),
| + | |
| − | self, SLOT('set_value(int)'))
| + | |
| − | Qt::Object.connect( @ui.hueSlider, SIGNAL('valueChanged(int)'),
| + | |
| − | self, SLOT('set_value(int)'))
| + | |
| − | Qt::Object.connect( @ui.colourSlider, SIGNAL('valueChanged(int)'),
| + | |
| − | self, SLOT('set_value(int)'))
| + | |
| − | Qt::Object.connect( @ui.contrastSlider, SIGNAL('valueChanged(int)'),
| + | |
| − | self, SLOT('set_value(int)'))
| + | |
| − | @timer = 0
| + | |
| − | open_camera
| + | |
| − | end
| + | |
| − | def open_camera
| + | |
| − | @ui.errorLabel.text = ""
| + | |
| − | begin
| + | |
| − | @input.close if @input != nil
| + | |
| − | @input = nil
| + | |
| − | @input = Hornetseye::V4LInput.new( @ui.deviceEdit.text )
| + | |
| − | @timer = startTimer( 0 ) if @timer == 0
| + | |
| − | rescue RuntimeError => e
| + | |
| − | @ui.errorLabel.text = e.to_s
| + | |
| − | @input = nil
| + | |
| − | end
| + | |
| − | @input
| + | |
| − | end
| + | |
| − | def set_value( value )
| + | |
| − | puts "hi"
| + | |
| − | @input.set_sensivity( @ui.brightnessSlider.value,
| + | |
| − | @ui.hueSlider.value,
| + | |
| − | @ui.colourSlider.value,
| + | |
| − | @ui.contrastSlider.value ) if @input != nil
| + | |
| − | end
| + | |
| − | def timerEvent( e )
| + | |
| − | begin
| + | |
| − | raise "No input available" if @input == nil
| + | |
| − | str = @input.read.to_magick.to_blob {
| + | |
| − | self.format = "PPM"
| + | |
| − | self.depth = 8
| + | |
| − | }
| + | |
| − | pix = Qt::Pixmap.new
| + | |
| − | pix.loadFromData( Qt::ByteArray.fromRawData( str, str.size ) )
| + | |
| − | @ui.displayLabel.setPixmap( pix )
| + | |
| − | @ui.displayLabel.update
| + | |
| − | rescue RuntimeError => e
| + | |
| − | killTimer( @timer )
| + | |
| − | @timer = 0
| + | |
| − | end
| + | |
| − | end
| + | |
| − | end
| + | |
| − | win = WebcamWindow.new
| + | |
| − | win.show
| + | |
| − | app.exec
| + | |
| − | </pre>
| + | |
| − | | + | |
| − | Also see [http://www.wedesoft.demon.co.uk/hornetseye-api/files/webcamapp-txt.html project documentation].
| + | |
| − | | + | |
| − | ==Phase Correlation==
| + | |
| − | {|align="right"
| + | |
| − | |-
| + | |
| − | |[[Image:Apollo left.jpg|thumb|76px|Left part of image]]||[[Image:Apollo right.jpg|thumb|114px|Right part of image]]||[[Image:Apollo result.jpg|thumb|163px|Stitched image]]
| + | |
| − | |-
| + | |
| − | |}
| + | |
| − | This is an implementation of the [http://en.wikipedia.org/wiki/Phase_correlation phase correlation] for aligning images.
| + | |
| − | The code depends on [[Hornetseye|HornetsEye-1.5]] and [http://rubyforge.org/projects/narray/ NArray-fftw3].
| + | |
| − | <pre>
| + | |
| − | #!/usr/bin/ruby
| + | |
| − | require 'hornetseye'
| + | |
| − | require 'fftw3'
| + | |
| − | include Hornetseye
| + | |
| − | syntax = <<END_OF_STRING
| + | |
| − | Shift estimation
| + | |
| − | Syntax: registration.rb <image1> <image2>
| + | |
| − | Example: registration.rb astronaut.jpg apollo-16.jpg
| + | |
| − | Example: registration.rb apollo_left.jpg apollo_right.jpg
| + | |
| − | END_OF_STRING
| + | |
| − | if ARGV.size != 2
| + | |
| − | puts syntax
| + | |
| − | raise "Wrong number of command-line arguments."
| + | |
| − | end
| + | |
| − | image = (0...2).collect { |i| NArray.load_grey8( ARGV[i] ) }
| + | |
| − | # TODO: Apply windowing function?
| + | |
| − | # Force images to have same size. Make image twice as big to avoid cyclical
| + | |
| − | # correlation.
| + | |
| − | maxwidth = [ image[0].shape[0], image[1].shape[0] ].max * 2
| + | |
| − | maxheight = [ image[0].shape[1], image[1].shape[1] ].max * 2
| + | |
| − | limage = image.collect { |img|
| + | |
| − | nimg = NArray.new( NArray::BYTE, maxwidth, maxheight )
| + | |
| − | nimg[ 0...img.shape[0], 0...img.shape[1] ] = img
| + | |
| − | nimg
| + | |
| − | }
| + | |
| − | fimage = limage.collect { |img|
| + | |
| − | FFTW3.dft( img.to_type( NArray::DCOMPLEX ), +1 )
| + | |
| − | }
| + | |
| − | limage = nil
| + | |
| − | fshift = ( fimage[0] * fimage[1].conj ) / ( fimage[0].abs * fimage[1].abs )
| + | |
| − | fimage = nil
| + | |
| − | # TODO: Replace with native implementation for higher performance.
| + | |
| − | fshift = fshift.collect { |value|
| + | |
| − | if value.real.nan? or value.imag.nan?
| + | |
| − | 0
| + | |
| − | else
| + | |
| − | value
| + | |
| − | end
| + | |
| − | }
| + | |
| − | shift = FFTW3.dft( fshift, -1 )
| + | |
| − | width = shift.shape[0]
| + | |
| − | height = shift.shape[1]
| + | |
| − | shiftx = nil
| + | |
| − | shifty = nil
| + | |
| − | maxvalue = 0
| + | |
| − | for i in 0...width
| + | |
| − | for j in 0...height
| + | |
| − | if shift[i,j] > maxvalue
| + | |
| − | shiftx = i
| + | |
| − | shifty = j
| + | |
| − | maxvalue = shift[i,j]
| + | |
| − | end
| + | |
| − | end
| + | |
| − | end
| + | |
| − | shiftx = shiftx - width if shiftx > width / 2
| + | |
| − | shifty = shifty - height if shifty > height / 2
| + | |
| − | shift = nil
| + | |
| − | puts "shift-x = #{shiftx}"
| + | |
| − | puts "shift-y = #{shifty}"
| + | |
| − | minx = [ 0, shiftx ].min
| + | |
| − | miny = [ 0, shifty ].min
| + | |
| − | maxx = [ image[0].shape[0], image[1].shape[0] + shiftx ].max - 1
| + | |
| − | maxy = [ image[0].shape[1], image[1].shape[1] + shifty ].max - 1
| + | |
| − | offsetx = -minx
| + | |
| − | offsety = -miny
| + | |
| − | resultwidth = maxx + 1 - minx
| + | |
| − | resultheight = maxy + 1 - miny
| + | |
| − | result1 = NArray.new( NArray::BYTE, resultwidth, resultheight )
| + | |
| − | result1[ offsetx...( offsetx + image[0].shape[0] ),
| + | |
| − | offsety...( offsety + image[0].shape[1] ) ] = image[0] / 2
| + | |
| − | result2 = NArray.new( NArray::BYTE, resultwidth, resultheight )
| + | |
| − | result2[ ( shiftx + offsetx )...( shiftx + offsetx + image[1].shape[0] ),
| + | |
| − | ( shifty + offsety )...( shifty + offsety + image[1].shape[1] ) ] = image[1] / 2
| + | |
| − | result = result1 + result2
| + | |
| − | result.display
| + | |
| − | </pre>
| + | |
| − | | + | |
| − | =Downloads=
| + | |
| − | [[Image:Hornetseye.png|48px]] Before downloading you may want to check the [http://www.wedesoft.demon.co.uk/hornetseye-api/files/Installation-txt.html installation instructions] for information on what other software you need to install and run Hornetseye.
| + | |
| − | See [http://www.wedesoft.demon.co.uk/hornetseye-api/files/Download-txt.html download instructions] on how to obtain Hornetseye.
| + | |
| | | | |
| | =See Also= | | =See Also= |
| − | * [[Mimas]] | + | * [[Interactive Presentation Software]] |
| − | * [[Computer Vision Software]] | + | * [[Just-in-time compiler]] |
| − | | + | * [[Lucas-Kanade tracker]] |
| − | =Software Engineering=
| + | * [[Hypercomplex Wavelets]] |
| − | '''HornetsEye''' brings the functionality of existing powerful free software packages into Ruby. '''HornetsEye''' also tries to make existing Ruby extension operate with each other to enable the development of novel solutions:
| + | * [[Qt4-QtRuby installer for Microsoft Windows]] |
| − | # [[Image:Qt logo.png|30px]] '''QtRuby''', [[Image:Kde.png|30px]] '''Korundum''': [http://rubyforge.org/projects/korundum/ QtRuby] and [http://developer.kde.org/language-bindings/ruby/ Korundum] can be used to develop graphical user interfaces and desktop applications.
| + | * [[TEM vision software]] |
| − | # [[Image:Xine logo.png|48px|]] '''Xine''': Using [http://www.xinehq.de/ Xine] one can read virtually any video file and it is even possible to read streaming videos.
| + | * [[Image:Mimasanim.gif|40px]] [[Mimas]] |
| − | # [[Image:Tanaka.png|30px]] '''NArray''': Masahiro Tanaka's [http://narray.rubyforge.org/ NArray] is an implementation of n-dimensional arrays for Ruby.
| + | |
| − | # [[Image:Fftw logo.gif|50px|]] '''FFTW'''. The [http://www.fftw.org/ fftw]-library can is maybe the fastest library for performing discrete Fourier transforms. It can be invoked by using Masahiro Tanaka's [http://narray.rubyforge.org/ fftw3] extension.
| + | |
| − | # [[Image:RMagick.png|80px]] '''RMagick''': The [http://rmagick.rubyforge.org/ RMagick] Ruby-extension allows to use the powerful [http://www.imagemagick.org/Magick++/ Magick++] library in Ruby for loading and saving images.
| + | |
| − | # [[Image:Coriander.png|30px]] '''libdc1394''': Using [http://damien.douxchamps.net/ieee1394/libdc1394/ libdc1394] one can make use of a large choice of firewire digital cameras.
| + | |
| − | # [[Image:OpenEXR.jpg|30px]] '''OpenEXR''': The [http://www.openexr.org/ OpenEXR library] is used for saving and loading [http://en.wikipedia.org/wiki/High_dynamic_range_imaging high dynamic range] images.
| + | |
| − | # [[Image:C--boost logo.gif|80px|]] '''Boost''': The [http://www.boost.org/ Boost Library] offers smart pointers to do exception safe programming, multi-dimensional arrays, template meta-programming, abstract data types for linear algebra and many other programming concepts. The Boost library is going to be part of a future C++ standard.
| + | |
| − | # [[Image:Stl logo.gif|30px|]] '''STL''': The software makes use of the [http://www.sgi.com/tech/stl/ Standard Template Library]
| + | |
| − | # [[Image:Ruby.png|30px]] [http://www.ruby-lang.org/ Ruby] programming language
| + | |
| − | # [[Image:Gnu-arch logo.png|30px|]] '''gnu-arch''': [http://www.gnu.org/software/gnu-arch/ gnu-arch] is being used for version control.
| + | |
| − | # [[Image:Gcc logo.png|30px|]] '''Gcc''': [http://www.gnu.org/software/gcc/ gcc] is the C++ compiler of the GNU project.
| + | |
| − | # [[Image:Gnu-head.jpg|35px|]] '''autoconf''', '''automake''' and '''make''': [http://www.gnu.org/software/make/ make], [http://www.gnu.org/software/autoconf/ autoconf] and [http://www.gnu.org/software/automake/ automake] are used to configure and perform the build of the software on various distributions of the Linux operating system.
| + | |
| − | # [[Image:Naturaldocs.png|80px]] '''Natural Docs''': [http://www.naturaldocs.org/ Natural Docs] is used to create the HTML documentation.
| + | |
| − | | + | |
| − | =See Also=
| + | |
| − | * [[Mimas]]
| + | |
| − | * [[Computer Vision Software]]
| + | |
| | | | |
| | =External Links= | | =External Links= |
| − | * [[Image:Hornetseye.png|48px]] [http://www.wedesoft.demon.co.uk/hornetseye-api/ Hornetseye homepage] | + | * [[Image:Hornetseye.png|48px]] [http://www.wedesoft.demon.co.uk/hornetseye-api/ HornetsEye homepage] |
| − | * [[Image:Rubyforge.png|75px]] [http://rubyforge.org/projects/hornetseye/ Hornetseye at Rubyforge] | + | * [[Image:Rubyforge.png|75px]] [http://rubyforge.org/projects/hornetseye/ HornetsEye at Rubyforge] |
| − | * [[Image:Sourceforge.png|58px]] [http://sourceforge.net/projects/hornetseye/ Hornetseye at Sourceforge] | + | * [[Image:Sourceforge.png|58px]] [http://sourceforge.net/projects/hornetseye/ HornetsEye at Sourceforge] |
| | * [[Image:Swig.png|48px]] [http://www.swig.org/ SWIG] (Simplified Wrapper and Interface Generator) | | * [[Image:Swig.png|48px]] [http://www.swig.org/ SWIG] (Simplified Wrapper and Interface Generator) |
| − | * [http://www.csie.ntnu.edu.tw/~bbailey/Moments%20in%20IP.htm Moments in image processing]
| + | |
| | + | {{AddThis}} |
| | | | |
| | [[Category:Projects]] | | [[Category:Projects]] |
| | [[Category:Nanorobotics]] | | [[Category:Nanorobotics]] |
Mimas
HornetsEye homepage
HornetsEye at Rubyforge
HornetsEye at Sourceforge
SWIG (Simplified Wrapper and Interface Generator)
