Development of an Intelligent Electro-optical Rangefinder using Stereoscopic Depth Estimation


Computer Vision/Digital Image Processing

Vivek Saxena <vivek_saxena@lycos.com>
     
#4071, IIT Kanpur, Kanpur - 16, IN 

 Abstract 

The recovery of three-dimensional information about structure is an important problem in the area of Computer Vision. The 3-D depth and structure information is lost during projection onto a 2-D plane. In this research, we have proposed the construction of an intelligent rangefinder system based on an approach termed as Electro-optics, in which we model optical functions of stereoscopic image formation on a system involving two cameras interfaced to a personal computer in order to extract spatial information about objects in scenes acquired as digital images. A modeling in the pixel space of point image formation of an object point in world space (X, Y, Z) is done and assuming projections of this object on the image planes for each camera as (x1, y1) and (x2, y2) respectively. We then estimate the distance of this object using simple mathematics of coordinate transformations, given the camera’s focal length. A calibration procedure based on [9] was carried out to obtain focal length and integer-valued pixel coordinates to real world distance transformation experiments were conducted for both cameras. The final transform yields information about distance ‘Z’ from a knowledge of the focal length f, the distance ‘B’ between the optical centers and the difference ∆x in the abscissas of the projections on the two image planes.

 

Keywords: calibration, image processing, information recovery, depth estimation, electro-optics and rangefinder.


About this research project

(Research Status: Continuing)

This project was originally meant to be displayed at the Intel Science Talent Discovery Fair 2003 at New Delhi, India in January 2003. However, due to certain technical problems, it did not take part in the event, but is fully functional. This project uses two LEGO Mindstorms Vision Command System digital cameras connected to the USB port of a personal computer running Microsoft Windows 2000, to implement the depth from stereo algorithm. It is working well enough with an estimated error degree of 0.05% for distances within 100cm. Each camera was calibrated using ten images of a 8x6 checkerboard image in order to estimate focal length (in pixels), radial distortion and skew coefficients before actual measurement of distance.

It is being overhauled and updated to reflect the changes in my proposed mathematical model and also to include extra features, error estimation, graphical result reporting and estimation of depth from blur and geometric modeling.

Currently, I am working on understanding and implementing depth estimation techniques from geometric modeling (reconstruction) of a scene. As described in the abstract above, a point in 3-D space (world space or world coordinate system) loses depth information on being projected on the 2-D image plane (CCD). Geometric modeling or reconstruction involves re-projection of 2-D image points onto a virtual three dimensional wireframe model that accurately depicts the relative depth of objects, within a certain level of accuracy. The geometric modeling approach has applications in the area of computer graphics, special effects, medical imaging and digitization of scenes.  This page will contain information on the project and the related computer algorithms and programs written using C/C++ and OpenCV under Linux.

The idea behind this project was to achieve a high degree of neural correspondence with the human vision system (HVS), i.e. to replicate certain functions of the human eye.


CAMERA CALIBRATION & POSE IMAGES:

Left Camera Calibration ImagesRight Camera Calibration Images

Extrinsic Camera Parameters for Left Camera [Camera Centered View]Extrinsic Camera Parameters for Right Camera [Camera Centered View]

Extrinsic Camera Parameters for Left Camera [World Centered View]

Extrinsic Camera Parameters for Right Camera [World Centered View]

LEFT CAMERA (CAMERA #1)

No. of images used                              : 10

No. of calibration trials                        :   5

No. of corner detection trials               :   2

No. of pixel-cm measurement trials      :    2

Dimensions of checkerboard                :  8x6 (25mm square side)

 

S.No.

Quantity
Abbreviation
Value/Estimated Error

1

Focal Length

fpc

818; 813 (pixels)

±22.0; ±20.4 (pixels)

2

Pixels per cm

pcm

13.3 (pixels/cm)

±0.1 cm (error in object motion)


RIGHT CAMERA (CAMERA #2)

No. of images used                               : 10

No. of calibration trials                         :   5

No. of corner detection trials                :   2

No. of pixel-cm measurement trials       :    2

Dimensions of checkerboard                :  8x6 (25mm square side)

 

S.No.

Quantity
Abbreviation
Value/Estimated Error

1

Focal Length

fpc

828; 824 (pixels)

±18.9; ±18.1 (pixels)

2

Pixels per cm

pcm

12.5 (pixels/cm)

±0.1 cm (error in object motion)


REFERENCES:  

[1]     Starling, S.G. and Woodall, A.J. [1963]. Physics Second Edition, English Language Book Society (ELBS) and Longmans Green and Co. Ltd., U.K.

[2]     Davies, W.G. [1970]. Light First Indian Edition, Blackie and Son (India) Ltd., School Certificate Physics Course Part Three.

[3]     Low, Adrian [1991]. Introductory Computer Vision and Image Processing, Mc-Graw Hill International Editions (Computer Science Series), Singapore.

[4]     Fu, K.S., Gonzales, R.C., and Lee, C.S.G. [1987]. Robotics: Control, Sensing, Vision and Intelligence, Mc-Graw Hill International Editions (Industrial Engineering Series), Singapore.

[5]     Liu, Y.F. [1998]. A Unified Approach to Image Focus and Defocus Analysis PhD. Dissertation (Thesis), E.E. Dept., State University of New York at Stony Brook, New York.

[6]     Logitech Corporation [2000]. QuickCam Software Development Kit Documentation (Win32) , http://developer.logitech.com/.

[7]     Intel Corporation [2002]. The OpenCV Reference Manual, http://www.intel.com/research/mrl/research/opencv/.

[8]     Intel Corporation [2001]. Open Source Computer Vision Library Reference Manual, http://www.cs.nott.ac.uk/~jzg/nottsvision/OpenCV_new.pdf.

[9]     Bouguet, J.Y. [2002]. Camera Calibration Toolbox for Matlab Documentation, http://www.vision.caltech.edu/bouguetj/calib_doc/index.html.

[10] The MathWorks, Inc. [1997]. Image Processing Toolbox User’s Guide Version 2 (for use with MATLAB), MATLAB Documentation, The MathWorks, Inc., Natick, Massachusetts.

[11] Ray, Sidney [1997]. Applied Photographic Optics, Lenses and Optical Systems for photography, film, video and electronic imaging (Second Edition), Focal Press, U.K.

[12] Jacobson, R.E., Ray, S.F., Attridge, G.G., and Axford, N.R. [2000]. The Manual of Photography (Photographic and Digital Imaging), Ninth Edition, Focal Press, U.K.

[13] Pratt, W.K. [2001]. Digital Image Processing PIKS Inside, Third Edition, John Wiley & Sons, Inc., Singapore.

[14] Bovik, Al [2000]. Handbook of Image and Video Processing, Academic Press Series in Communications, Networking and Multimedia, Academic Press, U.S.A.

[15] Hartman, P. [2000]. Teach Yourself Visual Basic 6, IDG Books Worldwide, Inc., Foster City, California, U.S.A./IDG Books India (P) Ltd., New Delhi.

[16] Perry, G. and Hettihawa, S. [1998]. SAMS Teach Yourself Visual Basic in 24 hours Starter Kit, Macmillan Computer Publishing, USA/Techmedia, Darya Ganj, New Delhi.

[17] Herniter, M.E. [2001]. Programming in MATLAB, Brooks Cole Thomson Learning, Thomson Asia Pte Ltd, Singapore.

[18] de Souza, B.M.T. and Penton, R. [2002]. Game Programming All in One, The Premier Press Game Development Series, Premier Press, Inc./Prentice Hall of India Private Limited, New Delhi.

[19] Foley, J.D., Dam, A.V., Feiner, S.K., and Hughes, J.F. [1999]. Computer Graphics, Principles and Practice, Second Edition in C, The Addison-Wesley System Programming Series, Addison-Wesley Longman, Inc., Singapore.

[20] Subbarao, M., Choi, T., and Nikzad, A. [1992]. Focusing Techniques, Computer Vision Laboratory, Department of Electrical Engineering, State University of New York at Stony Brook, New York.

[21] Subbarao, M. [1989]. Determining Distances from Defocused Images of Simple Objects, Department of Electrical Engineering, State University of New York at Stony Brook, New York.

[22] Subbarao, M. and Wei, T.C. [1992]. Depth from Defocus and Rapid Autofocusing: A Practical Approach, Department of Electrical Engineering, State University of New York at Stony Brook, New York.

[23] Subbarao, M [1989]. Efficient Depth Recovery through Inverse Optics, Department of Electrical Engineering, State University of New York at Stony Brook, New York.

[24] Subbarao, M [1987]. Depth Recovery of Depth-map I: Differential Methods, Department of Electrical Engineering, State University of New York at Stony Brook, New York.

[25] Falkenhagen, Lutz [1995]. 3D Object Based Depth Estimation from Stereoscopic Image Sequences, Universitat Hannover, Institut fur Nachrichtentechnik und Informationsverarbeitung, Appelstr, Hannover, Germany.

[26] Asada, N., Baba, M. and Odi, A. [June 2001]. Depth from Blur by Zooming, Department of Intelligent Systems, Hiroshima City University, Hiroshima, Japan.

[27] Ghita, O. and Whelan, P.F. [September 1999]. Real-time 3-D estimation using depth from defocus, School of Electronic Engineering, Dublin City University, Dublin, Ireland.

[28] Yokoya, N., Shakunaga, T., and Kanbara, M. [1999]. Passive Range Sensing Techniques: Depth from Images, IEICE Trans. Inf. & Syst., Vol. E82-D No.3, March 1999.

[29] Roy, S. and Meunier, J. [1993]. Stereoscopic analysis of multiple images, Department d’informatique et de recherché opertionnelle, Universite de Montreal, Canada.

[30] Koch, R. [1995]. 3-D Surface Reconstruction from Stereoscopic Image Sequences, , Institut fur Nachrichtentechnik und Informationsverarbeitung, Appelstr, Hannover, Germany.

[31] Eisert, P. [2002]. Model-based Camera Calibration Using Analysis by Synthetic Techniques, Computer Vision & Graphics Group, Image Processing Department, Heinrich-Hertz-Institute, Berlin, Germany.

[32] Microsoft Developer Network (MSDN Online): http://msdn.microsoft.com/

[33] Kongsberd Simrad [1997]. Underwater TV Cameras Depth of Field Calculation (Technical Note): http://www.kongsbergsimrad.com/cameras/download/depth.pdf

[34] Depth of Field: http://www.cs.mtu.edu/~shene/DigiCam/User-Guide/990/EXPOSURE/EV-depth-of-field.html

[35] Exposure Ovierview: http://www.cs.mtu.edu/~shene/DigiCam/User-Guide/990/EXPOSURE/EV-overview.html

[36] McCabe, B [1998]. Depth of Field Info: http://members.aol.com/Photoinfo/dof.html

[37] Depth of Field: http://www.azuswebworks.com/photography/dof.html

[38] Feature Point Detection in Multiframe Images: http://www.utia.cas.cz/library/prace/20000011.pdf 

[39] 15-462 Computer Graphics I Lecture 15 (RASTERIZATION): http://www-2.cs.cmu.edu/~fp/courses/02-graphics/pdf-6up/15-raster.pdf 

[40] Norman Koren’s Photography Tutorial [Digital Cameras]: http://www.normankoren.com/digital_cameras.html 

[41] Norman Koren’s Photography Tutorial [Introduction to resolution and MTF curves]: http://www.normankoren.com/Tutorials/MTF.html 

[42] Norman Koren’s Photography Tutorial [Resolution and MTF curves in scanners and sharpening]: http://www.normankoren.com/Tutorials/MTF2.html 

[43] Norman Koren’s Photography Tutorial [Depth of Field and Diffraction]: http://www.normankoren.com/Tutorials/MTF6.html

[44] Green, B. [2002]. Raster Data Tutorial: http://www.pages.drexel.edu/~weg22/raster.html 

[45] Anjum, Z. [1998]. Drawing an image in grayscale: http://www.codeguru.com/bitmap/index.shtml

      [46]OpenCV-Open Source Computer Vision Library Community Message Archives & Files Sections (Yahoo! Group):
               http://groups.yahoo.com/group/OpenCV/


The information on this webpage is the intellectual property of Vivek Saxena and must not be reproduced in part or in totality in any form, without the explicit written permission of the author. If you wish to link to this page, please use the link www.geocities.com/fsairin/rangefinder01.html.