Bobkov A.V. - Image registration in the real time applications, страница 2

Performance considerations............................................. 1185.1.2. Memory requirements...................................................... 1205.1.3. Accuracy...........................................................................1225.1.4. Reliability......................................................................... 1235.1.4.1.

Brightness and contrast variations............................. 1235.1.4.2. High spatial frequency noise......................................1235.1.4.3. Geometric distortions.................................................1245.1.4.4. Image type and contents.............................................1255.2. COMPARISON OF IMAGE REGISTRATION TECHNIQUES...........................1265.2.1. Line segment matching vs control point matching.......... 1265.2.2.

Line segment matching vs. graph matching..................... 1275.2.3. Line segment matching vs. area-based methods.............. 1275.3. CONCLUSION...............................................................................128CONCLUSION.................................................................................................... 129FURTHER RESEARCH....................................................................................

130CONTOUR DETECTION.......................................................................... 130FEATURE DETECTION........................................................................... 130IMAGE REGISTRATION.......................................................................... 130ANALYSIS OF IMAGE REGISTRATION METHOD IN THE PHYSICAL ENVIRONMENT131RELATED PUBLICATIONS............................................................................

1324REFERENCES.................................................................................................... 133APPENDIX A. EQUIPMENT FOR DYNAMICAL MODELLING..............137APPENDIX B. SOFTWARE FOR THE OFF-LINE IMAGE ANALYSIS...1425AbstractThis work is devoted to the task of image registration in real time. Theimage registration task is task of finding position of the one image on another.Images can be obtained at different time, from different sources and in differentconditions.The task of image registration is fundamental task and it is important formany practical applications, such like visual navigation and surface monitoring bysatellites and unmanned flying vehicles, road traffic analysis, different applicationsof computer vision in robotics and many others.

In this work we pay specialattention to the task of visual navigation. However, results obtained can be used inthe different areas.In the terms of visual navigation, the task can be described as follows.Suppose that we have a basic image (map) that is obtained before and known apriory, and the current image (frame), obtained from onboard video camera in realtime.

Our task is to find position of the current image on the basic image, or that isthe same, to find the area on the basic image that looks like the current image. Thistask is essential, for example, for unmanned flying vehicle navigation by givenpath or for inspecting given region, when other navigation methods (like inertialand global positioning system) cannot produce enough accuracy.The aim of this work is to develop a common approach to the task,investigate possible solutions, investigate its advantages and disadvantages,implement most promising algorithms, research its characteristics in differentconditions, compare them with existing ones, and, finally, make conclusion aboutpossibility to use them in practice.The basic idea of this work is to use image representation as a set of linesegments. Line segments are universal feature that are usually present on most ofnatural images in enough amounts, and they can be easily extracted and matched.This makes the approach very promising and popular for many researchers.However, the results already achieved are far from optimal.

The approach has bothadvantages and disadvantages, obvious and hidden. Our objective is to determinethe primary characteristics of this approach, discover the factors that affect them,research their behaviour in different conditions and for different data, comparethem with the characteristics of other methods used in practice and understand thepractical use of the approach.The primary strategy of the work is using of the Hough transform (HT).

HTis a common method that allows finding typical objects on the image. The specificrequirement of the task is high performance. However, you can not get extraperformance for free – the price is extra equipment or loss in accuracy andreliability. HT allows increasing in performance in exchange to extra memory forstoring temporary results. This looks promising because modern systems have a lotof relatively low cost memory.We will use the HT approach twice, once for line segment extraction andthen for matching them. Existing schemes of HT for line segment detection havegood reliability, but provides insufficient accuracy. The work proposes extended6scheme of HT that allow increasing the accuracy of line segment detection up toone pixel without loss of performance.

The accuracy becomes an important factorin enabling high reliability in further line matching.We then came back to the HT in a line segment matching task. Differentmatching schemes were analysed, and best of them tested and compared withexisting methods.The direct testing of proposed methods in a flying laboratory is veryexpensive and hard (imagine the software debugging process when every mistakecan cause loss of equipment), so another approach is used.

We reproduce the taskconditions in the robotic laboratory. We use a manipulator, which carries thecamera over the surface model, emulating the flying vehicle behaviour in theproper scale. This allows implementation of image analysis in the conditions thatare near to natural.The results of experiments show that proposed methods can be implementedin real time on existing equipment; they provide high accuracy and enoughreliability for use in practice. The methods are insensitive to the brightness andlight conditions that make them preferable in comparison with existing approaches.The results of the work expand the knowledge about HT, open new areaswhere it can be successfully implemented, open new ways to accurately detect ofgeometric objects and show the possibility of using Hough transforms to locaterandom shapes described by line segments.The results of the work can be used in different practical applications thatrequire accurate automatic visual navigation.

Other possible application areas areimage registration from different data sources, object detection in traffic analysistasks, object localisation in robot visual field and surely many others.7AcknowledgmentsI would like to use this occasion to thank many people, who helped meworking over this book.I would like to thank my supervisor Jeffrey Knight for the guidance and timespent on me,Konstantin Pupkov for the equipment and conditions for working,Evgeniy Lobusov for the priceless advices,MT and Inex for deep discussion of my ideas,Charlotte Kitson and Veena Vorra for help and support,and of course my wife Olenka who stimulated me to complete this work.8AbbreviationsARPSAutomated Robot Programming SystemDGDifferential GradientDGHTDynamic Generalised Hough TransformationEHTExtended Hough TransformationGHTGeneralised Hough TransformationHTHough TransformationIRImage RegistrationLSCLine Segment ComparisonLSPCLine Segment Pair ComparisonMATMedian Axis TransformationNMSNon-Maximum SuppressionNPNon-PolinomialPHTProbabilistic Hough TransformationRHTRandom Hough TransformationSHTStandard Hough TransformationTMTemplate Matching9CHAPTER 1.

INTRODUCTION AND LITERATURE REVIEW1.1. IntroductionThe use of video image for automatic control of dynamic object is a veryimportant task, which has attracted a significant amount of attention over the pastyears. On the one side this relates with the practical needs, and on the otherside – with the fast development of technology area, through the development ofcomputers with very high performance.The task of image registration can be described as follows. Suppose that wehave a basic image that is obtained before and known a priory, and the currentimage, obtained from video camera in real time.

Our task is to find position of thecurrent image on the basic image, or that is the same, to find the area on the basicimage that looks like the current image.The task of image registration is fundamental task and it is important formany practical applications, such like visual navigation and surface monitoring bysatellites and unmanned flying vehicles, road traffic analysis, different applicationsof computer vision in robotics and many others.

In this work we will take especialattention to the task of visual navigation in first of all. However, obtained resultscan be used in the different areas.Since the task must be solved in real time, the implemented algorithms mustprovide enough performance to process incoming image frame before then the nextone will arrive.

Usually it is about 25 frames per second; however, severalapplications require higher frequency (100 frames per second or may be evenmore). Another requirement related with system stability: computation delays mustprovide enough stability for the control loop. And finally, there are applicationspecific requirements. For example, if we looking for moving object, we mustprocess the frame before the object leaves the visual field. This gives a typicalcomputation time no greater than 50 milliseconds in the conditions, when theavailable hardware performance are significantly limited.A video image contains a huge amount of information, which can be usedfor analysis and control – much more than can be extracted from the other sensorysystems.

Image systems are today relatively simple, and can be exact, reliable, andflexible. Processing by way of different physical phenomena is also possible –image in infrared and ultraviolet diapasons, temperature and altitude field,microscopic and telescopic image. Video systems are capable of obtaininginformation in a wide range of conditions and in different applications.The main disadvantage of video system is related with its main advantage –the huge stream of information requires significant computation resources to beprocessed in real time.

A video sequence is a set of frames obtained with a smalldelay (usually it is 0,04 sec). The frame consists of a set of pixels – digitised values10of brightness at each point. So a camera with a tiny frame of 200x200 pixels andstandard speed of 25 frames per second will generate 200x200x25 = 1 Mb dataeach second. This is too much for the most of real time applications using areasonable amount of computer power.Imagine another situation. Suppose we have tiny map of 2048x2048 size,and trying to find the position of medium size frame 1024x1024 on it by directcomparison.