Bobkov A.V. - Image registration in the real time applications, страница 3

Possible amount of positions is 1024x1024, and we need at least1024x1024 of operations to check each of them. So we got 2^40, or 1.2x10^12operations – one quadrillion. It is uneasy task even for most powerful computers.So the effective computation algorithm is quite important.1.1.1.

Specifics of aerial image registrationThis research is devoted to the task of image registration. The primary areaof application is the task of searching the position of aerial or space image on thedigital map in real time. This problem often arises in navigation (observe-matchingnavigation) and in a different surface monitoring tasks. The particular area ofapplication puts specific conditions and limitations to the statement of the problem.- Significant map size.

This causes both significant computations foracceptable image registration, and significant memory to hold the map and thetemporary results of the computations. Since the image registration system is oftenan on-board device, it places a serious limitation in terms of computing power andthe amount of available memory.- Sensor parameters are often known or can be estimated in off-line mode.The geometric transformation to convert an observed mage into a map is a rigidtransformation without local distortions.

These factors can be used to reduce bothsize and dimensionality of search space in order to speed up calculations and tobound the amount of memory required.- Some of the important parameters like rotation and scale can be estimatedfrom other sources of information. For example, rotation can be found usingorientation information by inertial or magnetic compass; scale can be estimatedfrom altimeter data; projective transformation parameters can be fixed bymounting the camera on the gyro-vertical platform. This also limits thedimensionality of required transformation.- The brightness transformation always plays important role due to differentillumination conditions, which depends on the sun positions, time of year, weatherand so on.And finally, the most serious requirement is the working in a real time.The primary requirements for any real-time systems are performance,accuracy and reliability.1.

Performance. The system must analyse an image in a fixed time in orderto reserve time to take a decision and produce control signals. If the time limit isexceeded, the system cannot work properly (for example, the object we trace can11left the visual field, or object we must find appears on the skipped frames), orsystem stability may be dangerously reduced in the case when information is usedinside the control loop.2. Accuracy. Information extracted from the image must meet a prespecified accuracy.

A reduction in accuracy can also make a system fail since itcan be a problematic to tune manually the system parameters in a real-time.3. Reliability. A system is reliable when it can keep working to the specifiedaccuracy when external influence of different nature is present. This influence canbe noise, changing of visual conditions and perspective, parameter fluctuationswith respect to time and many others. A system that operates in real time mustreact adequately to such influences.It is easy to see that these three requirements are mutually exclusive: fastermethods can be less accurate and less reliable; more reliable and robust methodsmay have a lower accuracy and performance. The task is to provide suchcharacteristics that are acceptable for the concrete practical application in question.Indeed the relation between performance, accuracy and reliability is morecomplex.

For example, an increase in performance can provide analysis of moreframes in a given time. This increases both accuracy and reliability of the obtainedinformation, and can facilitate faster methods. In such a way, performance can beconsidered as a key factor that is affected in a complex way to the accuracy andreliability in a context of the challenge of a practical task and the availablemethods of image analysis.1.1.2. A brief review of image analysis tasksDynamic control using video images can be found in a wide range ofdifferent areas in mechanics and industry.

Thus there are a wide range of taskswhere an A number of classification schemes describing image extraction exist,and commonly used scheme is shown on fig.1.1.1.1.2.1. Recognition and identification of objectsIn relation to partial task requirements, this task can be also divided intoseveral groups.a) Looking for an object on a static imageThis is a classical image recognition task, which requires identifying theobject on the image as one of existing samples, usually using a function ofmaximum matching.

Real tasks are more complicated due to a presence of otherobjects and background with complex structure, overlapping by other objects,absence of priory information about object location, orientation, scale, illuminationetc. The task can be rewritten as a task of object position determination, the objectseparation from background and so on.12Fig.1.1.

Classification of image registration tasks13Object searching can be performed for the following identification (e.g.automatic recognition of fingerprints or iris), for measurement of objectcharacteristics (size, colour, texture, orientation etc.). Objects found can be alsoused to glue overlapped images (e.g.

obtaining a whole map by a series of photos),data merging from different sources (e.g. merging visible and infra red images inorder to enhance the image quality) and others.Tasks of this group can be reduced to task of searching one or severalsample objects on the image. The samples can be images, or sets of imagestructural elements, or even object parameter sets.b) Object tracingIn this task group the object to be found and recognised is moving, and itsmotion differs from the whole image motion. There are tasks of object positionmonitoring (object approaching, collision avoidance, following an object).

If thetask is solved for each frame pair, it transforms to a previous task. But in severalcases there are a set of limitation that can prevent to do this: for example, wholeobject or its significant part can move outside the visual field, or time limitationprevents each processing of frames at a fast enough refresh rate. Other approachesmust be used in this case.c) Trajectory reconstructionIn some cases it is impossible to identify the object perhaps due to lowimage quality or small object size. Nevertheless, the object can still be recognisedby the parameters of motion and trajectory.

In the simplest case there is only onemoving object on the image and it can be easy detected. In more complex cases allmoving objects must be monitored, and required object may be found by its motiondynamics.1.1.2.2. Tasks of self-motion determinationIn this group of task the motion of the camera (and the object that holds it)must be determined in relation to a visual surface – the scene. In regard to a motiontype, two different cases can be described. If the object moves mainly in a paralleldirection to a scene, and changes of distance to the scene can be neglected, this isknown as a flat task.

In this task, all parts of image have the same behaviour. Thistask often appears in navigation of flying and space object, which moves on fixedor slowly changing altitude, i.e. parallel to a earth surface. The task can arise inindustrial applications, when the tool with the camera moves over a workingsurface – for example, in tasks of microscopic scanning and monitoring.In the case when the motion is directed to a scene, then this is a threedimensional task. This task arises, for example, in mobile robot controlapplications, in tasks of automatic targeting and docking. The difficulty of this taskis that the motion parameters of object depend on the distance to them.

Anadditional task is necessity to measure the distances to the scene elements. Anadditional difficulty is the possible non-linear characteristics of the camera, whichwhen present is difficult to correct in real time.The first task deals mainly with image translations, rare with rotation andscaling. Often the rotation angle and scale is constant or can be found from other14sources. Second task deals with scaling first of all, since it allows estimation of thedistance to objects. Other transformations – translation and rotation – are lessimportant; often the rotation angle is known or constant, and translation is limitedby one direction, e.g. in the autonomous mobile navigation task.However, in common case more complex transformations must be taken intoaccount.

The typical reasons are that the visual scene can be not flat or observedfrom the small angle, or optical system often adds non-linear distortions. Theoverview of the different transformation affection can be found in [Brown 1992].1.1.2.3. Video scene analysisIn the common case, it can be required to solve both of the above mentionedtasks together. For example, in a mobile robot navigation task it is required notonly to monitor and correct the desired trajectory, but also avoid collisions withother objects, that stand or move, change direction in order to pass the obstacle,give a way to another moving object and so on. In many cases it is also required todetermine the relationships between detected objects, determine the character of itinteraction, and predict the future states of the scene.

This is a most complex task,and universal methods to solve them are unknown.In such a way, most image analysis tasks consider looking for a relationbetween two or more images. This can be either looking for an object position onan image (localisation task), or looking for a feature that is nearest to the selectedimage fragment (identification task).

These two tasks are tied: to recognise anobject, it must be firstly found, and vice versa, to find the object it is required toknow what exactly must be found, i.e. it is required to determine a parameter setthat belongs to desired object. The correspondence between localisation andidentification task allows applying solutions, found for one task, to another task.