Design of object surveillance system based on enhanced f ish-eye lens
A new method is proposed for the object surveillance system based on the enhanced fish-eye lens and the high speed digital signal processor (DSP). The improved fish-eye lens images an ellipse picture on the charge-coupled device (CCD) surface, which increases both the utilization rate of the 4:3 rectangular CCD and the imaging resolution, and remains the view angle of 183? . The algorithm of auto-adapted renewal background subtraction (ARBS) is also explored to extract the object from the monitoring image. The experimental result shows that the ARBS algorithm has high anti-jamming ability and high resolution, leading to excellent object detecting ability from the enhanced elliptical fish-eye image under varies environments. This system has potential applications in different security monitoring fields due to its wide monitoring space, simple structure, working stability, and reliability.
OCIS codes: 110.2970, 220.3630, 220.4830, 250.0040.
doi: 10.3788/COL20090702.0142.
In recent years, the optoelectronic surveillance system based on charge-coupled device (CCD) imaging and digital image processing (DIP) has been widely applied in high security domain[1,2], which has raised significant economic and social efficiencies. However, because the limitation of its field of view, the CCD imaging system using the ordinary lens has a small monitoring area and the multi-system like the ommateum, which greatly increases the complexity and the cost of the system. A fish-eye lens which has the ultra imaging field has attracted a lot of attention and has been studied in many detection and survey systems. For example, Kir′aly et al. studied the ultra-wide-angle stereoscopic vision measurement system[3]. Courbon et al. researched and designed the general robot application model that used the fish-eye camera[4]. Nishimoto et al. studied the threedimensional (3D) measurement that used the fish-eye stereo vision[5]. Li designed the 360? full-view panorama image camera[6]. They all used the 180? ultra-wide-angle fish-eye lens and achieved the 180??360? panorama monitoring.
However, because of the characteristics of the 180? hemisphere space imaging, the traditional fish-eye lens images a circular picture on the CCD surface, as shown in Fig. 1(a). Since the CCD is usually a 4:3 rectangle, the circular image wastes CCD area and then reduces the CCD utilization rate, decreasing the resolution and quality of the image directly. In order to increase the utilization rate of the CCD area, some fish-eye cameras are used to image as far as possible to overspread the CCD but neglect the fringing field information, as shown in Fig. 1(b). This method improves the image quality and increases the CCD utilization rate, but reduces the monitoring information of the field. These lenses are usually used in the art photography. However, for a widefield visualization supervisory system, this method cannot match the requirement obviously. In order to take the full advantage of the information for fish-eye lens, we proposed a method to design an enhanced elliptical imag ing fish-eye lens which can increase the CCD utilization rate through controlling the distortion parameter of the normal fish-eye lens[7]. The imaging result is shown in Fig. 1(c). This method increases the utilization rate of the CCD and also improves the image quality, remaining the monitor area of 183? .
In this letter, an object surveillance system which uses the enhanced elliptical imaging fish-eye lens is designed. Because the system needs to process the image in real time, a high-speed processing system is required to calculate the image fleetly. The high-speed digital signal processor (DSP) is the best selection for this system[8,9] . By analyzing and calculating the image parameters, the alarm system can be controlled and monitor the region automatically.
We analyze the demand of the surveillance system in detail and construct the block diagram of the object surveillance system, as shown in Fig. 2. The system is segmented as three parts: the image gathering part that includes the enhanced fish-eye lens, the CCD sensor, and the image data high-speed buffer; the DSP processing part that includes the high-speed DSP chip and some assistant module; the output part that includes the alarm and the image storage module.
In this system, the enhanced fish-eye lens images an ellipse image on the CCD surface and the monitoring region of the hemisphere space achieves 183? , which is wide enough for the surveillance system. The elliptical picture has a high image quality and a high resolution, which paves a way for the next processing steps. The CCD module is the Alta series made by American Apogee Instrument Company, which can match the request of industrial applications. It can realize highaccuracy analog-to-digital (A/D) conversion and output 12- or 16-bit digital image data. The high-speed buffer is used to store the data transitorily if the data stream cannot be processed by DSP in time. The TMS320C6203 DSP chip is used as the image processing module[10]. It has some advantageous features such as high-speed internal memory and assembly line processing, etc. The assistant part is a signal processing unit of field programmable gate array (FPGA). It controls the DSP to read the image data, as well as the clock interface of the CCD module and the DSP chip.
We design the enhanced fish-eye lens via controlling the lens distortion. The control of optical distortion is useful for the design of a variety of optical systems usually. The f-θ lens is widely used in laser scanning systems to produce a constant scanning velocity across the image plane. During the last 20 years, many distortion control correctors have been designed[11,12]. Today, many challenging digital imaging systems can use optical distortion to enhance the imaging capability. A well-known example is a reversed telephoto type. If the barrel distortion is increased rather than being corrected, the lens is the so-called fish-eye lens.
A normal fish-eye lens images a circular distortion image. Usually, the correcting algorithm is used to correct the distortion of the image. In this letter, we increase the distortion instead of correcting it through controlling the distortion parameter of the fish-eye lens to image a bigger distortion elliptical image. In this way, we can achieve the enhanced fish-eye lens. In order to get a better control of the distortion and to reduce the time of optimization, the design process is divided into four steps. The first step is the design of an anamorphic fish-eye lens. The distortion is not controlled here and the view field of the ultra-wide-angle lens is only about 65%—70% of the desired field-of-view for the enhanced elliptical imaging fish-eye lens. The second step is the determination of a first approximation for the first surface of lens. At this stage, the distortion is entirely controlled with this surface alone and only the tangential chief rays are considered for the computation. The third step is a re-optimization of the configuration resulting from the second step. In order to avoid the divergence caused by large variations of the curvature of the first surface, the re-optimization is done simultaneously on many parameters of the configuration, including those of the first surface. The last step is also for optimization. In this step, most parameters are declared as variable and the merit function is constructed to allow the control of the distortion. The size of the spots is similar to that of the anamorphosis.
In fact, we can also explain the design principle with the entrance pupil of lens[13]. The distortion of a fisheye lens can be controlled or corrected by controlling the aperture stop position (entrance pupil) as well as other aberrations, as shown in Fig. 3(a). From a certain point of view, this entrance pupil shift is equivalent to a stop shift. The stop shift can be used to control the distortion. Since the front lens group is responsible to the larger amount of distortion, it can be used as the distortion controller and the impact of the rear group can be negligible on the final distortion profile.
The enhanced fish-eye lens images a bigger distortion elliptical image on the CCD surface. The utilization rate of the CCD area, the imaging resolution, and the capability of image information are increased obviously, as shown in Fig. 3(b).
DSP processing is also very important in this system. The high-speed DSP of TMS320C6203 and the FPGA serve as the control and processing center. The hardware configuration of the processing system is shown in Fig. 4.
The peak value of operational capability of TMS320C6203 is 2400 million instructions per second (MIPS). The internal procedure memory is about 384 kB and 512-kB internal data storage space is provided by 2 blocks. This DSP chip has 4 direct memory access (DMA) channels and an auxiliary DMA channel, and can support expansion bus[9,10] .
Detecting the object from the enhanced fish-eye image is a moving-object detection problem[14,15]. The detection system and the focus of the enhanced fish-eye lens are both at the stationary positions and the detection algorithm of auto-adapted renewal background subtraction (ARBS) is explored. The principle of ARBS is described as follows.
Firstly, the system gathers an image b(x) as the background after pre-processing. Then it gathers another image f(x, t) which is separated for a certain time t and also pre-processed. Subtracting the anterior background image, the resulted image is given as
I(x, t) = f(x, t) ? b(x). (1)
If the object is not found, the program will renew the background image with a later image automatically. However, if there is an object but the background changes, the background has to be updated too. We consider this problem in two different cases.
The first case is that the background changes gradually. If the illumination of the background changes slowly, we can use sufficient time slices before time t to update the background b(x). Define a weight function w(t) as
where G(·) is the Gaussian function with G(0) = 1.0 and G(±T1 × WW) = 0.01. In real implementation, G(·) is calculated off-line to generate a look-up table and the summation in Eq. (3) can be computed iteratively from time t ? 1 to time t.
The second case is that the background changes abruptly. We make the assumption that the images before and after an abrupt illumination change satisfy the following linear relation:
During the recovery period, the average difference between the spatial gradient f ′ (x, t) of the current image f(x, t) and that of the old background image b(x) is calculated as
where
If there is no object, dg(t) should be very small under the linear illumination change model of Eq. (4). Therefore, the image discrimination criterion is defined as
where max(Dg) is the predefined maximum gradient difference and the background is refreshed as
Therefore, the background image could be automatically updated in the above mentioned situations. The program flow of ARBS execution in DSP is shown in Fig. 5.
The DSP program was designed and debugged with CCS2.0 software. The FPGA unit controls the image data read from buffer to the exterior memory of the DSP. The image data of exterior storage spreads to the internal random access memory (RAM) through the DMA. The internal RAM is divided into two equal pieces and located at different bolcks. Therefore, the center processing unit (CPU) and DMA could work simultaneously with Ping-Pong method and guarantee the image data to arrive at the internal RAM in real time.
The software code is written in the C language primarily. We tested the running time of the program using a timer and carried on the optimization using the integrated optimizer of CCS. The optimized method included the use of internal integration function, the optimized rank establishment, the DSP resource distribution, the writing of the parallel code, the use of the data pack to reduce the memory read-write time, and so on. After the entire program optimization was completed, the system performances like the function integrity and the time were tested.
The designed prototype system is used to carry out the experiment. It gathered five images of 1024 × 768 pixels per second. After subtracting the background images, it was used to analyze whether there was the target point. The experimental results of the object detection are shown in Fig. 6.
The testing data in daytime, night, and close room are shown in Table 1. The result shows that this system has a very high detection rate and a low false rate to the object at any environment.
In conclusion, we proposed an object surveillance system based on the enhanced fish-eye lens and high-speed DSP. The enhanced elliptical fish-eye lens has high utilization rate of the 4:3 rectangular CCD sensor and higher image resolution than the traditional fish-eye lens, which paves a way to object recognition. The high-speed DSP is used as the processing module and the ARBS algorithm is adopted to detect the object. The experimental results indicate that this system has a very high detection rate and a low false alarm rate to the object in any environment. Furthermore, this surveillance system has a simple structure, wide monitoring space, and a high image resolution.