Adaptively partitioned block-based contrast enhancement and its application to low light-level video surveillance
© Lee et al. 2015
Received: 10 May 2015
Accepted: 7 August 2015
Published: 19 August 2015
This paper presents a dark region detection and enhancement method with low computational complexity for low-cost imaging devices. Conventional contrast enhancement methods generally have an oversaturation problem while brightness of the dark region increases. To solve this problem, the proposed method first divides an input image into dark object and bright background regions using adaptively partitioned blocks. Next, the contrast stretching is performed only in the dark region. The major advantage of the proposed method is the minimized block artifacts using optimally partitioned blocks using fuzzy logic and a refining step to accurately detect boundaries between two regions. Experimental results show that the proposed method can efficiently enhance the contrast of backlit images without the oversaturation problem. Because of low computational complexity, the proposed method can be applied to enhance very low light-level video sequences for video surveillance systems.
Recent advances in digital image processing systems enable users to easily acquire high quality images using compact, inexpensive digital cameras. However, a limited dynamic range is still a bottleneck of the camera technology (Debevec and Malik 1997). Because of the limited dynamic range, an image having both dark objects and bright background either loses object information or becomes over-saturated in the background region. An efficient image enhancement algorithm is required to enhance the contrast of the dark objects without over-saturation in the background.
Histogram equalization (HE) is a global contrast enhancement method for solving the unbalanced illumination problem in the image (Wang and Ye 2005). However, it tends to make the background saturated and amplifies the noise in the dark region of the image. For addressing this issue, several versions of improved HE algorithms have been proposed. The adaptive histogram equalization (AHE) method adaptively partitions the image into multiple sub-blocks for block-based local histogram equalization at the cost of blocking artifacts (Zimmerman et al. 1998). The bi-histogram equalization (BHE) method enhances the contrast of backlit images while preserving the average brightness. It is difficult to accurately separate background and object regions using a single threshold value to bisect the histogram (Kim 1997). The dualistic sub-image histogram equalization (DSIHE) method is similar to BHE except that the threshold value is selected using the median value of an image (Wan et al. 2003). As a result, DSIHE enhances the contrast of the images while preserving the mean brightness. However, the disordered histogram results in either over-saturation or under exposure. The recursive mean-separate histogram equalization (RMSHE) method performs iterative BHE for preserving the average brightness (Chen and Ramlli 1999). However, the effect of contrast enhancement decreases as the iteration continues. The gain controllable clipped histogram equalization (GC-CHE) method dynamically controls the clipping level of the histogram for appropriately re-distributing the dynamic range (Kim and Paik 2008).
For solving the problem of above mentioned global contrast enhancement methods, locally adaptive contrast enhancement methods have also been proposed. Kim et al. divided backlit and background regions using a set of optimal threshold values. Contrast enhancement is then performed only in the backlit region (Kim et al. 2013). However, blocking artifacts are generated in the boundary between the two regions. The retinex-based method can be considered as a locally adaptive contrast enhancement method that reduces the illumination dependency and stretches the dynamic range of only reflectance component using a Gaussian filter (Kim et al. 2011). However, if the size of the Gaussian filter is not appropriately selected, the processed image contains halo effect and color distortion.
As show in Fig. 1, the proposed method separates the image into the dark and background regions using adaptively partitioned blocks based on the optimal threshold value computed by fuzzy C-means clustering (FCM). More specifically, the proposed method partitions the input image into non-overlapped blocks of size \(64\times 64 \), and classifies them as dark, background, and ambiguous regions using the optimal threshold. The ambiguous blocks are further partitioned into four sub-blocks, which are then re-classified in the same manner. This partitioning process is repeated until the size of a block becomes \(4\times 4 \). Finally, the detected block-based dark region is refined using the guided filter for removing block artifacts in the enhanced image region (He et al. 2010). The filtered dark region is enhanced by contrast stretching, and the final output is obtained by fusing the enhance dark and input background regions.
This paper is organized as follows. “Adaptively partitioned block-based dark region detection and enhancement” section presents the automatic object segmentation algorithm, and “Experimental results” section presents experimental results, and “Conclusion” section concludes the paper.
Adaptively partitioned block-based dark region detection and enhancement
Because of the limited dynamic range of a digital camera, many consumers photographs are subject to backlit image degradation. The backlit image has a bi-modal histogram where one mode corresponds to the dark range and the other to the bright range in the background region.
In order to selectively enhance the contrast with over-saturation problem, the dark backlit region is accurately detected using adaptively partitioned blocks. Contrast of the backlit region is then enhanced while preserving the brightness of the background region.
Adaptive backlit region detection
Contrast enhancement of the dark region
This section presents experimental results of the proposed adaptively partitioned block-based contrast enhancement method. The proposed enhancement method first transforms the input color image into the hue-saturation-value (HSV) color space, and performs contrast enhancement for only V channel, which contains brightness information. To evaluate the effectiveness of the proposed method, this paper compared the proposed method with gain-controllable clipped histogram equalization (GC-CHE) (Kim and Paik 2008) and multi scale Retinex with color restoration (MSRCR) method (Jobson et al. 1997). The MSRCR method restores the color contrast by applying the conventional Retinex method to each RGB channel.
A low light video containing a vehicle head lamp or streetlamps has a bi-modal histogram similar to the backlit image by the sum. Therefore, the proposed method can also be used to enhance such low light video sequences.
Table 1 shows AE value of various contrast enhancement methods on the set of three test images. The higher AE value indicates that more detail of the image is restored by the enhancement method. According to Table 1, the proposed method produces higher AE value than any other conventional methods.
The gain-controlled clipped histogram equalization (GC-CHE) method (Kim and Paik 2008) is compared with the proposed method since it is known as the best histogram modification approach in enhancing the contrast of digital images. Although various improved or modified versions were proposed in the literature, the original work of Kim and Paik (2008) is the best candidate of the performance comparison without significantly increasing the computational load. The multi-scale retinex method (Jobson et al. 1997) is also known as the first work or retinex theory-based contrast enhancement method. Although various different versions of retinex-based methods were proposed in the literature, the original work of Jobson et al. (1997) is the most appropriate for performance comparison with similar amount of computational load.
In order to demonstrate the performance of the proposed method, we first used subjective comparison as shown in Fig. 8. The input low-light video frames shown in Fig. 8a is suitable to evaluate the performance of contrast enhancement since it contains both dark and saturated regions. In addition to subjective comparison, we evaluated average entropy to represent how evenly the brightness is spread in the processed image.
This paper has presented a method to enhance the contrast of two-mode brightness image. Conventional contrast enhancement methods have over-saturation and color distortion problems. To solve these problems, the proposed method divides the image into dark and background regions using adaptively partitioned blocks by two optimal threshold values computed by fuzzy C-means clustering in the V channel of the HSV color space. The proposed contrast stretching process is performed only in the detected dark region. The major advantage of the proposed method is the minimized block artifacts due to adaptively partitioning the image according to the optimal threshold and the refining step to detect the dark regions. The proposed method automatically segments backlit region and the background region. It does not need manual seed region selection for segmentation and has low segmentation complexity than heavy segmentation methods such as graph cut-based method. Experimental results showed that the proposed method can better enhance the contrast than existing methods in the sense of both minimizing over-saturation in the bright background region and preserving details in the dark region.
SL initiated the research, and developed major algorithms, NK performed experiment, and JP wrote the paper. All authors read and approved the final manuscript.
ICT R&D program of MSIP/IITP. (14-824-09-002, Development of global multi-target tracking and event prediction techniques based on real-time large-scale video analysis), and the Technology Innovation Program (Development of Smart Video/Audio Surveillance SoC and Core Component for Onsite Decision Security System) under Grant 10047788.
Compliance with ethical guidelines
Competing interests The authors declare that they have no competing interests.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
- Chen S, Ramlli A (1999) Contrast enhancement using recursive mean-separate histogram equalization for scalable brightness preservation. IEEE Trans Consum Electr 45(1):68–75View ArticleGoogle Scholar
- Debevec P, Malik J (1997) Recovering high dynamic range radiance maps from photographs. Proc ACM SIGGRAPH 21:369–378Google Scholar
- Gonzalez RC, Woods RE (2008) Digital image processing, 3rd edn. Prentice Hall, Upper Saddle RiverGoogle Scholar
- He K, Sun J, Tang X (2010) Guided image filtering. European conference on computer vision, vol 6311, pp 1–14Google Scholar
- Jobson D, Rahman Z, Woodell G (1997) A multi-scale retinex for bridging the gap between color images and the human observation of scenes. IEEE Trans Image Process 6(7):965–976View ArticleGoogle Scholar
- Kim Y (1997) Contrast enhancement using brightness preserving bi-histogram equalization. IEEE Trans Consum Electr 43(1):1–8View ArticleGoogle Scholar
- Kim K, Bae J, Kim J (2011) Natural HDR image tone mapping based on retinex. IEEE Trans Consum Electr 57(4):1807–1814View ArticleGoogle Scholar
- Kim T, Paik J (2008) Adaptive contrast enhancement using gain-controllable clipped histogram equalization. IEEE Trans Consum Electr 54(4):1803–1810View ArticleGoogle Scholar
- Kim N, Lee S, Chon E, Hayes MH, Paik J (2013) Adaptively partitioned block-based backlit image enhancement for consumer mobile devices. In: Proceedings IEEE international conference consumer electronics, pp 393–394Google Scholar
- Paris S, Kornprobst P, Tumblin J, Durand F (2008) Bilateral filtering: theory and applications. Found Trends Comput Graph Vis. 4(1):1–73View ArticleGoogle Scholar
- Shen S, Sandham W, Granat M, Sterr A (2005) MRI fuzzy segmentation of brain tissue using neighborhood attraction with neural-network optimization. IEEE Trans Inf Technol Biomed 9(3):459–467View ArticleGoogle Scholar
- Srinivasan S, Balram N (2006) Adaptive contrast enhancement using local region stretching. In: Proceedings of the 9th Asian symposium on information display, pp 152–155Google Scholar
- Wan Y, Chen Q, Zhang B (2003) Image enhancement based on equal area dualistic sub-image histogram equalization method. IEEE Trans Consum Electr 49(4):1301–1309View ArticleGoogle Scholar
- Wang C, Ye Z (2005) Brightness preserving histogram equalization with maximum entropy: a variational perspective. IEEE Trans Consum Electr 51(4):1326–1334View ArticleGoogle Scholar
- Zimmerman J, Pizer S, Staab E, Perry J, McCartney W, Brenton B (1998) An evaluation of the effectiveness of adaptive histogram equalization for contrast enhancement. IEEE Trans Med Imag 7(4):304–312View ArticleGoogle Scholar