TE-GAN: Thermal Image Enhancement using Generative Adversarial Network for Pedestrian Detection

Mohamed Amine Marnissi 1      Hajer Fradi 1      Anis Sahbani 2       Najoua Essoukri Ben Amara 1
1 LATIS-Laboratory of Advanced Technology and Intelligent Systems, 4023, Sousse, Tunisie;
2 Enova Robotics, 4023, Sousse, Tunisie;

The TE-GAN framework. we propose a new generative adversarial network for thermal image enhancement task. Our proposed architecture is composed of two main modules, the first one is proposed to improve the contrast of the image. Since, by augmenting the contrast, the noise will be gradually more visible, we mitigate this effect by a second module that aims at removing the underlying noise. Both models operate instantly and simultaneously in an end-to-end architecture.

Highlights

  1. In this paper we focus on the problem of pedestrians detection and localization from low resolution infrared cameras for surveillance applications.
  2. The proposed architecture is composed of two modules with a post-processing step: contrast enhancement and denoising modules and an edge restoration step. These different cues are merged together in one single architecture to complement each others and to further improve the overall quality.

Materials


Abstract


Infrared imaging has recently played an important role in a wide range of applications including video surveillance, robotics and night vision. However, infrared cameras often suffer from some limitations, essentially about low-contrast and blurred details. These problems contribute to the loss of observation of target objects in infrared images, which could limit the feasibility of different infrared imaging applications. In this paper, we mainly focus on the problem of pedestrian detection on thermal images. Particularly, we emphasis the need for enhancing the visual quality of images before performing the detection step. To address that, we propose a novel thermal enhancement architecture called TE-GAN based on Generative Adversarial Network, and composed of two modules contrast enhancement and denoising with a post-processing step for edge restoration in order to improve the overall image quality. The effectiveness of the proposed architecture is assessed by means of visual quality metrics and better results are obtained compared to the original thermal images and to the obtained results by other existing enhancement methods. These results have been conducted on a subset of KAIST dataset that we make available to encourage research in this direction1. Using the same dataset, the impact of the proposed enhancement architecture has been demonstrated on the detection results by obtaining better performance with a significant margin using YOLOv3 detector.

Results

1. Intermediate Results


2. Qualitative Results


Citation

@inproceedings{marnissi2021thermal,
  title={Thermal Image Enhancement using Generative Adversarial Network for Pedestrian Detection},
  author={Marnissi, Mohamed Amine and Fradi, Hajer and Sahbani, Anis and Amara, Najoua Essoukri Ben},
  booktitle={2020 25th International Conference on Pattern Recognition (ICPR)},
  pages={6509--6516},
  year={2021},
  organization={IEEE}}

Contact

If you have any question, please contact Mohamed Amine Marnissi at mohamed.amine.marnissi@gmail.com.