Sound Pressure Estimation Method Using Traffic Cameras and Artificial Neural Networks

Lima, M. S. Universidade Federal do Rio de Janeiro/PEE/PADS, Brazil, mathlima@poli.ufrj.br

Pinto, P. C. C. Universidade Federal do Rio de Janeiro/PEE/PADS, Brazil, pedrocayres@poli.ufrj.br

Gomes, J. G. R. C. Universidade Federal do Rio de Janeiro/PEE/PADS, Brazil, gabriel@pads.ufrj.br

Torres, J. C. B. Universidade Federal do Rio de Janeiro/PEE/PADS, Brazil, julio@poli.ufrj.br

Abstract

Sound pressure in cities is an important information for urban planning. Measuring this information directly usually requires a complex microphone infrastructure, which may in turn require a considerable monetary investment and long term measuring periods. Cities usually have closed-circuit TV infrastructure to monitor traffic on streets, but such cameras do not have microphones. A new solution for sound pressure inference using closed-circuit TV infrastructure, i.e. for estimating sound pressure level from traffic images, is considered in this work. Specifically, we make a comparison between previously proposed non-temporal convolutional neural networks and our presently proposed solution, which uses long short-term memories (LSTM) to exploit temporal relationships within image blocks. The dataset is composed of 38 videos representing traffic under different conditions (day time, weather, and so forth), with an overall length of 995 minutes. In order to predict the average sound pressure from unseen image sequences (without audio), 130 model variations are trained using the images and audio signals from training videos within the dataset. Model performance is assessed using mean squared error and Pearson correlation between the predicted and target output signals, across ten different folds. The LSTM-based neural networks yield better results in comparison to the non-temporal architectures. The newly proposed models achieve estimation errors below those of the previously proposed models: 71.3% average correlation and mean-squared error equal to 1.14 (LSTM), in comparison to 60.2% and 1.47 (non-temporal). Regularization methods are essential for training, and the VGG16 convolutional neural networks yield the best results. Object detection architectures such as Faster R-CNN seem to have a potential for improving the prediction results. We conclude that predicting urban sound pressure from image sequences is possible. Future work topics include preparing new datasets, that represent different places and traffic viewpoints, and investigating how object detection may improve the results.

Keywords:

sound pressure

machine learning

convolutional neural networks

audiovisual signal processing

smart cities.

References

[1] UN General Assembly. Un general assembly, transforming our world : the 2030 agenda for sustainable development, 2015. URL https://www.refworld.org /docid/57b6e3e44.html

[2] Lisa Goines and Louis Hagler. Noise pollution: A modern plague, 2007. ISSN 00384348.

[3] Jing Ma, Chunjiang Li, Mei Po Kwan, and Yanwei Chai. A multilevel analysis of perceived noise pollution, geographic contexts and mental health in beijing. International Journal of Environmental Research and Public Health, 15, 2018. ISSN 16604601. doi:10.3390/ijerph15071479.

[4] Ravindra Khaiwal, Tanbir Singh, Jaya Prasad Tripathy, Suman Mor, Sanjay Munjal, Binod Patro, and Naresh Panda. Assessment of noise pollution in and around a sensitive zone in north india and its non-auditory impacts. Science of the Total Environment, 566-567, 2016. ISSN 18791026. doi:10.1016/j.scitotenv.2016.05.070.

[5] Alok Gupta, Anant Gupta, Khushbu Jain, and Sweta Gupta. Noise pollution and impact on children health, 2018. ISSN 09737693.

[6] Masayuki Senzaki, Taku Kadoya, and Clinton D. Francis. Direct and indirect effects of noise pollution alter biological communities in and near noiseexposed environments. Proceedings of the Royal Society B: Biological Sciences, 287, 2020. ISSN 14712954. doi:10.1098/rspb.2020.0176.

[7] Juan Miguel Barrigón Morillas, Guillermo Rey Gozalo, David Montes González, Pedro Atanasio Moraga, and Rosendo Vílchez-Gómez. Noise pollution and urban planning, 2018. ISSN 21986592.

[8] L. O. Mazza, J. G. R. C. Gomes, and J. C. B. Torres. Noise intensity prediction from video frames using deep convolutional neural networks. Anais 23rd International Congress on Acoustics, Aachen, Alemanha, pages 4999–5006, 9 2019.

[9] L. O. Mazza. Sound pressure level prediction from video frames using deep convolutional neural networks. Master’s thesis, Federal University of Rio de Janeiro, Rio de Janeiro, 2019.

[10] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, pages 1–14, 2015.

[11] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016-December:770–778, 2016. ISSN 10636919. doi:10.1109/CVPR.2016.90.

[12] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the Inception Architecture for Computer Vision. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 2016-December, pages 2818–2826, 2016. ISBN 9781467388504. doi:10.1109/CVPR.2016.308.

[13] Yoshua Bengio, Patrice Simard, and Paolo Frasconi. Learning Long-Term Dependencies with Gradient Descent is Difficult. IEEE Transactions on Neural Networks, 5(2):157–166, 1994. ISSN 19410093. doi:10.1109/72.279181.

[14] Sepp Hochreiter and Jürgen Schmidhuber. Long Short-Term Memory. Neural Computation, 9(8):1735–1780, 1997. ISSN 08997667. doi:10.1162/neco.1997.9.8.1735.

[15] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision, 115(3):211–252, 2015. ISSN 15731405. doi:10.1007/s11263-015-0816-y.

[16] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards realtime object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6):1137–1149, 2017. doi:10.1109/TPAMI.2016.2577031.

[17] Tsung Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft coco: Common objects in context. volume 8693 LNCS, pages 740– 755. Springer Verlag, 2014. doi:10.1007/978-3-319- 10602-148.

[18] João Carreira and Andrew Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, 2017-Janua:4724–4733, 2017. doi:10.1109/CVPR.2017.502.

[19] Du Tran, Heng Wang, Lorenzo Torresani, Jamie Ray, Yann Lecun, and Manohar Paluri. A closer look at spatiotemporal convolutions for action recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 6450–6459, 2018. ISSN 10636919. doi:10.1109/CVPR.2018.00675.

[20] Xingjian Shi, Zhourong Chen, Hao Wang, Dit Yan Yeung, Wai Kin Wong, and Wang Chun Woo. Convolutional lstm network: A machine learning approach for precipitation nowcasting. Advances in Neural Information Processing Systems, 2015-Janua:802–810, 2015. ISSN 10495258.

[21] Alexey Bochkovskiy, Chien-Yao Wang, and Hong- Yuan Mark Liao. Yolov4: Optimal speed and accuracy of object detection, 2020. URL https: //arxiv.org/abs/2004.10934.