Collaborative labeling protocol for polyphonic audio recordings from urban wetlands

Espejo, D. Institute of Acoustics, Faculty of Engineering Sciences, Audio Mining Laboratory (AuMiLab), Institute of Acoustics, Faculty of Engineering Sciences, Universidad Austral de Chile, Valdivia, Chile.

Gallardo, E. Institute of Acoustics, Faculty of Engineering Sciences, Audio Mining Laboratory (AuMiLab), Institute of Acoustics, Faculty of Engineering Sciences, Universidad Austral de Chile, Valdivia, Chile

Martínez, H. Institute of Acoustics, Faculty of Engineering Sciences, Audio Mining Laboratory (AuMiLab), Institute of Acoustics, Faculty of Engineering Sciences, Universidad Austral de Chile, Valdivia, Chile

Morales, G. Institute of Acoustics, Faculty of Engineering Sciences, Universidad Austral de Chile, Valdivia, Chile

Peñailillo, R. Institute of Acoustics, Faculty of Engineering Sciences, Audio Mining Laboratory (AuMiLab), Institute of Acoustics, Faculty of Engineering Sciences, Universidad Austral de Chile, Valdivia, Chile

Sandoval, C. Institute of Acoustics, Faculty of Engineering Sciences, Universidad Austral de Chile, Valdivia, Chile

Poblete, V. Institute of Acoustics, Faculty of Engineering Sciences, Audio Mining Laboratory (AuMiLab), Institute of Acoustics, Faculty of Engineering Sciences, Universidad Austral de Chile, Valdivia, Chile

Abstract

Large-scale manual labeling process is error-prone and time-consuming, whereby it is not a work to be done by one person. The present paper describes the development of a collaborative labeling protocol with the objective of generating manual annotations of polyphonic audios for the task of sound event detection. The audio recordings correspond to 1039 files each of 5 minutes of recording, which were collected over a period of one year (2019-2020) in three urban wetlands in the city of Valdivia, Chile. In this context of collaborative work, it is necessary to create a set of definitions that allow understanding and sharing concepts. Moreover, we propose a taxonomy that fits the audio data. Specifically, to carry out the labeling work, the open-source web tool called Label-Studio will be used. To understand the dataset based on the taxonomy proposed, it is expected to study the label distributions according to their: a) temporal information, such as hour, day, season; b) spatial, for example, geographic location and urbanization gradient; and c) origin of the sound source, in particular, biophone, anthropophone and geophone. The experience described here provides a contribution to collaborative research in the task of sound event detection. Also, it is expected to contribute to highlighting the importance of the urban wetland soundscapes for collective well-being.

Keywords:

Bioacoustics / Acoustical signal processing in biological systems: mathematical methods and algorithms.

References

[1] C. Rojas, E. Sepulveda, F. Jorquera, J. Munizaga, and J Pino. Accessibility disturbances to the biodi versity of urban wetlands due to built environment. City and Environment Interactions, 13, Jan 2022. doi:10.1016/j.cacint.2021.100076.

[2] P. Tourle. White noise: sound, materiality and the crowd in contemporary heritage practice. Interna tional Journal of Heritage Studies, 23(3):234–247, Mar 2017. doi:10.1080/13527258.2016.1261919.

[3] B. Kim and B. Pardo. A human-in-the-loop system for sound event detection and annotation. ACM Transactions on Interactive Intelligent Systems, 8(2, SI), Jul 2018. doi:10.1145/3214366.

[4] A. Hopson and F. de Szalay. Alteration of above and below-water soundscapes by roads. Wetlands, 41(1), Jan 2021. doi:10.1007/s13157-021-01407-8.

[5] H. Li, S. Yang, and W. Wang. Improved capsule routing for weakly labeled sound event detection. Eurasip Journal on Audio Speech and Music Pro cessing, 2022(1), Mar 7 2022. doi:10.1186/s13636 022-00239-6.

[6] X. Wu, Y. Qiao, X. Wang, and X. Tang. Bridging music and image via cross-modal ranking analysis. IEEETransactions on Multimedia, 18(7):1305–1318, Jul 2016. doi:10.1109/TMM.2016.2557722.

[7] S. Castano, A. Ferrara, and S. Montanelli. Leveraging crowd skills and consensus for col laborative web-resource labeling. Future Generation Computer Systems-The International Journal of Escience, 95:790–801, Jun 2019. doi:10.1016/j.future.2017.12.024. 17th International Conference on Collaboration Technologies and Sys tems (CTS), Orlando, FL, Oct 31-Nov 04, 2016.

[8] M. Cartwright, G. Dove, A. Méndez, J.P. Bello, and O. Nov. Crowdsourcing multi-label audio annotation tasks with citizen scientists. In Proceed ings of the 2019 CHI Conference on Human Factors in Computing Systems, page 1–11, New York, NY, USA, 2019. Association for Computing Machinery. doi:10.1145/3290605.3300522.

[9] R. Schapire and Y. Singer. Improved boost ing algorithms using confidence-rated predictions. Machine Learning, 37(3):297–336, Dec 1999. doi:10.1023/A:1007614523901. 11th Annual Con ference on Computational Learning Theory, Madi son, WI, Jul 24-26, 1998.

[10] J. Dortheimer. Collective intelligence in design crowdsourcing. Mathematics, 10(4), Feb 2022. doi:10.3390/math10040539.

[11] L. Wan, K. Tang, M. Li, Y. Zhong, and A. Qin. Collaborative active and semisupervised learning for hyperspectral remote sensing image classifica tion. IEEE Transactions on Geoscience and Remote Sensing, 53(5):2384–2396, May 2015. doi:10.1109/TGRS.2014.2359933.

[12] A. Lahiri, B. Paria, and P. Biswas. Forward stagewise additive model for collaborative multiview boosting. IEEE Transactions on Neural Networks and Learning Systems, 29(2):470–485, Feb 2018. doi:10.1109/TNNLS.2016.2623219.

[13] S. Morozov et al. A simplified cluster model and a tool adapted for collaborative labeling of lung cancer ct scans. Computer Methods and Programs in Biomedicine, 206, Jul 2021. doi:10.1016/j.cmpb.2021.106111.

[14] L. Band and K. Bharadwaj. An approach to en hance the quality of recommendation using collab orative tagging. International Journal of Compu tational Intelligence Systems, 7(4):650–659, 2014. doi:10.1080/18756891.2014.960225.

[15] S. Jazi, M. Kaedi, and A. Fatemi. An emotion-aware music recommender system: bridging the user’s in teraction and music recommendation. Multimedia Tools and Applications, 80(9):13559–13574, Apr 2021. doi:10.1007/s11042-020-10386-7.

[16] A. Dumitrache, O. Inel, B. Timmermans, C. Ortiz, R. Sips, L. Aroyo, andC. Welty. Empiricalmethodol ogy for crowdsourcing ground truth. Semantic Web, 12(3):403–421, 2021. doi:10.3233/SW-200415.

[17] F. Gontier, V. Lostanlen, M. Lagrange, N. Fortin, C. Lavandier, and J. Petiot. Polyphonic train ing set synthesis improves self-supervised urban sound classification. Journal of the Acoustical Society of America, 149(6):4309–4326, Jun 2021. doi:10.1121/10.0005277.

[18] W. Dhifli, S. Aridhi, and E. Nguifo. Mr-simlab: Scalable subgraph selection with label similarity for big data. Information Systems, 69:155–163, Sep 2017. doi:10.1016/j.is.2017.05.006.

[19] M. Cartwright et al. Seeing sound: Investigating the effects of visualizations and complexity on crowd sourced audio annotations. Proceedings of the ACM on Human Computer Interaction, 1(CSCW), Dec 2017. doi:10.1145/3134664.

[20] CrowdCurio. Audio-Annotator, 2017. URL ht tps://github.com/CrowdCurio/audio-annotator. An organization for crowdsourcing research at the University of Waterloo. Updated on 22 Aug 2017.

[21] R. Zhang. EchoML, 2020. URL https://github.c om/ritazh/EchoML. A web app to play, visualize, and annotate your audio files for machine learning. Updated on 4 Apr 2020.

[22] MIDAS Research Laboratory IIIT-Delhi. Audino, 2022. URL https://github.com/midas-research/aud ino. Open source audio annotation tool for humans. Updated on 1 Jun 2022.

[23] Manraj S. et al. Audino: A modern annotation tool for audio and speech. CoRR, abs/2006.05236, 2020. URLhttps://arxiv.org/abs/2006.05236.

[24] S. Ibarluzea et al. Universal Data Tool, 2022. URL https://github.com/UniversalDataTool/universal data-tool. Collaborate and label any type of data, images, text, or documents, in an easy web interface or desktop app. Updated on 3 May 2022.

[25] Heartex. Label-Studio, 2022. URL https://github.c om/heartexlabs. Data labeling and exploration tools for Machine Learning. Updated on 7 Jun 2022.

[26] M. Tkachenko, M. Malyuk, A. Holmanyuk, and N. Liubimov. Label Studio: Data labeling software, 2020-2022. URL https://github.com/heartexlabs/l abel-studio. Open source software available from https://github.com/heartexlabs/label-studio.

[27] Ministry of the Environment of Chile. Urban Wet land Inventory and Update National Wetland Catas tre, 2020. URL https://sinia.mma.gob.cl/. see Recent Studies Section.

[28] A.Farina andS. Gage. Ecoacoustics: The ecological role of sounds. John Wiley & Sons, 2017.

[29] R. Formas and L. Brieva. Population genetics of the chilean frog batrachyla leptopus (leptodactylidae). Genetics and Molecular Biology, 23(1):43–48, Mar 2000. doi:10.1590/S1415-47572000000100007.

[30] L. Brieva and R. Formas. Allozyme varia tion and geographic differentiation in the chilean leptodactylid frog batrachyla taeniata (girard, 1854). Amphibia-Reptilia, 22(4):413–420, 2001. doi:10.1163/15685380152770372.

[31] G. D’Elia, R. Beasley, S. Lance, K. Jones, and L. Bacigalupe. Development of polymorphic mi crosatellite markers for the pleuroderma thaul. Con servation Genetics Resources, 6(3):747–749, Sep 2014. doi:10.1007/s12686-014-0206-8.

[32] A. González, A. Vukasovic, and C. Estades. Tem poral variability in the abundance and diversity of aquatic birds in the Itata River wetland, Bio-Bio Re gion, Chile. Gayana, 75(2):170–181, 2011.

[33] L. I. Lunaschi, F. B. Drago, and R. Draghi. Dige neans and acanthocephalans of birds from Formosa Province, Argentina. Helminthologia, 52(1):17–27, Mar 2015. doi:10.1515/helmin-2015-0006.

[34] J. Cursach, J. Rau, and C. Tobar. Birds in a ma rine wetland of southern Chile. Revista de Biología Marina y Oceanografía, 45(3):441–450, 2010.

[35] C. Jara et al. Flora y Fauna de la Selva Valdiviana. Comité Ecológico Lemu Lahuen, Valdivia, Chile, 2nd edition, 2014. ISBN 978-9568667023.