[Back]

P. Knees:
"Towards Visual Interfaces to Sound and Music Retrieval";
Talk: National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba, Japan (invited); 2018-03-12.

From interviews with professional music producers and creators, i.e., practitioners and experts in sound retrieval, we find that existing methods to search for sounds in large collections such as sample databases are far from their actual needs, work practices, and intuition. This is stemming from a semantic mismatch, as computational representations do not reflect the users' mental concepts, which are often rooted in other domains, most prominently the visual domain.
In this talk, I will argue that the development of a search system that allows for visual queries to audio collections is desired by users and should be pursued in future research in audio retrieval. I report on a test of the idea of a sketch interface with music producers conducted in a semi-structured interview process by making use of a physical non-functional prototype. Finally, I review existing research, technology, and artworks that address bridging the conceptual gap between sound and vision, highlighting possible sources of inspiration and the next steps to be taken.