Project leader
I3S - Université de NicePartners
PARISSON, DEEZER, IRCAMFunders
ANR,
WASABI
Audio web and Semantic Web aggregate in the browser for indexing
Deezer, Spotify, Pandora or Apple-Music enrich listening by pieces of music by the biography or the albums of the same artist, and make suggestions to listen to other “close” works or artists (without the proximity not being defined). A radio operator journalist or presenter often uses the data of the Web and the press to prepare his emissions. A professor in Master degree in engineering uses tools for analysis to explain the production techniques with his students. All use knowledge bases going from most empirical (the press, Google) to the most formalized and accessible by machines (Spotify uses LastFM, MusicBrainz, DBPedia and of the audio extractors resulting from the startup The Echo Nest repurchased in 2014). The need for richer musical knowledge bases, and for tools of exploitation is important.
Project WASABI has as an originality to mix several approaches and proposes a method of enrichment of the results, this joint implementation which aims at producing a base of richer and better equipped knowledge:
1) By exploiting the databases of the Semantic Web (for example: DBPedia, MusicBrainz, LastFM) one can extract from the structured data, binding the song to elements such as the producer, the studio where it was recorded, the composer, the year, words, the Wikipedia page describing it, etc
2) By analyzing the data in free text (words of the song, text of pages related to this song), one can extract from the nonexplicit data (topics approached by the song, places, people, events, dates, emotions conveyed). The data obtained by these four methods can be dependent, confronted, confirmed or cancelled according to the assumptions, for example the description of a rock group and a producer can be used for configuring the initial parameters of the audio analysis and facilitating the démixage.
3) By using in a joint way this information resulting from the Semantic Web and the analysis of the words with the contained information in the audio signal one can improve the automatic extraction of musical information (the temporal structure, the presence and characterization of the voice, musical emotion or the presence of plagiarism, recovery).
4) When a song accessible in form is démixées (separate tracks) one can carry out a more precise audio analysis and extract from the richer data (notes, instruments, type of reverberation, etc). One will study in this project how the démixage can be obtained and how a result can be used even when it is imperfect in the context of the browser.
5) By supporting the serendipity and to find data noncommonplace with a tool like Discovery Hub (and to answer questions like: what connects Radiohead to Pink Floyd?)
Project WASABI will specify the whole of these cases of use thanks to the presence of potential users of technologies (musical Deezer, RadioFrance, journalists, composers and musicologists). WASABI will propose a succession of software bricks open source and on-line services open-dated for:
1) the visualization of audio metadata of the results of Music Retrieval Information and the listening of tracks démixées in the browser by exploiting last technologies resulting from the API Audio Web (mixing real time, audio effects).
2) automatic treatment of texts of songs, recognition and named tying of entity, annotation and collaborative correction,
3) the access to a service Web equipped with API proposing an environment of study of musical similarities resulting from the audio on the one hand and semantic analyses on the other hand. These software bricks will be used for the development of the demonstrators formalized with our collaborators, using the new standard API Audio Web and thus allowing the development of musical applications accessible to the general public since a web browser.