College of Engineering Unit:
An ongoing task of great interest for both the fields of AI and music information retrieval is the ability to automatically transcribe audio into musical notation. On one side, most musicians record their performances as raw audio through a microphone. While easy to play back, the notes and details cannot be discreetly read by a computer, or directly converted into sheet music. Notation based audio formats exist, mainly MIDI, but the issue then becomes how to transcribe the raw audio into this discrete, MIDI format.
Our project aims to be a conclusive first step in solving this difficult task. The MIDI Performance Data Website is a platform where the collection of audio-MIDI synchronized data can be crowdsourced to musician volunteers, and a place where the data needed for the main task can be openly searched for and acquired. Users can connect any MIDI-enabled digital instrument to their computer and navigate to the website, where they can record musical performances to contribute to the dataset. The platform records both the raw acoustic signal of their musical performance, as well as the discrete MIDI values being made by the instrument. With a large enough dataset, users can then search, filter, and download data of synchronized audio and MIDI files, which can be used to effectively train machine learning models to transcribe raw audio recordings into symbolic MIDI data.
Project Website(s):