ZHAW researchers develop revolutionary method to digitise sheet music
The new scanning method for musical notes represents a milestone in computer-aided object recognition. A digital music stand can now convert printed sheet music into machine-readable notes twice as precisely as was possible before.
The Optical Music Recognition (OMR) method helps professional musicians to switch from printed to digital notes. This technology automatically converts printed sheet music into machine-readable notes and also interprets them. In order to optimise the procedure, researchers at the ZHAW School of Engineering have developed a new method. Using the Deep Watershed Detector, they can scan musical notes much more precisely than was the case with computer-aided image or text-recognition methods previously investigated.
“This method analyses a sheet of music as a whole instead of breaking it up into small portions. In this way, notes are recognised in context. A neuronal network with triple output assigns each pixel to a certain object,” explains ZHAW researcher and project manager Thilo Stadelmann. “This method has doubled our accuracy rate when scanning notes.”
300,000 pages of sheet music analysed
While automatic text recognition is already fairly advanced, research into scanning and interpreting musical notes has not been able to achieve satisfactory results so far.
“Text typically contains only a small number of symbols, whereas music consists of composites. Musical notes can be combined not only horizontally but also vertically,” says Stadelmann. “By combining sub-symbols, we can basically generate an indefinite number of symbols, for example chords made up from individual note heads, stems and flags.” Thilo Stadelmann and his team have analysed around 300,000 pages of sheet music in order to develop this new method. In doing so, the ZHAW researchers have created the largest fully annotated data set worldwide in the field of computer-aided object recognition – and they have made it publicly and freely accessible for further research. The new method developed at the ZHAW can thus be applied to other problems in computer-aided object recognition.
Research flows into app
The researchers tested their system with digital and printed sheet music for several months, continually improving it. Their development work, which is financially supported by Innosuisse, will be used for a concrete product in the future. A Swiss start-up company called ScorePad offers a digital music stand for tablets. This device can scan printed sheet music. The new app allows individual editing of the digitised notes. This includes zooming in and out, transposing to different keys and selecting musical instruments. The app also facilitates playing the music. It is no longer necessary to turn the pages by hand, because the software recognises the notes played with the help of the tablet’s integrated microphone and scrolls down automatically.