Deep Learning for Optical Music Recognition

Research project in the area of Media Processing.

Keywords: Music, Document Analysis, Computer Vision, Machine Learning, Deep Learning, Pattern Recognition.

About this Project

Optical Music Recognition (OMR) aims at automatically processing written music scores, comparable to Optical Character Recognition for text, but significantly more complex. The goal is to teach the computer to "understand" music scores. The potential applications are manifold, reaching from digitization for preservation and enabling to edit music scores easily to simply playing music back or accompanying musicians that practice their performance.

OMR has been subject to research for many years and although there are even commercial products, many questions of OMR remain open. Especially when dealing with complex scenarios, like orchestral scores, handwritten drafts or deteriorated manuscripts, the state-of-the-art is far from perfect.

The goal of this project is to improve the state-of-the-art by applying machine learning, i.e. deep learning to reconstruct the visual information and the semantic interpretation. This should enable to build an extensible, robust OMR system, which is capable of handling complex scenarios with high accuracy, comparable to human performance.

Results

• Conception of a full-pipeline workflow for Optical Music Recognition with Deep Learning
• Collection of a list of datasets for Optical Music Recognition on Github.
• Development of a Universal Music Symbol Classifier for music symbols.
• Development of a Music Symbol Detector for detecting music symbols in scores.
• Journal paper on Understanding Optical Music Recognition