Validation report 001: MIDI-to-score Conversion Model

The goal of this project is to explore and understand the latest advancements in the automatic transcription of performance MIDI streams into musical scores, speci cally within the broader scope of Audio Music Transcription (AMT). We analyze the recent paper Performance MIDI-to-score Conversion by Neural Beat Tracking (Liu et al., 2022), which comprises two main components: rhythm quantization and score generation based on a Convolutional Recurrent Neural Network (CRNN) architecture.

We investigate the model’s behavior using sequence perturbations and a LIME-like approach.

We were able to discover certain artifacts of the score generation models, especially when it comes to velocity contribution to hand part assignment. The time signature model is not fully robust to alterations that should not have affect the output.

Unfortunately, the proposed methods have drawbacks and are not fully justified. Current XAI methods do not work well with symbolic music data in general. Developing more tailored and adaptable XAI methods for musical applications could contribute to improved model interpretability. One of the challenge would be to find a resonable and interpretable embedding of the pitch space that encodes musical features.

Link to original publication with a model: PERFORMANCE MIDI-TO-SCORE CONVERSION BY NEURAL BEAT TRACKING

Recommended citation: Mateusz Szymanski. (2024). "Performance MIDI To Score Automatic Transcription." Github: ModelOriented/CVE-AI.
Download Paper