Activity 09: Applications of Morphological Operations - Playing Notes by Image Processing
Listening to music has been one of my practices when I'm studying. It gives the sense that you are not alone. There is no deafening silence.
One to translate music is through a score sheet. A score sheet with notes will serve as guide to any musician to play the music. To produce these notes, any instrumental device is usually used. (For such, I prefer a piano).
However, we can also play music using image processing! Imagine that. Instead of just programming with recognizing text like from the previous activity, you can also play notes so long as you know the frequency and duration of each note.
The first part was to find the score scheet of a piece. A musica score for "Bahay Kubo" was my choice. The sheet is shown below.
The sheet was then divided into 3 parts, with removed clefs, numbers, and words. This is shown below.
The image was then binarized with appropriate threshold. The notes and line became white, while the background became black. Now the next problem was to identify which note is which.
The distinguishing factor of each note would determine its duration and frequency. The frequency corresponds to which note it is, therefore its position with respect to the lines. The duration on the other hand correspond to the shape of the note where a quarter note is fully shaded note, half note for hollow note, and eighth note for fully shaded note with horizontal line.
Now, what I did was compare a note with the whole image to determine which positions would contain a specific note. Two images are needed, the scoresheet image and the note. The image of the note was then filtered from the scoresheet image. For example, the note G4 was filtered out accordingly as shown in the figure below.
Afterwhich, the filtered image was dilated, and the blobs were identified. The centroid of each blob was saved, as well as the corresponding frequency and duration for this note. This was done to all the notes present in the song.
Having all the identified notes for a specific sheet, the x axis value of the centroid of the notes were sorted in increasing order to identify the correct order of notes. We now have the order of notes with specific frequency and duration. Using the functions note(f,t) and t=soundsec(t0), the music was played.
The original music as read is:
The music recovered through image processing of the music sheet is:
One of the issues encountered was that some notes were mistakenly identified as a different note. Overall, it was an exciting activity to be able to play through the program. It was quite hard as it took us three meetings. I give myself a 9/10 for this activity. Though I was able to extract musical notes and play music from the musical sheet, I wasn't able to fully reproduce correctly the song.
Thanks Aimee for the talk! :P
One to translate music is through a score sheet. A score sheet with notes will serve as guide to any musician to play the music. To produce these notes, any instrumental device is usually used. (For such, I prefer a piano).
However, we can also play music using image processing! Imagine that. Instead of just programming with recognizing text like from the previous activity, you can also play notes so long as you know the frequency and duration of each note.
The first part was to find the score scheet of a piece. A musica score for "Bahay Kubo" was my choice. The sheet is shown below.
![]() |
Bahay Kubo music sheet (source:) |
![]() |
Region of interest (with notes for the song) without the clefs, numbers, and words |
The distinguishing factor of each note would determine its duration and frequency. The frequency corresponds to which note it is, therefore its position with respect to the lines. The duration on the other hand correspond to the shape of the note where a quarter note is fully shaded note, half note for hollow note, and eighth note for fully shaded note with horizontal line.
Now, what I did was compare a note with the whole image to determine which positions would contain a specific note. Two images are needed, the scoresheet image and the note. The image of the note was then filtered from the scoresheet image. For example, the note G4 was filtered out accordingly as shown in the figure below.
![]() |
First line of music sheet (top), and the filtered G4 quarter note (bottom) |
Having all the identified notes for a specific sheet, the x axis value of the centroid of the notes were sorted in increasing order to identify the correct order of notes. We now have the order of notes with specific frequency and duration. Using the functions note(f,t) and t=soundsec(t0), the music was played.
The original music as read is:
The music recovered through image processing of the music sheet is:
One of the issues encountered was that some notes were mistakenly identified as a different note. Overall, it was an exciting activity to be able to play through the program. It was quite hard as it took us three meetings. I give myself a 9/10 for this activity. Though I was able to extract musical notes and play music from the musical sheet, I wasn't able to fully reproduce correctly the song.
Thanks Aimee for the talk! :P
Comments
Post a Comment