Evaluation
A. Statistical Analysis on Subjective Evaluation
Below you can see the results of subjective evaluation as stacked column charts. Each method was ranked by the participants from 1 to 5 (1 being the best) in terms of content preservation, style fit, and audio quality.
Content preservation
Style fit
Audio quality
B. Demucs Performance Analysis
The following tables shows the SDR, the metric to evaluate the performance of Demucs used for separating the stems of Strings-Piano and Clarinet-Vibraphone mixtures. The overall SDR is 7.36. You can listen to the isolated audio tracks in the Separation-Translation Pipline section.
| Clarinet-Vibraphone | Strings-Piano | |||
|---|---|---|---|---|
| Clarinet | Vibraphone | Strings | Piano | |
| Pirates of Caribbean Theme | 9.210 | 4.523 | 6.722 | 3.117 |
| My Heart Will Go on | 7.081 | 8.698 | 3.439 | 4.223 |
| Beethoven's String | 10.423 | 4.815 | 8.649 | 3.896 |
| Moonlight Sonata | 9.015 | 10.227 | 11.048 | 8.953 |
| Fur Elise | 3.647 | 3.529 | 3.672 | 3.101 |
| Brahms's Clarinet | 10.071 | 8.622 | 3.053 | 12.397 |
| Beethoven's Piano | 8.906 | 14.570 | 5.271 | 8.280 |
| Dvorak's String | 7.880 | 10.780 | 5.228 | 8.231 |
| Romeo and Juliet | 5.831 | 6.675 | 0.529 | 6.929 |
| Nuvole Blanche | 16.948 | 12.318 | 8.370 | 5.600 |
| Average SDR | 8.901 | 8.476 | 5.598 | 6.473 |