Research paper
Brasscribe: Transcribing Brass Recordings into Band Scores with Pretrained Models and Deterministic Rules
Abstract
We describe and evaluate Brasscribe, a system that transcribes recordings of brass ensembles into notated scores and parts for British-style brass band. Pretrained models for source separation, note transcription, fundamental-frequency estimation and beat tracking are combined, without further training, with deterministic rules for rhythm quantisation, pitch spelling, harmony reduction and arrangement. Each stage is evaluated on ChoraleBricks brass quartets (10 pieces), URMP brass chamber music (8 pieces), single-instrument recordings and synthesised material, with note-level precision, recall and F-measure as defined in mir_eval and with score-position accuracy for rhythm. The best transcriber reaches a mean onset F-measure of 0.56 at the standard ±50 ms tolerance (0.75 at ±100 ms) on the quartets and 0.69 (0.88) on the chamber pieces. Given reference beats, the quantiser places 95–99 % of notes at their notated position in the bar; with tracked beats it places 45–49 %, with large variation between pieces. End to end on single parts, at ±100 ms (the only tolerance stored for these suites), notes are detected with F = 0.72–0.80, whereas the written F-measure, which also requires the correct position in the bar, is 0.24–0.27. The tenor voice of a quartet is recovered with F = 0.31 (±100 ms), and the confidence marks flag at most 33 % of wrong notes. All rule thresholds were tuned on the evaluation sets and no brass-band recording with reference notes was available, so the results are upper bounds. Beat and metre tracking is the main obstacle to readable output.
Keywords: automatic music transcription, brass band, rhythm quantisation, beat tracking, source separation, evaluation.
Plain-language summary: In a brass ensemble, Brasscribe finds about two thirds of the played notes when the timing must be exact to 50 ms, and more than four fifths when 100 ms is allowed. It is much less reliable about where they belong in the bar, and its question marks warn about only some of the mistakes. The output is a draft to correct, not a finished part.
1 Introduction
Automatic music transcription (AMT) converts audio into a symbolic representation: which pitches sound, when they start and when they stop [1]. A musician needs more than a list of notes. A part is readable when its rhythms lie on a beat grid that can be counted, in bars of the correct length, with the pitch spellings a composer would choose; it is playable when it stays within the instrument's range, is written in its transposition, and does not cross neighbouring parts.
The British-style brass band is a demanding target. A contest score has 18 parts, from soprano cornet to B♭ bass and percussion, and Norwegian bands use the same instrumentation. Most of the instruments share one timbre family and double each other constantly. To our knowledge, no openly available transcription model has the band's instruments in its vocabulary, and none separates several players of the same instrument.
Brasscribe addresses the task with pretrained models that only listen (they separate sources, estimate notes and track beats) and with deterministic rules that write the notation. It runs on the musician's own devices; no recording is uploaded to a server. In the builds released at the time of writing, every score, including a solo part, is produced on the musician's computer on all platforms. The Android app transcribes a solo part on the device in builds that bundle the listening models, which the builds released so far did not; on iPhone, iPad and Mac the models are not yet included, and the Windows app has no on-device path.
The central finding is shown in Figure 1: the rules place nearly all notes correctly when the beats are known, and about half on average, with wide variation between pieces, when the beats are tracked from the audio.
The correct notes are given to the quantiser, so only the beat grid differs. A dot is one piece, the ringed dot the mean, the thick bar its 95 % confidence interval: with tracked beats the pieces spread over almost the whole scale.
Data table
| Measure | Beats given (from the score) | SD | 95 % CI | Beats tracked (Beat This!) | SD | 95 % CI |
|---|---|---|---|---|---|---|
| Right place in the bar | 99 % | 2.1 % | 97–100 % | 49 % | 32 % | 31–69 % |
| Right place within the beat | 99 % | 2.1 % | 97–100 % | 93 % | 14 % | 84–99 % |
| Right written length | 88 % | 7.3 % | 84–92 % | 77 % | 24 % | 61–88 % |
| Measure | Beats given (from the score) | SD | 95 % CI | Beats tracked (Beat This!) | SD | 95 % CI |
|---|---|---|---|---|---|---|
| Right place in the bar | 95 % | 2.5 % | 94–97 % | 45 % | 37 % | 21–70 % |
| Right place within the beat | 96 % | 2.3 % | 94–97 % | 83 % | 16 % | 72–92 % |
| Right written length | 54 % | 22 % | 41–69 % | 45 % | 23 % | 32–60 % |
eval/baselines.json, suites quant-chorales and quant-urmp; the pieces from per-piece.json.The contributions of this paper are:
- a description of a complete audio-to-score pipeline for brass ensembles that uses only pretrained models and deterministic rules (Section 3);
- a stage-by-stage evaluation on real brass recordings with reference notes, reported with the standard note-level metrics and with score-position metrics for rhythm (Sections 4 and 5);
- negative results and comparisons that determined the design, including models and settings that were rejected (Section 5.8);
- an error analysis and a statement of threats to validity, notably that all thresholds were tuned on the evaluation data (Sections 6 and 7);
- a page whose numbers and figures are generated from the regression baselines of the public repository, so that they cannot drift from the measurements (Section 9).
3 Method
3.1 Overview
Figure 2 shows the pipeline, and Table 1 lists the pretrained models and their roles. Learned models are used only for listening; every stage after the vote between the note models is deterministic, so identical notes always yield an identical score.
| Model | Role | Runs on | Licence |
|---|---|---|---|
| SwiftF0 [8] | f0 of a monophonic line: the basis of a solo part, and note offsets | Computer; Android phone in builds that bundle the models | MIT |
| Basic Pitch [5] | Second note estimate for every note; the only note model in a draft made on a phone | Computer; Android phone in builds that bundle the models | Apache-2.0 |
| Beat This! [16] | Beats and downbeats (checkpoint small0 on phones, final0 on the computer) | Computer; Android phone in builds that bundle the models | MIT, code and weights |
| MuScriptor medium [4] | Main note model for an ensemble or a mixture | Computer | CC BY-NC 4.0, gated; downloaded by each user |
| BS-RoFormer SW [11], [12] | Separates a pop or rock recording into stems before transcription | Computer | Code MIT; no licence stated for the weights |
| MVSep Mega-53 [13] | Extracts a soloist, the bass and the drums from an orchestral recording | Computer | Code MIT; no licence stated for the weights |
3.2 Routing
The user states what the recording is, and that answer selects the route. A solo part and a brass ensemble are passed directly to the note models, since an all-brass mixture offers a separator nothing to split. A pop or rock recording is first separated with BS-RoFormer SW, and each stem is transcribed. For a soloist with an orchestra, Mega-53 extracts the solo, the bass and the drums, and the orchestra is taken as the residual of the mixture.
3.3 Note consensus and confidence
The note models process the same audio, and their notes are clustered when they agree in pitch and onset. For a solo part, SwiftF0 supplies the notes and a second model confirms them; a note that SwiftF0 did not produce is discarded (Section 5.6 gives the evidence). A logistic regression then maps the evidence for each note to a probability that it is correct: which models produced it, its duration, the voicing confidence of the f0 contour at its start, and whether the line was separated from a mixture. A note below a threshold is marked with a question mark; neighbouring marked notes share one mark.
3.4 Beats, metre and quantisation
Beat This! tracks beats and downbeats on the full mixture. Rules select the metrical level a player would count (the tracker often follows half notes in slow music), repair bar lines where a beat was inserted or dropped, and label passages without a steady pulse as ad lib. The notes are then quantised: for each beat, the quantiser weighs how well a subdivision fits the onsets against how complex it is to read, at a resolution of 24 ticks per beat. Written durations are derived from where the f0 contour leaves the note, not from the performed duration.
3.5 Spelling, harmony and arrangement
Pitches are spelled for the whole ensemble at concert pitch, before transposition, and the key is estimated from the same notes. Dense accompaniment is reduced to the chord tones of each beat. Three arrangers write the parts: a full band, a small band, and a quartet whose inner parts follow part-writing rules (no crossing, no parallel fifths or octaves, chord tones only). Three difficulty levels are derived from the result.
All stages after the vote are implemented twice, in Python as the reference and in Rust for the apps, and a conformance suite requires identical output from the two.
4 Experimental setup
4.1 Datasets
No public brass-band recording with aligned reference notes was found. Table 2 lists the sets that were used instead. None of them is a brass band: the quartets have one player per part, recorded one at a time, and the largest set has 10 pieces.
| Set | Content | Size used | Licence | Gate |
|---|---|---|---|---|
| ChoraleBricks, brass quartets [21] | Chorales played by four separately recorded players (trumpet, flugelhorn, baritone, tuba), with aligned notes and score positions | 10 pieces, 1,887 notes; 93 single-instrument recordings | CC BY 4.0 | Continuous integration, from pinned annotations and cached model outputs |
| URMP (University of Rochester Multi-Modal Music Performance dataset), brass pieces [22] | Chamber pieces for two to five brass players; each part also recorded alone | 8 pieces, 5,129 notes; 144 single parts | None stated | Local only |
| Slakh, trumpet lead [23] | Synthesised songs with a trumpet melody over a band; used to compare routes, not for rhythm, since its notes are not quantised | 2 songs | CC BY 4.0 | Local only |
| Fast-note clips | Runs, repeated notes, trills and ornaments assembled from trumpet samples [24], [25], dry and convolved with a church impulse response [26], with an exact reference | 253 clips in continuous integration; 306 in all | Samples: unrestricted and CC0; impulse response: CC BY 4.0 | Continuous integration |
| One commercial recording | A soloist with an orchestra; no reference notes, the arranged output is frozen and must be reproduced exactly | 1 recording | Commercial; not published or distributed | Local only |
4.2 Splits and data leakage
No listening model was trained or fine-tuned; all six are used as published. Two components were fitted to our data. The confidence model (Section 3.3) was fitted on notes with reference labels from the sets in Table 2 and validated on a held-out half of the recordings. The thresholds of the rules (the quantiser's complexity penalties, the bar-line repair, the duration and staccato rules, the fast-note rules) were chosen while inspecting the same sets on which they are scored. There is no held-out brass set. The results in Section 5 therefore measure how well the rules fit these recordings and should be read as upper bounds for unseen recordings.
4.3 Metrics
All measures lie between 0 and 1; higher is better. The detection of a note and its placement in the score are measured separately. Note-level measures are computed with mir_eval.transcription [27], onset-only and, where stated, onset+offset. The primary onset tolerance is ±50 ms; ±100 ms is reported as a secondary, relaxed tolerance, and several suites store results at ±100 ms only, which is stated where it applies. An octave error is a reference note that remains unmatched while an estimated note one or two octaves away starts within the onset tolerance; the octave-error rate is the share of reference notes for which this holds.
-
Note-level precision, recall and F-measure
Notes are evaluated as in mir_eval [27], which implements the MIREX note-tracking convention [28]. An estimated note matches a reference note when its onset lies within ±50 ms of the reference onset and its pitch within ±50 cents; each note is matched at most once. With TP matched notes, FP unmatched estimated notes and FN unmatched reference notes:
P = TP / (TP + FP), R = TP / (TP + FN), F = 2PR / (P + R) (1)
Offsets are ignored unless stated. Where they are used (onset+offset F-measure), the offset must also lie within max(50 ms, 20 % of the reference note's duration). We also report F with a ±100 ms onset tolerance, and say so wherever it is used: softly attacked brass notes are detected late relative to their annotated onsets, and deviations of that size are absorbed when the notes are quantised.
-
Position and duration accuracy
Rhythm is scored on the N notes that have a position in the reference score. Let qi be the quantised position of note i (in ticks, 24 per beat) and si its notated position, after one global shift by a whole number of beats:
Abar = |{i : qi = si}| / N (2)
Position within the beat, Abeat, compares the positions modulo one beat, so it is unaffected by a displaced bar line. Duration accuracy is the share of notes whose written duration equals the notated one.
-
Written F-measure
The strictest measure combines pitch, onset and notation. It is the F-measure of (1) at a ±100 ms onset tolerance, with a matched note counted as a true positive only if its position in the bar also equals the score's:
TPw = |{matched i : position in the bar correct}|, Fw from TPw as in (1) (3)
Metre, bar phase and the quantisation grid must therefore all be correct for a note to count.
-
Metre accuracy
No time signature is annotated by the beat tracker, so metre is scored by bar length. For a set of parts, with L the bar length in quarter notes:
M = |{parts : Lwritten = Lscore}| / |parts| (4)
A tracker that follows the beat at half or double speed is not penalised by this measure.
-
Precision and recall of the confidence marks
A note whose estimated probability of being correct falls below a threshold is marked question mark. Let W be the written notes that are wrong (unmatched under (1)) and Q the marked notes:
P? = |Q ∩ W| / |Q|, R? = |Q ∩ W| / |W| (5)
The false-alarm rate is the share of correct notes that are marked. In the drawing three notes are wrong and two are marked, one of them correctly: R? = 1/3, P? = 1/2.
4.4 Protocol
Model outputs are computed once and stored; scribe bench rescores every suite from the stored outputs and fails when a value has moved by more than the tolerance recorded in eval/baselines.json. Results are means over pieces. Where the per-piece values are available we also report the sample standard deviation (SD) and a 95 % confidence interval (CI) of the mean, obtained with a percentile bootstrap over pieces (10,000 resamples, fixed seed) [29]; with 8 to 10 pieces these intervals are wide. No significance tests were carried out, and differences between conditions are reported descriptively. F-measure, precision and recall are given as proportions to two decimals, and shares as percentages (with one decimal below 10 %); the parity of converted models is given to three decimals because its threshold is 0.98. Precision and recall are reported beside the F-measure where the baselines store them or the scorer provides them for the ensemble suites (Table 3). Most figures read the file used by the gate; figures built from other files (the model-conversion reports, and a small set of values measured once, listed in the Appendix) state this in their source line.
5 Results
5.1 Note detection in ensembles
Figure 3 gives the onset F-measure of the two transcribers. On the quartets the best model reaches 0.56 at the standard ±50 ms tolerance (SD 0.07, 95 % CI 0.51–0.60) and 0.75 at the relaxed ±100 ms tolerance (SD 0.06, 95 % CI 0.71–0.78; range 0.60–0.81). Table 3 gives precision and recall beside the F-measure: at ±100 ms the best model finds 83 % of the played notes of the quartets, and 68 % of the notes it reports are correct. The difference between the two tolerances is large for brass, consistent with late detection of soft attacks. On the two synthesised songs with a trumpet lead, separating before transcription scored higher than transcribing the mixture; with two songs this is suggestive only.
Notes found in the whole recording, every part at once. A dot is one piece, the ringed dot the mean, the thick bar its 95 % confidence interval.
Data table
| Measure | Onset within ±50 ms (standard) | SD | 95 % CI | Onset within ±100 ms (relaxed) | SD | 95 % CI |
|---|---|---|---|---|---|---|
| MuScriptor medium | 0.56 | 0.07 | 0.51–0.60 | 0.75 | 0.06 | 0.71–0.78 |
| Basic Pitch | 0.47 | 0.06 | 0.44–0.51 | 0.60 | 0.05 | 0.57–0.63 |
| Measure | Onset within ±50 ms (standard) | SD | 95 % CI | Onset within ±100 ms (relaxed) | SD | 95 % CI |
|---|---|---|---|---|---|---|
| MuScriptor medium | 0.69 | 0.17 | 0.57–0.79 | 0.88 | 0.04 | 0.86–0.91 |
| Basic Pitch | 0.66 | 0.11 | 0.59–0.74 | 0.73 | 0.09 | 0.67–0.79 |
| Measure | Onset within ±50 ms (standard) | SD | 95 % CI | Onset within ±100 ms (relaxed) | SD | 95 % CI |
|---|---|---|---|---|---|---|
| Separated first, then MuScriptor | – | 0.70 | ||||
| MuScriptor on the whole mix | – | 0.56 | ||||
| Separated first, then Basic Pitch | – | 0.56 | ||||
| Basic Pitch on the whole mix | – | 0.47 |
eval/baselines.json, suites chorales-transcription, urmp-transcription and slakh-transcription; the pieces from site/shared/research/measured/per-piece.json.| Set and model | P, ±50 ms | R, ±50 ms | F, ±50 ms | P, ±100 ms | R, ±100 ms | F, ±100 ms | n |
|---|---|---|---|---|---|---|---|
| Brass quartets, MuScriptor medium | 0.51 | 0.63 | 0.56 | 0.68 | 0.83 | 0.75 | 10 pieces |
| Brass quartets, Basic Pitch | 0.37 | 0.64 | 0.47 | 0.47 | 0.81 | 0.60 | 10 pieces |
| Chamber brass, MuScriptor medium | 0.70 | 0.69 | 0.69 | 0.89 | 0.88 | 0.88 | 8 pieces |
| Chamber brass, Basic Pitch | 0.64 | 0.71 | 0.66 | 0.71 | 0.78 | 0.73 | 8 pieces |
5.2 Rhythm quantisation
Figure 1 (Section 1) isolates the rhythm stage by quantising the reference notes. With reference beats, 95–99 % of notes are written at their notated position in the bar. With tracked beats the mean falls to 45–49 %, and single pieces range from 3.5–100 %; the mean is therefore a poor summary. Position within the beat is more robust (83–93 %), so the rhythm inside a beat is usually readable even when the bar lines are displaced. One tuning step underlies the reference-beat results: scaling the quantiser's complexity penalties raised exact positions on the chorales from 89 % to 99 %. The step was tuned on these chorales, so the gain is a fit and not a test.
The tracked beats were also scored against the notated quarter-note pulse, with the F-measure of mir_eval.beat (tolerance ±70 ms; beats in the first 5 s are discarded, the default of mir_eval); Figure 4 gives the results. No annotated beats exist for these sets, so the reference is the quarter-note pulse interpolated from the notated positions of the reference notes. The values are therefore not comparable with published beat F-measures, which rest on annotated beats. The raw output of Beat This! reaches a mean of 0.82 (SD 0.16, 95 % CI 0.72–0.91) on the quartets and 0.78 (SD 0.17, 95 % CI 0.68–0.90) on the chamber pieces. A tracker that follows every second quarter note scores about 0.67 under this measure, and 5 of the 8 chamber pieces lie between 0.60 and 0.75. Read together with Figure 1, this indicates that the beats themselves are mostly found, whereas the metrical level and the bar are not. Downbeat F-measure and the continuity measures (CMLt, AMLt) were not computed, because the references contain no bar-start times.
The figure also includes a candidate tracker that is under evaluation and is not part of the released system, md_beat_this (model run1) [30]. Against the same pulse its F-measure is 0.85 and 0.79, close to that of Beat This!. Quantising the reference notes on its beats raises the mean position accuracy from 49 % to 53 % on the quartets and from 45 % to 57 % on the chamber pieces; single pieces move in both directions, and with 8 to 10 pieces and no test the difference is suggestive only. These values are computed from stored beat files and are not gated.
The tracked beats themselves, and what they do to the notes. A dot is one piece, the ringed dot the mean, the thick bar its 95 % confidence interval.
Data table
| Measure | Beat This! (in use) | SD | 95 % CI | md_beat_this, model run1 (candidate under evaluation) | SD | 95 % CI |
|---|---|---|---|---|---|---|
| F-measure against the notated quarter-note pulse, ±70 ms | 0.82 | 0.16 | 0.72–0.91 | 0.85 | 0.12 | 0.77–0.91 |
| Reference notes written at their notated position in the bar | 0.49 | 0.32 | 0.31–0.69 | 0.53 | 0.39 | 0.30–0.76 |
| Measure | Beat This! (in use) | SD | 95 % CI | md_beat_this, model run1 (candidate under evaluation) | SD | 95 % CI |
|---|---|---|---|---|---|---|
| F-measure against the notated quarter-note pulse, ±70 ms | 0.78 | 0.17 | 0.68–0.90 | 0.79 | 0.19 | 0.67–0.91 |
| Reference notes written at their notated position in the bar | 0.45 | 0.37 | 0.21–0.70 | 0.57 | 0.43 | 0.29–0.84 |
site/shared/research/measured/per-piece.json, computed by per_piece.py from the beat files stored beside each mixture; the position accuracy of Beat This! is the gated value of quant-chorales and quant-urmp.Written durations are examined separately in Figure 5, which compares three rules for deriving a written duration. The rule in use is the best of the three by a small margin, and the chamber-music result remains low under every rule. These values are higher than the duration accuracy in Figure 1 (54 % for chamber music) because the suites differ: there the duration is taken from the quantiser alone and must equal the notated one exactly, whereas here a duration rule is applied per part and a deviation of one tick is accepted.
Share of notes whose written length equals the score's, with the score's beats, under three rules for how long a note is.
Data table
| Measure | Chamber brass | Brass quartets |
|---|---|---|
| As long as it was played | 60 % | 88 % |
| Held to the next note when the gap is small | 63 % | 91 % |
| From where the pitch leaves the note (used) | 65 % | 93 % |
eval/baselines.json, suite durations (the chorale part runs in CI).5.3 A single part, end to end
Figure 6 evaluates the complete solo path on recordings of single parts, with the models available on a phone. Notes are detected well, but the written F-measure is about a quarter, and the bar length is correct for 24–60 % of the parts. A monophonic line offers a beat tracker little evidence: there is no bass and no percussion. Replacing the small beat-tracking checkpoint by the larger one (second view) changes the results little; no statistical test was applied to the difference.
The whole solo path with the models a phone has, scored against the player's own part: the same notes, counted first by pitch and onset, then also by their place in the bar.
Data table
| Measure | Detected: pitch and onset within ±100 ms | Written in place: also in the right place in the bar | Measured on |
|---|---|---|---|
| Chamber brass, single parts (URMP) | 0.80 | 0.27 | 144 parts |
| Chorales, single parts (ChoraleBricks) | 0.72 | 0.24 | 40 parts |
| Measure | Detected: pitch and onset within ±100 ms | Written in place: also in the right place in the bar | Measured on |
|---|---|---|---|
| Chamber brass, single parts (URMP) | 0.80 | 0.27 | 144 parts |
| Chorales, single parts (ChoraleBricks) | 0.72 | 0.27 | 40 parts |
eval/baselines.json, suite solo-beat-model (runs locally, not in CI).5.4 Single instruments
Figure 7 reports the solo path per instrument. Every brass instrument recorded alone is transcribed well, including the tuba, with an octave-error rate of at most 0.2 %. Without knowledge of the instrument, the solo path retains only notes in a cornet's range, so the tuba is lost entirely and the trombone retains 44 % of its notes; the f0 estimator is not the limiting factor. The second view compares SwiftF0 with Basic Pitch on the same recordings. Basic Pitch confirms the notes of SwiftF0, and on the tuba it agrees with 73 % of them, which leaves 9.5 unmarked wrong notes per 100 written notes, against 0.2 on the trumpet.
Onset F-measure at ±100 ms of the written solo line, per instrument, with a fixed cornet range and with the player's own.
Data table
| Measure | Instrument chosen: its own range | No instrument chosen: fixed E3–E6 | Measured on |
|---|---|---|---|
| Trumpet | 0.97 | 0.97 | 20 recordings |
| Flugelhorn | 0.96 | 0.96 | 20 recordings |
| French horn | 0.90 | 0.90 | 11 recordings |
| Baritone | 0.97 | 0.78 | 24 recordings |
| Trombone | 0.97 | 0.56 | 8 recordings |
| Tuba | 0.88 | 0.00 | 10 recordings |
| Measure | SwiftF0 | Basic Pitch | Measured on |
|---|---|---|---|
| Trumpet | 0.96 | 0.88 | 20 recordings |
| Flugelhorn | 0.96 | 0.96 | 20 recordings |
| French horn | 0.90 | 0.80 | 11 recordings |
| Baritone | 0.97 | 0.87 | 24 recordings |
| Trombone | 0.97 | 0.79 | 8 recordings |
| Tuba | 0.89 | 0.48 | 10 recordings |
eval/baselines.json, suite solo-instruments (runs in CI).5.5 Individual voices of an ensemble
Figure 8 asks whether the line of one player can be extracted from a quartet recording by the range of the instrument. The top voice is recovered and the tenor largely is not (F = 0.31 with the best model). The further views give the recall of each voice's notes anywhere in the transcription; these are upper bounds, and the lower two voices are the ones most often missing. For this reason the system writes only the melody and the bass of an ensemble from the audio and arranges the parts between them from the harmony.
One player's line picked out of the four-part mix by the range of their instrument, scored against that player's part.
Data table
| Measure | MuScriptor medium | Basic Pitch | Notes both found |
|---|---|---|---|
| Soprano (trumpet) | 0.92 | 0.36 | 0.80 |
| Alto (flugelhorn) | 0.83 | 0.38 | 0.55 |
| Tenor (baritone) | 0.31 | 0.25 | 0.28 |
| Bass (tuba) | 0.69 | 0.30 | 0.38 |
| Measure | MuScriptor medium | Basic Pitch |
|---|---|---|
| Soprano (trumpet) | 97 % | 90 % |
| Alto (flugelhorn) | 92 % | 89 % |
| Tenor (baritone) | 76 % | 82 % |
| Bass (tuba) | 72 % | 66 % |
| Measure | MuScriptor medium | Basic Pitch |
|---|---|---|
| Soprano (trumpet) | 84 % | 74 % |
| Alto (flugelhorn) | 77 % | 66 % |
| Tenor (baritone) | 48 % | 71 % |
| Bass (tuba) | 45 % | 46 % |
eval/baselines.json, suite seat-voices (runs in CI); the recall views from per-piece.json, computed with the gate's scorer but not gated.Figure 9 evaluates those arranged parts against the composer's. Given the harmony of the score, the quartet arranger chooses the composer's pitch class for about four notes in five, with 0.94 parallel fifths or octaves per 100 chord changes, against 0.79 in the chorales themselves. From a recording it matches about half, with 5.4 parallels per 100. That the arrangers never write an unplayable note or cross parts holds by construction and is not evidence of musical quality.
Share of the chorale's own alto and tenor notes that the quartet arranger also chose (same note name, any octave).
Data table
| Measure | From the score's harmony | From the recording (MuScriptor) | From the recording (Basic Pitch only) |
|---|---|---|---|
| Alto | 80 % | 53 % | 17 % |
| Tenor | 78 % | 43 % | 18 % |
eval/baselines.json, suites arrange and quartet-audio (the chorale parts run in CI).5.6 Agreement between models and the confidence marks
Figure 10 shows that agreement between models is strong evidence that a note is correct, provided that SwiftF0 is one of the models. This motivates the rule of Section 3.3. The marks derived from it are weaker (Figure 11): few correct notes are marked, but most wrong notes carry no mark, and on the French horn none did. The recordings in Figure 11 are those on which the confidence model was fitted, so performance on new recordings should be expected to be no better. The marks judge pitch and onset only; they say nothing about the most frequent error, a correct note at the wrong position in the bar (Section 5.3).
Share of the notes that are right, by which of the three models found them, on solo lines separated from a mix.
Data table
| Measure | Precision | Measured on |
|---|---|---|
| SwiftF0 and MuScriptor | 99 % | 110 notes |
| All three | 98 % | 193 notes |
| SwiftF0 and Basic Pitch | 91 % | 578 notes |
| SwiftF0 alone | 54 % | 297 notes |
| MuScriptor and Basic Pitch, not SwiftF0 | 36 % | 14 notes |
eval/baselines.json, suite solo-vote (runs locally); note counts from docs/shared/research/10-benchmark-results.md via measured.toml.Of a solo line's wrong notes, the share that carry a ?; and of its right notes, the share marked anyway. Each is a mean over the instrument's recordings.
Data table
| Measure | Wrong notes that carry a ? | Right notes marked ? anyway | Measured on |
|---|---|---|---|
| Trumpet | 33 % | 0.4 % | 20 recordings |
| Flugelhorn | 33 % | 0.2 % | 20 recordings |
| French horn | 0 % | 0.4 % | 11 recordings |
| Baritone | 20 % | 0.5 % | 24 recordings |
| Trombone | 32 % | 0 % | 8 recordings |
| Tuba | 18 % | 3.6 % | 10 recordings |
eval/baselines.json, suite solo-instruments, q_hit and q_false_alarm; counts from measured.toml.5.7 Fast passages
Figure 12 follows the notes of fast figures (runs, repeated notes, trills) through the stages of the solo path. The f0-based note segmentation retains about six notes in ten. With reference beats nearly all of these reach the score; with tracked beats more are lost at quantisation. Of the trills and two-note alternations, 41 % keep at least four fifths of their notes with reference beats and 27 % with tracked beats. The further views show the effect of the rules that were added for fast notes: recall of the figure's notes improved and remains below one half. The clips are assembled from samples and the rules were developed on them.
Share of the notes in a fast run, repeated-note figure or trill that are still there after each stage of the solo path.
Data table
| Measure | Beats given | Beats tracked (Beat This! small0) |
|---|---|---|
| 1. Detected by the f0 estimator (SwiftF0) | 59 % | 59 % |
| 2. In the solo line after the vote | 60 % | 60 % |
| 3. After snapping to the beat grid | 55 % | 47 % |
| 4. Written in the score | 55 % | 47 % |
| Measure | Before the fast-note rules | After |
|---|---|---|
| Notes of the fast figure written (recall) | 26 % | 44 % |
| All notes of the clip (F-measure) | 40 % | 54 % |
| Trills and alternations that keep four fifths of their notes | 11 % | 30 % |
| Measure | Before the fast-note rules | After |
|---|---|---|
| Notes of the fast figure written (recall) | 22 % | 36 % |
| All notes of the clip (F-measure) | 37 % | 48 % |
| Trills and alternations that keep four fifths of their notes | 9.5 % | 18 % |
eval/baselines.json, suite fast-notes (runs in CI); before and after from docs/brasscribe/plan/fast-notes.md, section 7.2, via measured.toml.5.8 Model and design comparisons
The comparisons that determined the design are summarised in Figure 13, Figure 14, Figure 15 and Figure 16.
Every model and setting that was run on the ten brass quartets, at the strict 50 ms tolerance.
Data table
| Measure | Onset within 50 ms | Onset and note end both right |
|---|---|---|
| MuScriptor medium (used) | 0.56 | 0.48 |
| Basic Pitch, recording retuned to A = 440 (used) | 0.48 | 0.33 |
| Basic Pitch, as recorded | 0.47 | 0.28 |
| MuScriptor large | 0.44 | 0.39 |
| MuScriptor medium, conditioned on brass instruments | 0.33 | 0.30 |
| MuScriptor large, conditioned on brass instruments | 0.28 | 0.25 |
eval/baselines.json, suite chorales-transcription (runs in CI).- Model size. MuScriptor large scored lower than medium on the quartets (Figure 13); medium is used.
- Instrument conditioning. Conditioning MuScriptor on brass instruments lowered its onset F-measure from 0.56 to 0.33; conditioning is not used.
- Retuning. Shifting the recording to A = 440 Hz before Basic Pitch raised its F-measure from 0.60 to 0.62 at ±100 ms; the combined result of both models did not change (0.76 and 0.76).
- Separation before transcription scored 0.70 against 0.56 on the mixture, on 2 synthesised songs (Figure 3, third view).
- Beat-tracker size made little difference on single parts (Section 5.3); phones keep the smaller checkpoint.
Each model alone on trumpet lines separated with Mega-53, and the line kept when two models agree. Onset F-measure at ±100 ms.
Data table
| Measure | Onset F-measure |
|---|---|
| SwiftF0 | 0.75 |
| Notes at least two models found | 0.73 |
| MuScriptor | 0.50 |
| Basic Pitch | 0.42 |
eval/baselines.json, suite solo-vote (runs locally).On a solo line that a separator has already extracted, the f0 estimator is the strongest single source and MuScriptor, the best model on mixtures, is far weaker (Figure 14).
The best single route against the combined vote of the models, on each kind of recording. Onset F-measure at ±100 ms.
Data table
| Measure | Best single route | SD | 95 % CI | Vote between the models | SD | 95 % CI | Measured on |
|---|---|---|---|---|---|---|---|
| Brass quartets | 0.75 | 0.06 | 0.71–0.78 | 0.76 | 10 chorales, 1,887 notes | ||
| Chamber brass | 0.88 | 0.04 | 0.86–0.91 | 0.88 | 8 pieces, 5,129 notes | ||
| Trumpet over a band (separated first) | 0.70 | 0.63 | 2 songs |
eval/baselines.json, suites consensus-chorales, consensus-urmp, consensus-slakh and the transcription suites.Retaining only notes on which two models agree does not improve the F-measure, and lowers it on a full band (Figure 15). The vote is therefore used to estimate confidence and not to select notes.
Three ways to choose which notes the melody may be drawn from, scored on the melody alone. Onset F-measure at ±100 ms.
Data table
| Measure | Trumpet over a band (Slakh) | Chamber brass | Brass quartets |
|---|---|---|---|
| Notes MuScriptor found (used) | 0.65 | 0.72 | 0.90 |
| Only notes both models found | 0.67 | 0.65 | 0.80 |
| Everything either model found | 0.36 | 0.55 | 0.36 |
eval/baselines.json, suite melody (runs locally).For melody extraction, the top line of the notes supported by MuScriptor is preferable to the top line of the union of both models on all three sets (Figure 16).
5.9 Computational cost
Figure 17 gives the time and memory of each model on one laptop, Figure 18 the time of each model call in one solo job, and Figure 19 the parity of the models converted to Core ML or to the Open Neural Network Exchange format (ONNX) with their originals. Whole jobs were timed on an Apple M-series computer (the notes do not record the model): a solo of 30 s took 21.4–23.2 s, and a band recording of 247 s took 96 s, of which MuScriptor accounted for 75 s. A converted model is enabled in the apps only if its output matches that of the original with F ≥ 0.98; three conversions failed this requirement, and each failure determined where a model may run. Time and memory were not measured on any phone or tablet, on a Windows computer, or on a computer with less memory.
Each model alone, converted for Apple devices and in its reference framework, on one Apple M5 Pro laptop with 48 GB. No phone has been timed.
Data table
| Measure | Converted (best conversion that passes) | Reference framework on the processor |
|---|---|---|
| SwiftF0 (pitch of one line) | 0.02 s | 0.13 s |
| Basic Pitch (notes) | 0.18 s | 0.44 s |
| Beat This! small0 (beats) | 0.05 s | 0.55 s |
| BS-RoFormer SW (separation, 6 stems) | 13 s | 126 s |
| Mega-53 (separation, 53 stems) | 29 s | 208 s |
| MuScriptor medium (notes), on the processor only | 97 s | 84 s |
| Measure | Converted (best conversion that passes) | Reference framework on the processor |
|---|---|---|
| SwiftF0 (pitch of one line) | 0.38 GB | 0.32 GB |
| Basic Pitch (notes) | 0.57 GB | 0.70 GB |
| Beat This! small0 (beats) | 0.49 GB | 0.48 GB |
| BS-RoFormer SW (separation, 6 stems) | 3.8 GB | 3.6 GB |
| Mega-53 (separation, 53 stems) | 21 GB | 14 GB |
| MuScriptor medium (notes), on the processor only | 5.0 GB | 2.5 GB |
convert/reports/*.json (made by the scripts in convert/; python3 convert/summarize.py prints them).Seconds each model call took in one solo job, as the engine runs them, each starting its own process.
Data table
| Measure | Seconds |
|---|---|
| Beats (Beat This!) | 1.8 s |
| Pitch of the line (SwiftF0) | 0.30 s |
| Notes (Basic Pitch) | 2.5 s |
| Notes (MuScriptor, graphics processor) | 9.1 s |
| Pitch contour for note ends | 0.30 s |
docs/shared/research/13-performance-audit.md via measured.toml.How closely a converted model gives the original's result: F-measure of its notes against the original's own output (for the beat tracker, of its beats and downbeats at ±50 ms).
Data table
| Measure | Parity with the original |
|---|---|
| SwiftF0, full precision | 1.000 |
| SwiftF0, half precision on the Neural Engine: fails | 0.370 |
| SwiftF0 split: only its middle on the Neural Engine | 0.999 |
| Basic Pitch, full precision | 1.000 |
| Basic Pitch, half precision: fails | 0.815 |
| Beat This! small0, half precision | 0.994 |
| BS-RoFormer SW, half precision | 0.999 |
| Mega-53, half precision | 0.999 |
| MuScriptor, ONNX | 1.000 |
| MuScriptor, Core ML half precision: fails a stricter test | 0.999 |
convert/reports/*.json, parity (the lower score where a report has two).6 Discussion
6.1 Error analysis
- Rhythm is the dominant error. On single parts the onset F-measure is 0.80 and the written F-measure 0.27 (Section 5.3). Most of the difference is attributable to the beat grid: with reference beats the same rules place 95 % of correct notes (Section 5.2).
- Metre. The bar length is wrong for most single parts. A part in triple time written in quadruple time has every bar line misplaced, whatever the quality of its notes.
- Inner voices. From a quartet, the tenor is recovered with F = 0.31 and the bass with 0.69, against 0.92 for the top voice (Section 5.5).
- Octave errors are infrequent. Of the reference notes of the quartets, 2.7 % were missed while MuScriptor produced a note one or two octaves away at the same time, and 11 % with Basic Pitch; for chamber music the rates are 1.2 % and 4.1 %. The count includes estimated notes that belong to another part an octave away and therefore overstates the error.
- Fast passages lose about four notes in ten before notation (Section 5.7).
- Ornaments are split into extra notes. On control clips that should remain one note, with tracked beats, a fall adds 0.20 notes per reference note, a scoop 0.30, a rip 0.85 and a wide downward vibrato 0.70; an ordinary vibrato adds 0.05. No control clip was notated as a trill.
- Triplets were examined only on computed onset times, not on recordings: with 15 ms of timing jitter, 97 % of triplet eighth notes are placed correctly. The evaluation sets contain no recorded triplets with a reference.
- Durations. With reference beats, 54 % of chamber-music notes and 88 % of chorale notes receive the notated duration (Section 5.2).
6.2 Threats to validity
- Tuning on the test data. The thresholds of the rules were selected on the recordings on which they are scored, and the confidence marks are evaluated on their fitting data (Section 4.2). The reported values are optimistic to an unknown degree.
- Sample size. The sets have 8 to 10 pieces, and the route comparison on a full band rests on 2 synthesised songs. The confidence intervals are correspondingly wide, and one agreement class contains 14 notes.
- Representativeness. The chorales are slow and rhythmically simple and were recorded one player at a time; neither they nor the chamber pieces resemble a full band in a hall.
- Synthetic material. The fast-note clips are assembled from single sampled notes, and the full-band songs are synthesised.
- Tolerance. Several suites (single parts end to end, voices, the vote on separated lines, melody, consensus) store results at a ±100 ms onset tolerance only, which is more lenient than the ±50 ms convention; where both exist, both are reported and ±50 ms is given first.
- Measurement. Timing results come from single runs on one computer; some values were measured once and are not gated (Appendix).
7 Limitations
- No brass-band recording with reference notes was evaluated; behaviour on a full band is not measured.
- Only one route is checked end to end, and without reference notes: the soloist-with-orchestra route must reproduce a frozen output. The solo, ensemble and pop routes are composed of the same tested components but have no end-to-end gate.
- In an ensemble score only the melody and the bass are transcribed; the inner parts are arranged.
- The confidence marks cover pitch and onset only and miss most wrong notes.
- Computational cost is known for one laptop and for no phone.
8 Conclusion and future work
On brass ensembles the best pretrained model reaches an onset F-measure of 0.56 to 0.69 at ±50 ms and 0.75 to 0.88 at ±100 ms (recall 0.63 to 0.69 and 0.83 to 0.88), and deterministic rules notate the notes correctly when the beat grid is correct. The evaluation indicates that the score is lost mainly at beat and metre tracking, in particular for single parts, and that inner voices of an ensemble are not recoverable with the models examined. The confidence marks are precise but have low recall.
Future work follows from these results: beat and metre estimation for monophonic lines, including a step in which the user confirms the time signature and the first downbeat; a held-out brass set, so that a tuned rule can be distinguished from one that generalises; confidence estimates that cover rhythm; brass-band data, both rendered from public-domain scores and recorded with corrected scores; and measurements of computational cost on phones.
9 Reproducibility statement
- Code. The system, the benchmarks and this page are in the public repository github.com/181192/brasscribe. This page was built from the commit named under the title.
- Commands.
pixi run scribe bench listlists the suites;pixi run scribe bench cpurescores all of them from stored model outputs and compares them witheval/baselines.json;pixi run python site/shared/research/per_piece.pyproduces the per-piece values;site/build.shbuilds this page and fails if a stated finding no longer holds. - Reproducible from the repository alone. The following results are rescored in continuous integration on every change, from pinned ChoraleBricks annotations and cached model outputs kept in the repository, and can be reproduced without downloading a dataset: the quartet views of Figure 3 and Figure 1, Figure 13, Figure 7, Figure 11, the first view of Figure 8, Figure 9, the stage view of Figure 12, and the chorale series of Figure 5. Figure 17 and Figure 19 are read from committed reports; rerunning them requires the models.
- Requiring a local download. Results on URMP (the chamber views, Figure 6), on Slakh and on separated stems (Figure 10, Figure 14, Figure 15, Figure 16) need the datasets and the models on a local computer; the per-piece values and the beat F-measure need the evaluation data as well. The frozen output of the commercial recording cannot be reproduced by a reader.
- Data availability. ChoraleBricks, URMP and Slakh are available from their publishers (Table 2). The commercial recording is not distributed.
- Models and scorer. All models are public; MuScriptor's weights require acceptance of its licence. The checkpoints are MuScriptor medium, Basic Pitch as published, SwiftF0 as published, Beat This! small0 and final0, BS-RoFormer SW and MVSep Mega-53; the package versions are pinned in
pixi.lock. Scoring uses mir_eval 0.8.2. - Not gated. The values listed in the Appendix were measured once; their sources and commands are recorded in
site/shared/research/measured/measured.toml.
10 Ethics and licence statement
- Model licences. MuScriptor's weights are licensed CC BY-NC 4.0; scores produced with them may be used non-commercially only, and each user accepts the licence and downloads the weights personally. No licence is stated for the BS-RoFormer SW and Mega-53 weights; they are downloaded by the user's computer from their publishers and are not redistributed by the project. SwiftF0, Basic Pitch and Beat This! are MIT or Apache-2.0 (Table 1).
- Dataset licences are given in Table 2. The repository contains small data derived from ChoraleBricks (CC BY 4.0) and one test file derived from 30 s of a URMP track.
- Copyright. No copyrighted music is published on this page or distributed with the apps; the built-in example is a public-domain hymn tune.
- Privacy. No recording, and no score derived from one, is sent to a server.
- Telemetry. ONNX Runtime, the Microsoft library that executes the models in the Android and Windows apps and in parts of the engine, reports its own usage to Microsoft with a device identifier.
THIRD_PARTY_NOTICES.mddocuments what is sent.
Acknowledgements
This work depends on the authors of the models and datasets cited above, who made them openly available. Parts of the software and of this text were written with the assistance of an AI coding assistant and reviewed by the author.
How to cite
K.-A. Kalliainen, “Brasscribe: Transcribing brass recordings into band scores with pretrained models and deterministic rules,” The Brasscribe project, 2026. [Online]. Available: https://github.com/181192/brasscribe
@misc{kalliainen2026brasscribe,
author = {Kalliainen, Kristoffer-Andre},
title = {Brasscribe: Transcribing Brass Recordings into Band Scores
with Pretrained Models and Deterministic Rules},
year = {2026},
howpublished = {The Brasscribe project},
url = {https://github.com/181192/brasscribe}
}
References
- E. Benetos, S. Dixon, Z. Duan, and S. Ewert, “Automatic music transcription: An overview,” IEEE Signal Processing Magazine, vol. 36, no. 1, pp. 20–30, 2019, doi: 10.1109/MSP.2018.2869928.
- J. Gardner, I. Simon, E. Manilow, C. Hawthorne, and J. Engel, “MT3: Multi-task multitrack music transcription,” in Proc. Int. Conf. on Learning Representations (ICLR), 2022, arXiv: 2111.03017.
- S. Chang, E. Benetos, H. Kirchhoff, and S. Dixon, “YourMT3+: Multi-instrument music transcription with enhanced transformer architectures and cross-dataset stem augmentation,” in Proc. IEEE Int. Workshop on Machine Learning for Signal Processing (MLSP), 2024, pp. 1–6, doi: 10.1109/MLSP58920.2024.10734819.
- S. Rouard, M. Krause, A. Roebel, C.-J. Simon-Gabriel, and A. Défossez, “MuScriptor: An open model for multi-instrument music transcription,” in Proc. Int. Society for Music Information Retrieval Conf. (ISMIR), 2026, arXiv: 2607.08168. Code: MIT. Weights (muscriptor-medium): CC BY-NC 4.0, gated.
- R. M. Bittner, J. J. Bosch, D. Rubinstein, G. Meseguer-Brocal, and S. Ewert, “A lightweight instrument-agnostic model for polyphonic note transcription and multipitch estimation,” in Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 781–785, doi: 10.1109/ICASSP43922.2022.9746549. Software: Basic Pitch, Apache-2.0.
- C. Hawthorne, E. Elsen, J. Song, A. Roberts, I. Simon, C. Raffel, J. Engel, S. Oore, and D. Eck, “Onsets and Frames: Dual-objective piano transcription,” in Proc. Int. Society for Music Information Retrieval Conf. (ISMIR), 2018, pp. 50–57, doi: 10.5281/zenodo.1492341.
- J. W. Kim, J. Salamon, P. Li, and J. P. Bello, “CREPE: A convolutional representation for pitch estimation,” in Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2018, pp. 161–165, doi: 10.1109/ICASSP.2018.8461329.
- L. Nieradzik, “SwiftF0: Fast and accurate monophonic pitch detection,” 2025, arXiv: 2508.18440. Software: swift-f0, MIT.
- pitch-benchmark, software repository. [Online]. Available: https://github.com/lars76/pitch-benchmark. Accessed: 13 Nov. 2026. MIT. Published under the same account as SwiftF0.
- S. Rouard, F. Massa, and A. Défossez, “Hybrid transformers for music source separation,” in Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2023, pp. 1–5, doi: 10.1109/ICASSP49357.2023.10096956.
- W.-T. Lu, J.-C. Wang, Q. Kong, and Y.-N. Hung, “Music source separation with band-split RoPE transformer,” in Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2024, pp. 481–485, doi: 10.1109/ICASSP48485.2024.10446843.
- python-audio-separator, software repository. [Online]. Available: https://github.com/nomadkaraoke/python-audio-separator. Accessed: 13 Nov. 2026. Code: MIT. Distributes the BS-RoFormer SW checkpoint; no licence is stated for the weights.
- R. Solovyev, I. Kiselev, A. Stempkovskiy, and T. Gabruseva, “Music-Source-Separation-Training (MSST): A unified framework for training and evaluating music demixing models,” 2026, arXiv: 2607.23395. Code: MIT. The MVSep Mega-53 checkpoint is published with release v1.0.21 of the repository; no licence is stated for the weights.
- F. Krebs, S. Böck, and G. Widmer, “An efficient state-space model for joint tempo and meter tracking,” in Proc. Int. Society for Music Information Retrieval Conf. (ISMIR), 2015, pp. 72–78, doi: 10.5281/zenodo.1414966.
- S. Böck, F. Korzeniowski, J. Schlüter, F. Krebs, and G. Widmer, “madmom: A new Python audio and music signal processing library,” in Proc. ACM Int. Conf. on Multimedia, 2016, pp. 1174–1178, doi: 10.1145/2964284.2973795.
- F. Foscarin, J. Schlüter, and G. Widmer, “Beat this! Accurate beat tracking without DBN postprocessing,” in Proc. Int. Society for Music Information Retrieval Conf. (ISMIR), 2024, arXiv: 2407.21658. Code and published weights: MIT.
- D. Meredith, “The ps13 pitch spelling algorithm,” Journal of New Music Research, vol. 35, no. 2, pp. 121–159, 2006, doi: 10.1080/09298210600834961.
- C. Cancino-Chacón, S. D. Peter, E. Karystinaios, F. Foscarin, M. Grachten, and G. Widmer, “Partitura: A Python package for symbolic music processing,” in Proc. Music Encoding Conf. (MEC), 2022, arXiv: 2206.01071. Apache-2.0.
- C. L. Krumhansl, Cognitive Foundations of Musical Pitch. New York, NY, USA: Oxford University Press, 1990.
- W3C Music Notation Community Group, “MusicXML version 4.0,” Final Community Group Report, 2021. [Online]. Available: https://www.w3.org/2021/06/musicxml40/. Accessed: 13 Nov. 2026.
- S. Balke, A. Berndt, and M. Müller, “ChoraleBricks: A modular multitrack dataset for wind music research,” Transactions of the International Society for Music Information Retrieval, vol. 8, no. 1, pp. 39–54, 2025, doi: 10.5334/tismir.252. Data: version 1.1.0, doi: 10.5281/zenodo.20849469, CC BY 4.0.
- B. Li, X. Liu, K. Dinesh, Z. Duan, and G. Sharma, “Creating a multitrack classical music performance dataset for multimodal music analysis: Challenges, insights, and applications,” IEEE Transactions on Multimedia, vol. 21, no. 2, pp. 522–535, 2019, doi: 10.1109/TMM.2018.2856090. The dataset page states no licence.
- E. Manilow, G. Wichern, P. Seetharaman, and J. Le Roux, “Cutting music source separation some Slakh: A dataset to study the impact of training data quality and quantity,” in Proc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2019, pp. 45–49, doi: 10.1109/WASPAA.2019.8937170. Data: doi: 10.5281/zenodo.4599666, CC BY 4.0.
- University of Iowa Electronic Music Studios, “Musical instrument samples.” [Online]. Available: https://theremin.music.uiowa.edu/MIS.html. Accessed: 13 Nov. 2026. The page permits use “for any projects, without restrictions”.
- Versilian Studios, “VSCO 2 Community Edition,” sample library. [Online]. Available: https://github.com/sgossner/VSCO-2-CE. Accessed: 13 Nov. 2026. CC0 1.0.
- OpenAIR, the Open Acoustic Impulse Response Library, University of York. [Online]. Available: https://www.openair.hosted.york.ac.uk/. Accessed: 13 Nov. 2026. Impulse response of St Margaret’s Church (National Centre for Early Music), York. The repository’s notices record the licence as CC BY 4.0; the library’s site could not be reached to confirm it when this list was checked.
- C. Raffel, B. McFee, E. J. Humphrey, J. Salamon, O. Nieto, D. Liang, and D. P. W. Ellis, “mir_eval: A transparent implementation of common MIR metrics,” in Proc. Int. Society for Music Information Retrieval Conf. (ISMIR), 2014, pp. 367–372, doi: 10.5281/zenodo.1416528.
- M. Bay, A. F. Ehmann, and J. S. Downie, “Evaluation of multiple-F0 estimation and tracking systems,” in Proc. Int. Society for Music Information Retrieval Conf. (ISMIR), 2009, pp. 315–320, doi: 10.5281/zenodo.1418241.
- B. Efron and R. J. Tibshirani, An Introduction to the Bootstrap. Boston, MA, USA: Springer, 1993, doi: 10.1007/978-1-4899-4541-9.
- F. Foscarin, F. Korzeniowski, and R. Vogl, “Masked diffusion enables coherent beat tracking,” in Proc. Int. Society for Music Information Retrieval Conf. (ISMIR), 2026, arXiv: 2608.04624. Software and pretrained models: md_beat_this, CC BY-NC 4.0.
Appendix A: Values that are not gated
Table 4 lists the values on this page that were measured once and are not held by the regression gate, with their sources.
| Value | As reported | Source |
|---|---|---|
| Notes per agreement class (Figure 10) | 193, 110, 578, 297, 14 | docs/shared/research/10-benchmark-results.md, three-way vote |
| Whole jobs (Section 5.9) | 30 s solo: 21.4–23.2 s; 247 s band: 96 s | docs/shared/research/13-performance-audit.md, M5 |
| Model calls of a solo job (Figure 18) | 1.8 s, 0.30 s, 2.5 s, 9.1 s, 0.30 s | Same document, M6 |
| Quantiser before penalty scaling (Section 5.2) | 89 % position, 89 % within beat, 76 % duration | docs/shared/research/10-benchmark-results.md, rhythm |
| Fast notes before and after (Figure 12) | Figure recall 26 % → 44 % on 306 clips | docs/brasscribe/plan/fast-notes.md, Section 7.2 |
| Triplets on computed onsets (Section 6.1) | 97 % | Same document, Section 8.2 |
| Beat F-measure of the tracked beats (Section 5.2) | 0.82 and 0.78 | site/shared/research/measured/per-piece.json |
| Recall per voice (Figure 8) | Means over pieces, computed with the gate's scorer | site/shared/research/measured/per-piece.json |
| Model speed, memory and conversion parity (Figure 17, Figure 19) | See the figures | convert/reports/*.json |
Appendix B: Data tables of the figures
Figure 1
| Measure | Beats given (from the score) | SD | 95 % CI | Beats tracked (Beat This!) | SD | 95 % CI |
|---|---|---|---|---|---|---|
| Right place in the bar | 99 % | 2.1 % | 97–100 % | 49 % | 32 % | 31–69 % |
| Right place within the beat | 99 % | 2.1 % | 97–100 % | 93 % | 14 % | 84–99 % |
| Right written length | 88 % | 7.3 % | 84–92 % | 77 % | 24 % | 61–88 % |
| Measure | Beats given (from the score) | SD | 95 % CI | Beats tracked (Beat This!) | SD | 95 % CI |
|---|---|---|---|---|---|---|
| Right place in the bar | 95 % | 2.5 % | 94–97 % | 45 % | 37 % | 21–70 % |
| Right place within the beat | 96 % | 2.3 % | 94–97 % | 83 % | 16 % | 72–92 % |
| Right written length | 54 % | 22 % | 41–69 % | 45 % | 23 % | 32–60 % |
Figure 3
| Measure | Onset within ±50 ms (standard) | SD | 95 % CI | Onset within ±100 ms (relaxed) | SD | 95 % CI |
|---|---|---|---|---|---|---|
| MuScriptor medium | 0.56 | 0.07 | 0.51–0.60 | 0.75 | 0.06 | 0.71–0.78 |
| Basic Pitch | 0.47 | 0.06 | 0.44–0.51 | 0.60 | 0.05 | 0.57–0.63 |
| Measure | Onset within ±50 ms (standard) | SD | 95 % CI | Onset within ±100 ms (relaxed) | SD | 95 % CI |
|---|---|---|---|---|---|---|
| MuScriptor medium | 0.69 | 0.17 | 0.57–0.79 | 0.88 | 0.04 | 0.86–0.91 |
| Basic Pitch | 0.66 | 0.11 | 0.59–0.74 | 0.73 | 0.09 | 0.67–0.79 |
| Measure | Onset within ±50 ms (standard) | SD | 95 % CI | Onset within ±100 ms (relaxed) | SD | 95 % CI |
|---|---|---|---|---|---|---|
| Separated first, then MuScriptor | – | 0.70 | ||||
| MuScriptor on the whole mix | – | 0.56 | ||||
| Separated first, then Basic Pitch | – | 0.56 | ||||
| Basic Pitch on the whole mix | – | 0.47 |
Figure 4
| Measure | Beat This! (in use) | SD | 95 % CI | md_beat_this, model run1 (candidate under evaluation) | SD | 95 % CI |
|---|---|---|---|---|---|---|
| F-measure against the notated quarter-note pulse, ±70 ms | 0.82 | 0.16 | 0.72–0.91 | 0.85 | 0.12 | 0.77–0.91 |
| Reference notes written at their notated position in the bar | 0.49 | 0.32 | 0.31–0.69 | 0.53 | 0.39 | 0.30–0.76 |
| Measure | Beat This! (in use) | SD | 95 % CI | md_beat_this, model run1 (candidate under evaluation) | SD | 95 % CI |
|---|---|---|---|---|---|---|
| F-measure against the notated quarter-note pulse, ±70 ms | 0.78 | 0.17 | 0.68–0.90 | 0.79 | 0.19 | 0.67–0.91 |
| Reference notes written at their notated position in the bar | 0.45 | 0.37 | 0.21–0.70 | 0.57 | 0.43 | 0.29–0.84 |
Figure 5
| Measure | Chamber brass | Brass quartets |
|---|---|---|
| As long as it was played | 60 % | 88 % |
| Held to the next note when the gap is small | 63 % | 91 % |
| From where the pitch leaves the note (used) | 65 % | 93 % |
Figure 6
| Measure | Detected: pitch and onset within ±100 ms | Written in place: also in the right place in the bar | Measured on |
|---|---|---|---|
| Chamber brass, single parts (URMP) | 0.80 | 0.27 | 144 parts |
| Chorales, single parts (ChoraleBricks) | 0.72 | 0.24 | 40 parts |
| Measure | Detected: pitch and onset within ±100 ms | Written in place: also in the right place in the bar | Measured on |
|---|---|---|---|
| Chamber brass, single parts (URMP) | 0.80 | 0.27 | 144 parts |
| Chorales, single parts (ChoraleBricks) | 0.72 | 0.27 | 40 parts |
Figure 7
| Measure | Instrument chosen: its own range | No instrument chosen: fixed E3–E6 | Measured on |
|---|---|---|---|
| Trumpet | 0.97 | 0.97 | 20 recordings |
| Flugelhorn | 0.96 | 0.96 | 20 recordings |
| French horn | 0.90 | 0.90 | 11 recordings |
| Baritone | 0.97 | 0.78 | 24 recordings |
| Trombone | 0.97 | 0.56 | 8 recordings |
| Tuba | 0.88 | 0.00 | 10 recordings |
| Measure | SwiftF0 | Basic Pitch | Measured on |
|---|---|---|---|
| Trumpet | 0.96 | 0.88 | 20 recordings |
| Flugelhorn | 0.96 | 0.96 | 20 recordings |
| French horn | 0.90 | 0.80 | 11 recordings |
| Baritone | 0.97 | 0.87 | 24 recordings |
| Trombone | 0.97 | 0.79 | 8 recordings |
| Tuba | 0.89 | 0.48 | 10 recordings |
Figure 8
| Measure | MuScriptor medium | Basic Pitch | Notes both found |
|---|---|---|---|
| Soprano (trumpet) | 0.92 | 0.36 | 0.80 |
| Alto (flugelhorn) | 0.83 | 0.38 | 0.55 |
| Tenor (baritone) | 0.31 | 0.25 | 0.28 |
| Bass (tuba) | 0.69 | 0.30 | 0.38 |
| Measure | MuScriptor medium | Basic Pitch |
|---|---|---|
| Soprano (trumpet) | 97 % | 90 % |
| Alto (flugelhorn) | 92 % | 89 % |
| Tenor (baritone) | 76 % | 82 % |
| Bass (tuba) | 72 % | 66 % |
| Measure | MuScriptor medium | Basic Pitch |
|---|---|---|
| Soprano (trumpet) | 84 % | 74 % |
| Alto (flugelhorn) | 77 % | 66 % |
| Tenor (baritone) | 48 % | 71 % |
| Bass (tuba) | 45 % | 46 % |
Figure 9
| Measure | From the score's harmony | From the recording (MuScriptor) | From the recording (Basic Pitch only) |
|---|---|---|---|
| Alto | 80 % | 53 % | 17 % |
| Tenor | 78 % | 43 % | 18 % |
Figure 10
| Measure | Precision | Measured on |
|---|---|---|
| SwiftF0 and MuScriptor | 99 % | 110 notes |
| All three | 98 % | 193 notes |
| SwiftF0 and Basic Pitch | 91 % | 578 notes |
| SwiftF0 alone | 54 % | 297 notes |
| MuScriptor and Basic Pitch, not SwiftF0 | 36 % | 14 notes |
Figure 11
| Measure | Wrong notes that carry a ? | Right notes marked ? anyway | Measured on |
|---|---|---|---|
| Trumpet | 33 % | 0.4 % | 20 recordings |
| Flugelhorn | 33 % | 0.2 % | 20 recordings |
| French horn | 0 % | 0.4 % | 11 recordings |
| Baritone | 20 % | 0.5 % | 24 recordings |
| Trombone | 32 % | 0 % | 8 recordings |
| Tuba | 18 % | 3.6 % | 10 recordings |
Figure 12
| Measure | Beats given | Beats tracked (Beat This! small0) |
|---|---|---|
| 1. Detected by the f0 estimator (SwiftF0) | 59 % | 59 % |
| 2. In the solo line after the vote | 60 % | 60 % |
| 3. After snapping to the beat grid | 55 % | 47 % |
| 4. Written in the score | 55 % | 47 % |
| Measure | Before the fast-note rules | After |
|---|---|---|
| Notes of the fast figure written (recall) | 26 % | 44 % |
| All notes of the clip (F-measure) | 40 % | 54 % |
| Trills and alternations that keep four fifths of their notes | 11 % | 30 % |
| Measure | Before the fast-note rules | After |
|---|---|---|
| Notes of the fast figure written (recall) | 22 % | 36 % |
| All notes of the clip (F-measure) | 37 % | 48 % |
| Trills and alternations that keep four fifths of their notes | 9.5 % | 18 % |
Figure 13
| Measure | Onset within 50 ms | Onset and note end both right |
|---|---|---|
| MuScriptor medium (used) | 0.56 | 0.48 |
| Basic Pitch, recording retuned to A = 440 (used) | 0.48 | 0.33 |
| Basic Pitch, as recorded | 0.47 | 0.28 |
| MuScriptor large | 0.44 | 0.39 |
| MuScriptor medium, conditioned on brass instruments | 0.33 | 0.30 |
| MuScriptor large, conditioned on brass instruments | 0.28 | 0.25 |
Figure 14
| Measure | Onset F-measure |
|---|---|
| SwiftF0 | 0.75 |
| Notes at least two models found | 0.73 |
| MuScriptor | 0.50 |
| Basic Pitch | 0.42 |
Figure 15
| Measure | Best single route | SD | 95 % CI | Vote between the models | SD | 95 % CI | Measured on |
|---|---|---|---|---|---|---|---|
| Brass quartets | 0.75 | 0.06 | 0.71–0.78 | 0.76 | 10 chorales, 1,887 notes | ||
| Chamber brass | 0.88 | 0.04 | 0.86–0.91 | 0.88 | 8 pieces, 5,129 notes | ||
| Trumpet over a band (separated first) | 0.70 | 0.63 | 2 songs |
Figure 16
| Measure | Trumpet over a band (Slakh) | Chamber brass | Brass quartets |
|---|---|---|---|
| Notes MuScriptor found (used) | 0.65 | 0.72 | 0.90 |
| Only notes both models found | 0.67 | 0.65 | 0.80 |
| Everything either model found | 0.36 | 0.55 | 0.36 |
Figure 17
| Measure | Converted (best conversion that passes) | Reference framework on the processor |
|---|---|---|
| SwiftF0 (pitch of one line) | 0.02 s | 0.13 s |
| Basic Pitch (notes) | 0.18 s | 0.44 s |
| Beat This! small0 (beats) | 0.05 s | 0.55 s |
| BS-RoFormer SW (separation, 6 stems) | 13 s | 126 s |
| Mega-53 (separation, 53 stems) | 29 s | 208 s |
| MuScriptor medium (notes), on the processor only | 97 s | 84 s |
| Measure | Converted (best conversion that passes) | Reference framework on the processor |
|---|---|---|
| SwiftF0 (pitch of one line) | 0.38 GB | 0.32 GB |
| Basic Pitch (notes) | 0.57 GB | 0.70 GB |
| Beat This! small0 (beats) | 0.49 GB | 0.48 GB |
| BS-RoFormer SW (separation, 6 stems) | 3.8 GB | 3.6 GB |
| Mega-53 (separation, 53 stems) | 21 GB | 14 GB |
| MuScriptor medium (notes), on the processor only | 5.0 GB | 2.5 GB |
Figure 18
| Measure | Seconds |
|---|---|
| Beats (Beat This!) | 1.8 s |
| Pitch of the line (SwiftF0) | 0.30 s |
| Notes (Basic Pitch) | 2.5 s |
| Notes (MuScriptor, graphics processor) | 9.1 s |
| Pitch contour for note ends | 0.30 s |
Figure 19
| Measure | Parity with the original |
|---|---|
| SwiftF0, full precision | 1.000 |
| SwiftF0, half precision on the Neural Engine: fails | 0.370 |
| SwiftF0 split: only its middle on the Neural Engine | 0.999 |
| Basic Pitch, full precision | 1.000 |
| Basic Pitch, half precision: fails | 0.815 |
| Beat This! small0, half precision | 0.994 |
| BS-RoFormer SW, half precision | 0.999 |
| Mega-53, half precision | 0.999 |
| MuScriptor, ONNX | 1.000 |
| MuScriptor, Core ML half precision: fails a stricter test | 0.999 |