Research paper

Brasscribe: Transcribing Brass Recordings into Band Scores with Pretrained Models and Deterministic Rules

Abstract

We describe and evaluate Brasscribe, a system that transcribes recordings of brass ensembles into notated scores and parts for British-style brass band. Pretrained models for source separation, note transcription, fundamental-frequency estimation and beat tracking are combined, without further training, with deterministic rules for rhythm quantisation, pitch spelling, harmony reduction and arrangement. Each stage is evaluated on ChoraleBricks brass quartets (10 pieces), URMP brass chamber music (8 pieces), single-instrument recordings and synthesised material, with note-level precision, recall and F-measure as defined in mir_eval and with score-position accuracy for rhythm. The best transcriber reaches a mean onset F-measure of 0.56 at the standard ±50 ms tolerance (0.75 at ±100 ms) on the quartets and 0.69 (0.88) on the chamber pieces. Given reference beats, the quantiser places 95–99 % of notes at their notated position in the bar; with tracked beats it places 45–49 %, with large variation between pieces. End to end on single parts, at ±100 ms (the only tolerance stored for these suites), notes are detected with F = 0.72–0.80, whereas the written F-measure, which also requires the correct position in the bar, is 0.24–0.27. The tenor voice of a quartet is recovered with F = 0.31 (±100 ms), and the confidence marks flag at most 33 % of wrong notes. All rule thresholds were tuned on the evaluation sets and no brass-band recording with reference notes was available, so the results are upper bounds. Beat and metre tracking is the main obstacle to readable output.

Keywords: automatic music transcription, brass band, rhythm quantisation, beat tracking, source separation, evaluation.

Plain-language summary: In a brass ensemble, Brasscribe finds about two thirds of the played notes when the timing must be exact to 50 ms, and more than four fifths when 100 ms is allowed. It is much less reliable about where they belong in the bar, and its question marks warn about only some of the mistakes. The output is a draft to correct, not a finished part.

1 Introduction

Automatic music transcription (AMT) converts audio into a symbolic representation: which pitches sound, when they start and when they stop [1]. A musician needs more than a list of notes. A part is readable when its rhythms lie on a beat grid that can be counted, in bars of the correct length, with the pitch spellings a composer would choose; it is playable when it stays within the instrument's range, is written in its transposition, and does not cross neighbouring parts.

The British-style brass band is a demanding target. A contest score has 18 parts, from soprano cornet to B♭ bass and percussion, and Norwegian bands use the same instrumentation. Most of the instruments share one timbre family and double each other constantly. To our knowledge, no openly available transcription model has the band's instruments in its vocabulary, and none separates several players of the same instrument.

Brasscribe addresses the task with pretrained models that only listen (they separate sources, estimate notes and track beats) and with deterministic rules that write the notation. It runs on the musician's own devices; no recording is uploaded to a server. In the builds released at the time of writing, every score, including a solo part, is produced on the musician's computer on all platforms. The Android app transcribes a solo part on the device in builds that bundle the listening models, which the builds released so far did not; on iPhone, iPad and Mac the models are not yet included, and the Windows app has no on-device path.

The central finding is shown in Figure 1: the rules place nearly all notes correctly when the beats are known, and about half on average, with wide variation between pieces, when the beats are tracked from the audio.

The correct notes are given to the quantiser, so only the beat grid differs. A dot is one piece, the ringed dot the mean, the thick bar its 95 % confidence interval: with tracked beats the pieces spread over almost the whole scale.

Data table
Brass quartetsChoraleBricks: four real players (trumpet, flugelhorn, baritone, tuba), recorded one at a time and mixed. Measured on: 10 chorales, 1,887 notes.
MeasureBeats given (from the score)SD95 % CIBeats tracked (Beat This!)SD95 % CI
Right place in the bar99 %2.1 %97–100 %49 %32 %31–69 %
Right place within the beat99 %2.1 %97–100 %93 %14 %84–99 %
Right written length88 %7.3 %84–92 %77 %24 %61–88 %
Chamber brassURMP: real duos to quintets of trumpet, horn, trombone and tuba. Measured on: 8 pieces, 5,129 notes.
MeasureBeats given (from the score)SD95 % CIBeats tracked (Beat This!)SD95 % CI
Right place in the bar95 %2.5 %94–97 %45 %37 %21–70 %
Right place within the beat96 %2.3 %94–97 %83 %16 %72–92 %
Right written length54 %22 %41–69 %45 %23 %32–60 %
Figure 1. With reference beats nearly every note is written at its place in the bar; with tracked beats 45 % to 49 % are on average, with wide variation between pieces. Rhythm with the beats given and with the beats tracked. Position in the bar needs the bar lines to be right; position within the beat does not. n: Brass quartets: 10 chorales, 1,887 notes · Chamber brass: 8 pieces, 5,129 notes. Shown here: Brass quartets; the other views, with n per view, are tabulated in the appendix of data tables. Source: eval/baselines.json, suites quant-chorales and quant-urmp; the pieces from per-piece.json.

The contributions of this paper are:

  • a description of a complete audio-to-score pipeline for brass ensembles that uses only pretrained models and deterministic rules (Section 3);
  • a stage-by-stage evaluation on real brass recordings with reference notes, reported with the standard note-level metrics and with score-position metrics for rhythm (Sections 4 and 5);
  • negative results and comparisons that determined the design, including models and settings that were rejected (Section 5.8);
  • an error analysis and a statement of threats to validity, notably that all thresholds were tuned on the evaluation data (Sections 6 and 7);
  • a page whose numbers and figures are generated from the regression baselines of the public repository, so that they cannot drift from the measurements (Section 9).

3 Method

3.1 Overview

Figure 2 shows the pipeline, and Table 1 lists the pretrained models and their roles. Learned models are used only for listening; every stage after the vote between the note models is deterministic, so identical notes always yield an identical score.

The Brasscribe pipelineStages from recording to score. Learned models (outlined in the brand colour) only listen: they separate layers, find notes and find beats. Everything after the vote is music rules that always give the same result. Dashed stages are where the musician decides.Recordinga file, the microphone, or sound on the computerWhat is this?solo · brass band · soloist with orchestra · pop/rockSeparatea song or an orchestra only:BS-RoFormer SW, Mega-53BeatsBeat This!on the whole mixNotes per layerMuScriptor, Basic Pitchand SwiftF0Metrical levelthe beat a playerwould countVote and confidenceagreement between models→ which notes get a ?Bar lines, free timedownbeat cleanup,ad lib. passagesPut the notes on the gridquantisation, 24 ticks per beatDurationsnote ends from the pitch contour; staccatoSpelling and keyps13 at concert pitch · Krumhansl–KesslerHarmonythe chord tones of each beatArrangefull band · small band · quartetDifficultyas played · standard · easierCheck the notes marked ?grouped review, the player's own part firstShare or printMusicXML · PDF · MIDI · MP3 · talking score
Figure 2. From recording to score. The two columns run side by side, notes on the left and the beat grid on the right, and meet where the notes are put on the grid.
Table 1. Pretrained models used by the system, their role, where they run and their licence. Source: THIRD_PARTY_NOTICES.md and engine/src/scribe_engine/profiles.py.
ModelRoleRuns onLicence
SwiftF0 [8]f0 of a monophonic line: the basis of a solo part, and note offsetsComputer; Android phone in builds that bundle the modelsMIT
Basic Pitch [5]Second note estimate for every note; the only note model in a draft made on a phoneComputer; Android phone in builds that bundle the modelsApache-2.0
Beat This! [16]Beats and downbeats (checkpoint small0 on phones, final0 on the computer)Computer; Android phone in builds that bundle the modelsMIT, code and weights
MuScriptor medium [4]Main note model for an ensemble or a mixtureComputerCC BY-NC 4.0, gated; downloaded by each user
BS-RoFormer SW [11], [12]Separates a pop or rock recording into stems before transcriptionComputerCode MIT; no licence stated for the weights
MVSep Mega-53 [13]Extracts a soloist, the bass and the drums from an orchestral recordingComputerCode MIT; no licence stated for the weights

3.2 Routing

The user states what the recording is, and that answer selects the route. A solo part and a brass ensemble are passed directly to the note models, since an all-brass mixture offers a separator nothing to split. A pop or rock recording is first separated with BS-RoFormer SW, and each stem is transcribed. For a soloist with an orchestra, Mega-53 extracts the solo, the bass and the drums, and the orchestra is taken as the residual of the mixture.

3.3 Note consensus and confidence

The note models process the same audio, and their notes are clustered when they agree in pitch and onset. For a solo part, SwiftF0 supplies the notes and a second model confirms them; a note that SwiftF0 did not produce is discarded (Section 5.6 gives the evidence). A logistic regression then maps the evidence for each note to a probability that it is correct: which models produced it, its duration, the voicing confidence of the f0 contour at its start, and whether the line was separated from a mixture. A note below a threshold is marked with a question mark; neighbouring marked notes share one mark.

3.4 Beats, metre and quantisation

Beat This! tracks beats and downbeats on the full mixture. Rules select the metrical level a player would count (the tracker often follows half notes in slow music), repair bar lines where a beat was inserted or dropped, and label passages without a steady pulse as ad lib. The notes are then quantised: for each beat, the quantiser weighs how well a subdivision fits the onsets against how complex it is to read, at a resolution of 24 ticks per beat. Written durations are derived from where the f0 contour leaves the note, not from the performed duration.

3.5 Spelling, harmony and arrangement

Pitches are spelled for the whole ensemble at concert pitch, before transposition, and the key is estimated from the same notes. Dense accompaniment is reduced to the chord tones of each beat. Three arrangers write the parts: a full band, a small band, and a quartet whose inner parts follow part-writing rules (no crossing, no parallel fifths or octaves, chord tones only). Three difficulty levels are derived from the result.

All stages after the vote are implemented twice, in Python as the reference and in Rust for the apps, and a conformance suite requires identical output from the two.

4 Experimental setup

4.1 Datasets

No public brass-band recording with aligned reference notes was found. Table 2 lists the sets that were used instead. None of them is a brass band: the quartets have one player per part, recorded one at a time, and the largest set has 10 pieces.

Table 2. Evaluation sets, their size as used, their licence and where the regression gate runs on them. Sizes are counted by site/shared/research/per_piece.py or taken from eval/baselines.json.
SetContentSize usedLicenceGate
ChoraleBricks, brass quartets [21]Chorales played by four separately recorded players (trumpet, flugelhorn, baritone, tuba), with aligned notes and score positions10 pieces, 1,887 notes; 93 single-instrument recordingsCC BY 4.0Continuous integration, from pinned annotations and cached model outputs
URMP (University of Rochester Multi-Modal Music Performance dataset), brass pieces [22]Chamber pieces for two to five brass players; each part also recorded alone8 pieces, 5,129 notes; 144 single partsNone statedLocal only
Slakh, trumpet lead [23]Synthesised songs with a trumpet melody over a band; used to compare routes, not for rhythm, since its notes are not quantised2 songsCC BY 4.0Local only
Fast-note clipsRuns, repeated notes, trills and ornaments assembled from trumpet samples [24], [25], dry and convolved with a church impulse response [26], with an exact reference253 clips in continuous integration; 306 in allSamples: unrestricted and CC0; impulse response: CC BY 4.0Continuous integration
One commercial recordingA soloist with an orchestra; no reference notes, the arranged output is frozen and must be reproduced exactly1 recordingCommercial; not published or distributedLocal only

4.2 Splits and data leakage

No listening model was trained or fine-tuned; all six are used as published. Two components were fitted to our data. The confidence model (Section 3.3) was fitted on notes with reference labels from the sets in Table 2 and validated on a held-out half of the recordings. The thresholds of the rules (the quantiser's complexity penalties, the bar-line repair, the duration and staccato rules, the fast-note rules) were chosen while inspecting the same sets on which they are scored. There is no held-out brass set. The results in Section 5 therefore measure how well the rules fit these recordings and should be read as upper bounds for unseen recordings.

4.3 Metrics

All measures lie between 0 and 1; higher is better. The detection of a note and its placement in the score are measured separately. Note-level measures are computed with mir_eval.transcription [27], onset-only and, where stated, onset+offset. The primary onset tolerance is ±50 ms; ±100 ms is reported as a secondary, relaxed tolerance, and several suites store results at ±100 ms only, which is stated where it applies. An octave error is a reference note that remains unmatched while an estimated note one or two octaves away starts within the onset tolerance; the octave-error rate is the share of reference notes for which this holds.

  • Note-level precision, recall and F-measure

    Notes are evaluated as in mir_eval [27], which implements the MIREX note-tracking convention [28]. An estimated note matches a reference note when its onset lies within ±50 ms of the reference onset and its pitch within ±50 cents; each note is matched at most once. With TP matched notes, FP unmatched estimated notes and FN unmatched reference notes:

    P = TP / (TP + FP),   R = TP / (TP + FN),   F = 2PR / (P + R) (1)

    Offsets are ignored unless stated. Where they are used (onset+offset F-measure), the offset must also lie within max(50 ms, 20 % of the reference note's duration). We also report F with a ±100 ms onset tolerance, and say so wherever it is used: softly attacked brass notes are detected late relative to their annotated onsets, and deviations of that size are absorbed when the notes are quantised.

    played ±50±100 ms +30+80extra ✓ ✓✓✗
  • Position and duration accuracy

    Rhythm is scored on the N notes that have a position in the reference score. Let qi be the quantised position of note i (in ticks, 24 per beat) and si its notated position, after one global shift by a whole number of beats:

    Abar = |{i : qi = si}| / N (2)

    Position within the beat, Abeat, compares the positions modulo one beat, so it is unaffected by a displaced bar line. Duration accuracy is the share of notes whose written duration equals the notated one.

    scorewritten ✓✓✗
  • Written F-measure

    The strictest measure combines pitch, onset and notation. It is the F-measure of (1) at a ±100 ms onset tolerance, with a matched note counted as a true positive only if its position in the bar also equals the score's:

    TPw = |{matched i : position in the bar correct}|,   Fw from TPw as in (1) (3)

    Metre, bar phase and the quantisation grid must therefore all be correct for a note to count.

    heardin place ✓✓ ✓✗ ✗–
  • Metre accuracy

    No time signature is annotated by the beat tracker, so metre is scored by bar length. For a set of parts, with L the bar length in quarter notes:

    M = |{parts : Lwritten = Lscore}| / |parts| (4)

    A tracker that follows the beat at half or double speed is not penalised by this measure.

    score3/4 written4/4
  • Precision and recall of the confidence marks

    A note whose estimated probability of being correct falls below a threshold is marked question mark. Let W be the written notes that are wrong (unmatched under (1)) and Q the marked notes:

    P? = |Q ∩ W| / |Q|,   R? = |Q ∩ W| / |W| (5)

    The false-alarm rate is the share of correct notes that are marked. In the drawing three notes are wrong and two are marked, one of them correctly: R? = 1/3, P? = 1/2.

    ?? wrong notes are hollow

4.4 Protocol

Model outputs are computed once and stored; scribe bench rescores every suite from the stored outputs and fails when a value has moved by more than the tolerance recorded in eval/baselines.json. Results are means over pieces. Where the per-piece values are available we also report the sample standard deviation (SD) and a 95 % confidence interval (CI) of the mean, obtained with a percentile bootstrap over pieces (10,000 resamples, fixed seed) [29]; with 8 to 10 pieces these intervals are wide. No significance tests were carried out, and differences between conditions are reported descriptively. F-measure, precision and recall are given as proportions to two decimals, and shares as percentages (with one decimal below 10 %); the parity of converted models is given to three decimals because its threshold is 0.98. Precision and recall are reported beside the F-measure where the baselines store them or the scorer provides them for the ensemble suites (Table 3). Most figures read the file used by the gate; figures built from other files (the model-conversion reports, and a small set of values measured once, listed in the Appendix) state this in their source line.

5 Results

5.1 Note detection in ensembles

Figure 3 gives the onset F-measure of the two transcribers. On the quartets the best model reaches 0.56 at the standard ±50 ms tolerance (SD 0.07, 95 % CI 0.51–0.60) and 0.75 at the relaxed ±100 ms tolerance (SD 0.06, 95 % CI 0.71–0.78; range 0.60–0.81). Table 3 gives precision and recall beside the F-measure: at ±100 ms the best model finds 83 % of the played notes of the quartets, and 68 % of the notes it reports are correct. The difference between the two tolerances is large for brass, consistent with late detection of soft attacks. On the two synthesised songs with a trumpet lead, separating before transcription scored higher than transcribing the mixture; with two songs this is suggestive only.

Notes found in the whole recording, every part at once. A dot is one piece, the ringed dot the mean, the thick bar its 95 % confidence interval.

Data table
Brass quartetsChoraleBricks: four real players (trumpet, flugelhorn, baritone, tuba), recorded one at a time and mixed. Measured on: 10 chorales, 1,887 notes.
MeasureOnset within ±50 ms (standard)SD95 % CIOnset within ±100 ms (relaxed)SD95 % CI
MuScriptor medium0.560.070.51–0.600.750.060.71–0.78
Basic Pitch0.470.060.44–0.510.600.050.57–0.63
Chamber brassURMP: real duos to quintets of trumpet, horn, trombone and tuba. Measured on: 8 pieces, 5,129 notes.
MeasureOnset within ±50 ms (standard)SD95 % CIOnset within ±100 ms (relaxed)SD95 % CI
MuScriptor medium0.690.170.57–0.790.880.040.86–0.91
Basic Pitch0.660.110.59–0.740.730.090.67–0.79
Trumpet over a bandSlakh: synthesised songs with a trumpet lead. Only the ±100 ms result is stored for this set. Measured on: 2 songs.
MeasureOnset within ±50 ms (standard)SD95 % CIOnset within ±100 ms (relaxed)SD95 % CI
Separated first, then MuScriptor–0.70
MuScriptor on the whole mix–0.56
Separated first, then Basic Pitch–0.56
Basic Pitch on the whole mix–0.47
Figure 3. The best model reaches an onset F-measure of 0.56 on brass quartets and 0.69 on chamber brass at ±50 ms, and 0.75 and 0.88 at ±100 ms. Onset F-measure of the two transcribers on real brass, and of the two routes on a full band. A note counts as detected when its pitch is correct and its onset lies within the tolerance. n: Brass quartets: 10 chorales, 1,887 notes · Chamber brass: 8 pieces, 5,129 notes · Trumpet over a band: 2 songs. Shown here: Brass quartets; the other views, with n per view, are tabulated in the appendix of data tables. Source: eval/baselines.json, suites chorales-transcription, urmp-transcription and slakh-transcription; the pieces from site/shared/research/measured/per-piece.json.
Table 3. Note-level precision (P), recall (R) and F-measure (F) of the two transcribers on the ensembles of Figure 3, means over pieces. F is the gated value; P and R are computed with the same scorer by per_piece.py and are not gated.
Set and modelP, ±50 msR, ±50 msF, ±50 msP, ±100 msR, ±100 msF, ±100 msn
Brass quartets, MuScriptor medium0.510.630.560.680.830.7510 pieces
Brass quartets, Basic Pitch0.370.640.470.470.810.6010 pieces
Chamber brass, MuScriptor medium0.700.690.690.890.880.888 pieces
Chamber brass, Basic Pitch0.640.710.660.710.780.738 pieces

5.2 Rhythm quantisation

Figure 1 (Section 1) isolates the rhythm stage by quantising the reference notes. With reference beats, 95–99 % of notes are written at their notated position in the bar. With tracked beats the mean falls to 45–49 %, and single pieces range from 3.5–100 %; the mean is therefore a poor summary. Position within the beat is more robust (83–93 %), so the rhythm inside a beat is usually readable even when the bar lines are displaced. One tuning step underlies the reference-beat results: scaling the quantiser's complexity penalties raised exact positions on the chorales from 89 % to 99 %. The step was tuned on these chorales, so the gain is a fit and not a test.

The tracked beats were also scored against the notated quarter-note pulse, with the F-measure of mir_eval.beat (tolerance ±70 ms; beats in the first 5 s are discarded, the default of mir_eval); Figure 4 gives the results. No annotated beats exist for these sets, so the reference is the quarter-note pulse interpolated from the notated positions of the reference notes. The values are therefore not comparable with published beat F-measures, which rest on annotated beats. The raw output of Beat This! reaches a mean of 0.82 (SD 0.16, 95 % CI 0.72–0.91) on the quartets and 0.78 (SD 0.17, 95 % CI 0.68–0.90) on the chamber pieces. A tracker that follows every second quarter note scores about 0.67 under this measure, and 5 of the 8 chamber pieces lie between 0.60 and 0.75. Read together with Figure 1, this indicates that the beats themselves are mostly found, whereas the metrical level and the bar are not. Downbeat F-measure and the continuity measures (CMLt, AMLt) were not computed, because the references contain no bar-start times.

The figure also includes a candidate tracker that is under evaluation and is not part of the released system, md_beat_this (model run1) [30]. Against the same pulse its F-measure is 0.85 and 0.79, close to that of Beat This!. Quantising the reference notes on its beats raises the mean position accuracy from 49 % to 53 % on the quartets and from 45 % to 57 % on the chamber pieces; single pieces move in both directions, and with 8 to 10 pieces and no test the difference is suggestive only. These values are computed from stored beat files and are not gated.

The tracked beats themselves, and what they do to the notes. A dot is one piece, the ringed dot the mean, the thick bar its 95 % confidence interval.

Data table
Brass quartetsChoraleBricks: four real players (trumpet, flugelhorn, baritone, tuba), recorded one at a time and mixed. Measured on: 10 chorales, 1,887 notes.
MeasureBeat This! (in use)SD95 % CImd_beat_this, model run1 (candidate under evaluation)SD95 % CI
F-measure against the notated quarter-note pulse, ±70 ms0.820.160.72–0.910.850.120.77–0.91
Reference notes written at their notated position in the bar0.490.320.31–0.690.530.390.30–0.76
Chamber brassURMP: real duos to quintets of trumpet, horn, trombone and tuba. Measured on: 8 pieces, 5,129 notes.
MeasureBeat This! (in use)SD95 % CImd_beat_this, model run1 (candidate under evaluation)SD95 % CI
F-measure against the notated quarter-note pulse, ±70 ms0.780.170.68–0.900.790.190.67–0.91
Reference notes written at their notated position in the bar0.450.370.21–0.700.570.430.29–0.84
Figure 4. Against the notated quarter-note pulse the two trackers score alike; the candidate places more notes at their notated position on average, and both vary widely between pieces. Beat tracking on the brass mixtures, by the F-measure of mir_eval.beat against the notated pulse (not comparable with published beat F-measures) and by its effect on rhythm quantisation (reference notes quantised on the tracked beats). The candidate is not part of the released system. n: Brass quartets: 10 chorales, 1,887 notes · Chamber brass: 8 pieces, 5,129 notes. Shown here: Brass quartets; the other views, with n per view, are tabulated in the appendix of data tables. Source: site/shared/research/measured/per-piece.json, computed by per_piece.py from the beat files stored beside each mixture; the position accuracy of Beat This! is the gated value of quant-chorales and quant-urmp.

Written durations are examined separately in Figure 5, which compares three rules for deriving a written duration. The rule in use is the best of the three by a small margin, and the chamber-music result remains low under every rule. These values are higher than the duration accuracy in Figure 1 (54 % for chamber music) because the suites differ: there the duration is taken from the quantiser alone and must equal the notated one exactly, whereas here a duration rule is applied per part and a deviation of one tick is accepted.

Share of notes whose written length equals the score's, with the score's beats, under three rules for how long a note is.

Data table
Note ends from the annotationMeasured on: 8 chamber pieces, 10 chorales.
MeasureChamber brassBrass quartets
As long as it was played60 %88 %
Held to the next note when the gap is small63 %91 %
From where the pitch leaves the note (used)65 %93 %
Figure 5. Written note lengths: right for about nine in ten chorale notes and about two in three in chamber music, whichever rule is used. Three rules for written length. The rule in use gains two or three points. In chamber music the score often writes a short note and a rest where the player holds on, and no rule here recovers that. n: 8 chamber pieces, 10 chorales. Source: eval/baselines.json, suite durations (the chorale part runs in CI).

5.3 A single part, end to end

Figure 6 evaluates the complete solo path on recordings of single parts, with the models available on a phone. Notes are detected well, but the written F-measure is about a quarter, and the bar length is correct for 24–60 % of the parts. A monophonic line offers a beat tracker little evidence: there is no bass and no percussion. Replacing the small beat-tracking checkpoint by the larger one (second view) changes the results little; no statistical test was applied to the difference.

The whole solo path with the models a phone has, scored against the player's own part: the same notes, counted first by pitch and onset, then also by their place in the bar.

Data table
Beat This! small0 (on phones)The small beat tracker the apps carry.
MeasureDetected: pitch and onset within ±100 msWritten in place: also in the right place in the barMeasured on
Chamber brass, single parts (URMP)0.800.27144 parts
Chorales, single parts (ChoraleBricks)0.720.2440 parts
Beat This! final0 (larger)The larger beat tracker; the notes are the same.
MeasureDetected: pitch and onset within ±100 msWritten in place: also in the right place in the barMeasured on
Chamber brass, single parts (URMP)0.800.27144 parts
Chorales, single parts (ChoraleBricks)0.720.2740 parts
Figure 6. On recordings of a single part the notes are detected with a high F-measure, but the written F-measure is about a quarter. The solo path end to end, on each part recorded alone. The gap between the two bars of a row is what the beat grid, the metre and the bar lines cost. n: Chamber brass, single parts (URMP): 144 parts; Chorales, single parts (ChoraleBricks): 40 parts. Shown here: Beat This! small0 (on phones); the other views, with n per view, are tabulated in the appendix of data tables. Source: eval/baselines.json, suite solo-beat-model (runs locally, not in CI).

5.4 Single instruments

Figure 7 reports the solo path per instrument. Every brass instrument recorded alone is transcribed well, including the tuba, with an octave-error rate of at most 0.2 %. Without knowledge of the instrument, the solo path retains only notes in a cornet's range, so the tuba is lost entirely and the trombone retains 44 % of its notes; the f0 estimator is not the limiting factor. The second view compares SwiftF0 with Basic Pitch on the same recordings. Basic Pitch confirms the notes of SwiftF0, and on the tuba it agrees with 73 % of them, which leaves 9.5 unmarked wrong notes per 100 written notes, against 0.2 on the trumpet.

Onset F-measure at ±100 ms of the written solo line, per instrument, with a fixed cornet range and with the player's own.

Data table
The written line, with and without the instrument chosen
MeasureInstrument chosen: its own rangeNo instrument chosen: fixed E3–E6Measured on
Trumpet0.970.9720 recordings
Flugelhorn0.960.9620 recordings
French horn0.900.9011 recordings
Baritone0.970.7824 recordings
Trombone0.970.568 recordings
Tuba0.880.0010 recordings
The two models, each alone
MeasureSwiftF0Basic PitchMeasured on
Trumpet0.960.8820 recordings
Flugelhorn0.960.9620 recordings
French horn0.900.8011 recordings
Baritone0.970.8724 recordings
Trombone0.970.798 recordings
Tuba0.890.4810 recordings
Figure 7. One instrument alone is found well, tuba included, once the app knows which instrument it is. Each instrument recorded alone. Pitch and onset only: where the notes land in the bar is the figure before this one. The second view is SwiftF0 measured on brass, beside Basic Pitch, the model that confirms its notes. n: Trumpet: 20 recordings; Flugelhorn: 20 recordings; French horn: 11 recordings; Baritone: 24 recordings; Trombone: 8 recordings; Tuba: 10 recordings. Shown here: The written line, with and without the instrument chosen; the other views, with n per view, are tabulated in the appendix of data tables. Source: eval/baselines.json, suite solo-instruments (runs in CI).

5.5 Individual voices of an ensemble

Figure 8 asks whether the line of one player can be extracted from a quartet recording by the range of the instrument. The top voice is recovered and the tenor largely is not (F = 0.31 with the best model). The further views give the recall of each voice's notes anywhere in the transcription; these are upper bounds, and the lower two voices are the ones most often missing. For this reason the system writes only the melody and the bass of an ensemble from the audio and arranges the parts between them from the harmony.

One player's line picked out of the four-part mix by the range of their instrument, scored against that player's part.

Data table
One player's line taken out wholeMeasured on: 10 chorales, 1,887 notes.
MeasureMuScriptor mediumBasic PitchNotes both found
Soprano (trumpet)0.920.360.80
Alto (flugelhorn)0.830.380.55
Tenor (baritone)0.310.250.28
Bass (tuba)0.690.300.38
Notes of each voice found, within 100 msA note is credited to a voice when any written note matches it, so these are upper bounds. Measured on: 10 chorales, 1,887 notes.
MeasureMuScriptor mediumBasic Pitch
Soprano (trumpet)97 %90 %
Alto (flugelhorn)92 %89 %
Tenor (baritone)76 %82 %
Bass (tuba)72 %66 %
Within 50 msA note is credited to a voice when any written note matches it, so these are upper bounds. Measured on: 10 chorales, 1,887 notes.
MeasureMuScriptor mediumBasic Pitch
Soprano (trumpet)84 %74 %
Alto (flugelhorn)77 %66 %
Tenor (baritone)48 %71 %
Bass (tuba)45 %46 %
Figure 8. From a quartet recording the top voice is found and the tenor is mostly lost. First view: onset F-measure at ±100 ms of the line picked out for each player. Other views: the share of each player's notes that appear anywhere in the transcription of the quartet (upper bounds: any written note may match). This is why Brasscribe writes only the melody and the bass from the audio of an ensemble, and arranges the parts between them. n: 10 chorales, 1,887 notes. Shown here: One player's line taken out whole; the other views, with n per view, are tabulated in the appendix of data tables. Source: eval/baselines.json, suite seat-voices (runs in CI); the recall views from per-piece.json, computed with the gate's scorer but not gated.

Figure 9 evaluates those arranged parts against the composer's. Given the harmony of the score, the quartet arranger chooses the composer's pitch class for about four notes in five, with 0.94 parallel fifths or octaves per 100 chord changes, against 0.79 in the chorales themselves. From a recording it matches about half, with 5.4 parallels per 100. That the arrangers never write an unplayable note or cross parts holds by construction and is not evidence of musical quality.

Share of the chorale's own alto and tenor notes that the quartet arranger also chose (same note name, any octave).

Data table
Brass quartetsMeasured on: 10 chorales, 1,887 notes.
MeasureFrom the score's harmonyFrom the recording (MuScriptor)From the recording (Basic Pitch only)
Alto80 %53 %17 %
Tenor78 %43 %18 %
Figure 9. Arranged inner parts match the composer's about four notes in five from the right harmony, and about half from a recording. The quartet arranger sees a melody, a bass and the harmony, and writes the two parts between. Matching the composer is a strict test: other inner parts can be equally good music. The two kinds of row are matched differently: from the score's harmony note by note at the same onset; from a recording by the note name sounding 50 ms after each attack. n: 10 chorales, 1,887 notes. Source: eval/baselines.json, suites arrange and quartet-audio (the chorale parts run in CI).

5.6 Agreement between models and the confidence marks

Figure 10 shows that agreement between models is strong evidence that a note is correct, provided that SwiftF0 is one of the models. This motivates the rule of Section 3.3. The marks derived from it are weaker (Figure 11): few correct notes are marked, but most wrong notes carry no mark, and on the French horn none did. The recordings in Figure 11 are those on which the confidence model was fitted, so performance on new recordings should be expected to be no better. The marks judge pitch and onset only; they say nothing about the most frequent error, a correct note at the wrong position in the bar (Section 5.3).

Share of the notes that are right, by which of the three models found them, on solo lines separated from a mix.

Data table
Separated solo lines
MeasurePrecisionMeasured on
SwiftF0 and MuScriptor99 %110 notes
All three98 %193 notes
SwiftF0 and Basic Pitch91 %578 notes
SwiftF0 alone54 %297 notes
MuScriptor and Basic Pitch, not SwiftF036 %14 notes
Figure 10. A note produced by SwiftF0 and one further model is correct in about nine cases in ten or more; other combinations in about half or fewer. Agreement between models is what the confidence of a note is built on. The trumpet lines were separated with Mega-53 from the ten chorale mixes and two Slakh songs. n: SwiftF0 and MuScriptor: 110 notes; All three: 193 notes; SwiftF0 and Basic Pitch: 578 notes; SwiftF0 alone: 297 notes; MuScriptor and Basic Pitch, not SwiftF0: 14 notes. Source: eval/baselines.json, suite solo-vote (runs locally); note counts from docs/shared/research/10-benchmark-results.md via measured.toml.

Of a solo line's wrong notes, the share that carry a ?; and of its right notes, the share marked anyway. Each is a mean over the instrument's recordings.

Data table
ChoraleBricksFew notes are wrong to begin with: 3 on all the trumpet recordings together and 8 on the flugelhorn's, so a share here can turn on one note. The confidence model behind the marks was fitted on these same recordings.
MeasureWrong notes that carry a ?Right notes marked ? anywayMeasured on
Trumpet33 %0.4 %20 recordings
Flugelhorn33 %0.2 %20 recordings
French horn0 %0.4 %11 recordings
Baritone20 %0.5 %24 recordings
Trombone32 %0 %8 recordings
Tuba18 %3.6 %10 recordings
Figure 11. The ? marks are rarely false alarms, but they catch a third of the wrong notes at best. Doubt marks on solo lines. These recordings are fitting data for the marks, not a test of them, so a new recording should be expected to do no better. n: Trumpet: 20 recordings; Flugelhorn: 20 recordings; French horn: 11 recordings; Baritone: 24 recordings; Trombone: 8 recordings; Tuba: 10 recordings. Source: eval/baselines.json, suite solo-instruments, q_hit and q_false_alarm; counts from measured.toml.

5.7 Fast passages

Figure 12 follows the notes of fast figures (runs, repeated notes, trills) through the stages of the solo path. The f0-based note segmentation retains about six notes in ten. With reference beats nearly all of these reach the score; with tracked beats more are lost at quantisation. Of the trills and two-note alternations, 41 % keep at least four fifths of their notes with reference beats and 27 % with tracked beats. The further views show the effect of the rules that were added for fast notes: recall of the figure's notes improved and remains below one half. The clips are assembled from samples and the rules were developed on them.

Share of the notes in a fast run, repeated-note figure or trill that are still there after each stage of the solo path.

Data table
Stage by stageMeasured on: 253 clips frozen in the repository: built from samples, dry and with a room; development data.
MeasureBeats givenBeats tracked (Beat This! small0)
1. Detected by the f0 estimator (SwiftF0)59 %59 %
2. In the solo line after the vote60 %60 %
3. After snapping to the beat grid55 %47 %
4. Written in the score55 %47 %
Before and after, beats givenMeasured on: 306 clips built from samples; the rules were developed on them.
MeasureBefore the fast-note rulesAfter
Notes of the fast figure written (recall)26 %44 %
All notes of the clip (F-measure)40 %54 %
Trills and alternations that keep four fifths of their notes11 %30 %
Before and after, beats trackedMeasured on: 306 clips built from samples; the rules were developed on them.
MeasureBefore the fast-note rulesAfter
Notes of the fast figure written (recall)22 %36 %
All notes of the clip (F-measure)37 %48 %
Trills and alternations that keep four fifths of their notes9.5 %18 %
Figure 12. Fast figures: about six notes in ten are detected, and fewer remain after notation. Where fast notes are lost. The clips are rendered from real trumpet samples, dry and with a hall, with an exact reference; they are development data, not held out. The before-and-after views are the full set of 306 clips, so their numbers differ a little from the frozen part the gate holds. n: Stage by stage: 253 clips frozen in the repository: built from samples, dry and with a room; development data · Before and after, beats given: 306 clips built from samples; the rules were developed on them · Before and after, beats tracked: 306 clips built from samples; the rules were developed on them. Shown here: Stage by stage; the other views, with n per view, are tabulated in the appendix of data tables. Source: eval/baselines.json, suite fast-notes (runs in CI); before and after from docs/brasscribe/plan/fast-notes.md, section 7.2, via measured.toml.

5.8 Model and design comparisons

The comparisons that determined the design are summarised in Figure 13, Figure 14, Figure 15 and Figure 16.

Every model and setting that was run on the ten brass quartets, at the strict 50 ms tolerance.

Data table
Brass quartetsMeasured on: 10 chorales, 1,887 notes.
MeasureOnset within 50 msOnset and note end both right
MuScriptor medium (used)0.560.48
Basic Pitch, recording retuned to A = 440 (used)0.480.33
Basic Pitch, as recorded0.470.28
MuScriptor large0.440.39
MuScriptor medium, conditioned on brass instruments0.330.30
MuScriptor large, conditioned on brass instruments0.280.25
Figure 13. The larger model did not score higher, and conditioning the model on brass instruments lowered its F-measure. Model comparison on the same recordings, as means over the pieces. n: 10 chorales, 1,887 notes. Source: eval/baselines.json, suite chorales-transcription (runs in CI).
  • Model size. MuScriptor large scored lower than medium on the quartets (Figure 13); medium is used.
  • Instrument conditioning. Conditioning MuScriptor on brass instruments lowered its onset F-measure from 0.56 to 0.33; conditioning is not used.
  • Retuning. Shifting the recording to A = 440 Hz before Basic Pitch raised its F-measure from 0.60 to 0.62 at ±100 ms; the combined result of both models did not change (0.76 and 0.76).
  • Separation before transcription scored 0.70 against 0.56 on the mixture, on 2 synthesised songs (Figure 3, third view).
  • Beat-tracker size made little difference on single parts (Section 5.3); phones keep the smaller checkpoint.

Each model alone on trumpet lines separated with Mega-53, and the line kept when two models agree. Onset F-measure at ±100 ms.

Data table
Separated solo linesMeasured on: the solo lines of 10 chorales and 2 songs.
MeasureOnset F-measure
SwiftF00.75
Notes at least two models found0.73
MuScriptor0.50
Basic Pitch0.42
Figure 14. On a solo line separated from a mixture the f0 estimator has the highest F-measure, and MuScriptor, the best model on mixtures, a much lower one. Why a solo is built on the f0 estimator. The two-model line has precision 0.89 and recall 0.66. n: the solo lines of 10 chorales and 2 songs. Source: eval/baselines.json, suite solo-vote (runs locally).

On a solo line that a separator has already extracted, the f0 estimator is the strongest single source and MuScriptor, the best model on mixtures, is far weaker (Figure 14).

The best single route against the combined vote of the models, on each kind of recording. Onset F-measure at ±100 ms.

Data table
Recordings
MeasureBest single routeSD95 % CIVote between the modelsSD95 % CIMeasured on
Brass quartets0.750.060.71–0.780.7610 chorales, 1,887 notes
Chamber brass0.880.040.86–0.910.888 pieces, 5,129 notes
Trumpet over a band (separated first)0.700.632 songs
Figure 15. Keeping only what two models agree on does not find more notes, and on a full band it finds fewer. A negative result. The vote is therefore used to say how sure a note is, not to decide which notes are written. n: Brass quartets: 10 chorales, 1,887 notes; Chamber brass: 8 pieces, 5,129 notes; Trumpet over a band (separated first): 2 songs. Source: eval/baselines.json, suites consensus-chorales, consensus-urmp, consensus-slakh and the transcription suites.

Retaining only notes on which two models agree does not improve the F-measure, and lowers it on a full band (Figure 15). The vote is therefore used to estimate confidence and not to select notes.

Three ways to choose which notes the melody may be drawn from, scored on the melody alone. Onset F-measure at ±100 ms.

Data table
MelodyMeasured on: 2 songs, 8 chamber pieces, 10 chorales.
MeasureTrumpet over a band (Slakh)Chamber brassBrass quartets
Notes MuScriptor found (used)0.650.720.90
Only notes both models found0.670.650.80
Everything either model found0.360.550.36
Figure 16. For the melody, the top line of MuScriptor's notes beats the top line of everything both models found, on every set. Choosing the melody. Basic Pitch hears overtones as high notes, and they land above the tune when everything is kept. n: 2 songs, 8 chamber pieces, 10 chorales. Source: eval/baselines.json, suite melody (runs locally).

For melody extraction, the top line of the notes supported by MuScriptor is preferable to the top line of the union of both models on all three sets (Figure 16).

5.9 Computational cost

Figure 17 gives the time and memory of each model on one laptop, Figure 18 the time of each model call in one solo job, and Figure 19 the parity of the models converted to Core ML or to the Open Neural Network Exchange format (ONNX) with their originals. Whole jobs were timed on an Apple M-series computer (the notes do not record the model): a solo of 30 s took 21.4–23.2 s, and a band recording of 247 s took 96 s, of which MuScriptor accounted for 75 s. A converted model is enabled in the apps only if its output matches that of the original with F ≥ 0.98; three conversions failed this requirement, and each failure determined where a model may run. Time and memory were not measured on any phone or tablet, on a Windows computer, or on a computer with less memory.

Each model alone, converted for Apple devices and in its reference framework, on one Apple M5 Pro laptop with 48 GB. No phone has been timed.

Data table
TimeLower is faster. The scale is logarithmic: the models differ by a factor of thousands. Measured on: 30 s of audio (Beat This!: 60 s), median of 1 to 5 runs.
MeasureConverted (best conversion that passes)Reference framework on the processor
SwiftF0 (pitch of one line)0.02 s0.13 s
Basic Pitch (notes)0.18 s0.44 s
Beat This! small0 (beats)0.05 s0.55 s
BS-RoFormer SW (separation, 6 stems)13 s126 s
Mega-53 (separation, 53 stems)29 s208 s
MuScriptor medium (notes), on the processor only97 s84 s
MemoryThe whole process, with the framework loaded. Logarithmic scale. Measured on: 30 s of audio (Beat This!: 60 s), median of 1 to 5 runs.
MeasureConverted (best conversion that passes)Reference framework on the processor
SwiftF0 (pitch of one line)0.38 GB0.32 GB
Basic Pitch (notes)0.57 GB0.70 GB
Beat This! small0 (beats)0.49 GB0.48 GB
BS-RoFormer SW (separation, 6 stems)3.8 GB3.6 GB
Mega-53 (separation, 53 stems)21 GB14 GB
MuScriptor medium (notes), on the processor only5.0 GB2.5 GB
Figure 17. The three small models take under a second per minute of music; the two separators take 13 s and 29 s, and Mega-53 21 GB of memory. Speed and memory per model. MuScriptor's only passing conversion runs on the processor and is slower than the original; the engine runs the original on the graphics processor, which the next figure times. n: 30 s of audio (Beat This!: 60 s), median of 1 to 5 runs. Shown here: Time; the other views, with n per view, are tabulated in the appendix of data tables. Source: convert/reports/*.json (made by the scripts in convert/; python3 convert/summarize.py prints them).

Seconds each model call took in one solo job, as the engine runs them, each starting its own process.

Data table
Solo jobMeasured on: one job on one M-series Mac.
MeasureSeconds
Beats (Beat This!)1.8 s
Pitch of the line (SwiftF0)0.30 s
Notes (Basic Pitch)2.5 s
Notes (MuScriptor, graphics processor)9.1 s
Pitch contour for note ends0.30 s
Figure 18. A 30-second solo on the computer: MuScriptor is 9.1 s of the model time, the f0 estimator 0.30 s. One job, one machine: a single measurement, not a benchmark. The whole job took longer (the text below): arranging and exporting come on top. n: one job on one M-series Mac. Source: docs/shared/research/13-performance-audit.md via measured.toml.

How closely a converted model gives the original's result: F-measure of its notes against the original's own output (for the beat tracker, of its beats and downbeats at ±50 ms).

Data table
ConversionsMeasured on: the 10 chorale and 8 chamber mixes and their single parts; separators on four 30-second excerpts.
MeasureParity with the original
SwiftF0, full precision1.000
SwiftF0, half precision on the Neural Engine: fails0.370
SwiftF0 split: only its middle on the Neural Engine0.999
Basic Pitch, full precision1.000
Basic Pitch, half precision: fails0.815
Beat This! small0, half precision0.994
BS-RoFormer SW, half precision0.999
Mega-53, half precision0.999
MuScriptor, ONNX1.000
MuScriptor, Core ML half precision: fails a stricter test0.999
Figure 19. Half precision on the Neural Engine broke two small models: SwiftF0 fell to 0.37 and Basic Pitch to 0.82. Conversions to Core ML and ONNX, against the gate. MuScriptor writes its notes one token after another, so it must also produce the very same tokens as the original; its Core ML form does not, although its notes nearly match. n: the 10 chorale and 8 chamber mixes and their single parts; separators on four 30-second excerpts. Source: convert/reports/*.json, parity (the lower score where a report has two).

6 Discussion

6.1 Error analysis

  • Rhythm is the dominant error. On single parts the onset F-measure is 0.80 and the written F-measure 0.27 (Section 5.3). Most of the difference is attributable to the beat grid: with reference beats the same rules place 95 % of correct notes (Section 5.2).
  • Metre. The bar length is wrong for most single parts. A part in triple time written in quadruple time has every bar line misplaced, whatever the quality of its notes.
  • Inner voices. From a quartet, the tenor is recovered with F = 0.31 and the bass with 0.69, against 0.92 for the top voice (Section 5.5).
  • Octave errors are infrequent. Of the reference notes of the quartets, 2.7 % were missed while MuScriptor produced a note one or two octaves away at the same time, and 11 % with Basic Pitch; for chamber music the rates are 1.2 % and 4.1 %. The count includes estimated notes that belong to another part an octave away and therefore overstates the error.
  • Fast passages lose about four notes in ten before notation (Section 5.7).
  • Ornaments are split into extra notes. On control clips that should remain one note, with tracked beats, a fall adds 0.20 notes per reference note, a scoop 0.30, a rip 0.85 and a wide downward vibrato 0.70; an ordinary vibrato adds 0.05. No control clip was notated as a trill.
  • Triplets were examined only on computed onset times, not on recordings: with 15 ms of timing jitter, 97 % of triplet eighth notes are placed correctly. The evaluation sets contain no recorded triplets with a reference.
  • Durations. With reference beats, 54 % of chamber-music notes and 88 % of chorale notes receive the notated duration (Section 5.2).

6.2 Threats to validity

  • Tuning on the test data. The thresholds of the rules were selected on the recordings on which they are scored, and the confidence marks are evaluated on their fitting data (Section 4.2). The reported values are optimistic to an unknown degree.
  • Sample size. The sets have 8 to 10 pieces, and the route comparison on a full band rests on 2 synthesised songs. The confidence intervals are correspondingly wide, and one agreement class contains 14 notes.
  • Representativeness. The chorales are slow and rhythmically simple and were recorded one player at a time; neither they nor the chamber pieces resemble a full band in a hall.
  • Synthetic material. The fast-note clips are assembled from single sampled notes, and the full-band songs are synthesised.
  • Tolerance. Several suites (single parts end to end, voices, the vote on separated lines, melody, consensus) store results at a ±100 ms onset tolerance only, which is more lenient than the ±50 ms convention; where both exist, both are reported and ±50 ms is given first.
  • Measurement. Timing results come from single runs on one computer; some values were measured once and are not gated (Appendix).

7 Limitations

  • No brass-band recording with reference notes was evaluated; behaviour on a full band is not measured.
  • Only one route is checked end to end, and without reference notes: the soloist-with-orchestra route must reproduce a frozen output. The solo, ensemble and pop routes are composed of the same tested components but have no end-to-end gate.
  • In an ensemble score only the melody and the bass are transcribed; the inner parts are arranged.
  • The confidence marks cover pitch and onset only and miss most wrong notes.
  • Computational cost is known for one laptop and for no phone.

8 Conclusion and future work

On brass ensembles the best pretrained model reaches an onset F-measure of 0.56 to 0.69 at ±50 ms and 0.75 to 0.88 at ±100 ms (recall 0.63 to 0.69 and 0.83 to 0.88), and deterministic rules notate the notes correctly when the beat grid is correct. The evaluation indicates that the score is lost mainly at beat and metre tracking, in particular for single parts, and that inner voices of an ensemble are not recoverable with the models examined. The confidence marks are precise but have low recall.

Future work follows from these results: beat and metre estimation for monophonic lines, including a step in which the user confirms the time signature and the first downbeat; a held-out brass set, so that a tuned rule can be distinguished from one that generalises; confidence estimates that cover rhythm; brass-band data, both rendered from public-domain scores and recorded with corrected scores; and measurements of computational cost on phones.

9 Reproducibility statement

  • Code. The system, the benchmarks and this page are in the public repository github.com/181192/brasscribe. This page was built from the commit named under the title.
  • Commands. pixi run scribe bench list lists the suites; pixi run scribe bench cpu rescores all of them from stored model outputs and compares them with eval/baselines.json; pixi run python site/shared/research/per_piece.py produces the per-piece values; site/build.sh builds this page and fails if a stated finding no longer holds.
  • Reproducible from the repository alone. The following results are rescored in continuous integration on every change, from pinned ChoraleBricks annotations and cached model outputs kept in the repository, and can be reproduced without downloading a dataset: the quartet views of Figure 3 and Figure 1, Figure 13, Figure 7, Figure 11, the first view of Figure 8, Figure 9, the stage view of Figure 12, and the chorale series of Figure 5. Figure 17 and Figure 19 are read from committed reports; rerunning them requires the models.
  • Requiring a local download. Results on URMP (the chamber views, Figure 6), on Slakh and on separated stems (Figure 10, Figure 14, Figure 15, Figure 16) need the datasets and the models on a local computer; the per-piece values and the beat F-measure need the evaluation data as well. The frozen output of the commercial recording cannot be reproduced by a reader.
  • Data availability. ChoraleBricks, URMP and Slakh are available from their publishers (Table 2). The commercial recording is not distributed.
  • Models and scorer. All models are public; MuScriptor's weights require acceptance of its licence. The checkpoints are MuScriptor medium, Basic Pitch as published, SwiftF0 as published, Beat This! small0 and final0, BS-RoFormer SW and MVSep Mega-53; the package versions are pinned in pixi.lock. Scoring uses mir_eval 0.8.2.
  • Not gated. The values listed in the Appendix were measured once; their sources and commands are recorded in site/shared/research/measured/measured.toml.

10 Ethics and licence statement

  • Model licences. MuScriptor's weights are licensed CC BY-NC 4.0; scores produced with them may be used non-commercially only, and each user accepts the licence and downloads the weights personally. No licence is stated for the BS-RoFormer SW and Mega-53 weights; they are downloaded by the user's computer from their publishers and are not redistributed by the project. SwiftF0, Basic Pitch and Beat This! are MIT or Apache-2.0 (Table 1).
  • Dataset licences are given in Table 2. The repository contains small data derived from ChoraleBricks (CC BY 4.0) and one test file derived from 30 s of a URMP track.
  • Copyright. No copyrighted music is published on this page or distributed with the apps; the built-in example is a public-domain hymn tune.
  • Privacy. No recording, and no score derived from one, is sent to a server.
  • Telemetry. ONNX Runtime, the Microsoft library that executes the models in the Android and Windows apps and in parts of the engine, reports its own usage to Microsoft with a device identifier. THIRD_PARTY_NOTICES.md documents what is sent.

Acknowledgements

This work depends on the authors of the models and datasets cited above, who made them openly available. Parts of the software and of this text were written with the assistance of an AI coding assistant and reviewed by the author.

How to cite

K.-A. Kalliainen, “Brasscribe: Transcribing brass recordings into band scores with pretrained models and deterministic rules,” The Brasscribe project, 2026. [Online]. Available: https://github.com/181192/brasscribe

@misc{kalliainen2026brasscribe,
  author       = {Kalliainen, Kristoffer-Andre},
  title        = {Brasscribe: Transcribing Brass Recordings into Band Scores
                  with Pretrained Models and Deterministic Rules},
  year         = {2026},
  howpublished = {The Brasscribe project},
  url          = {https://github.com/181192/brasscribe}
}

References

  1. E. Benetos, S. Dixon, Z. Duan, and S. Ewert, “Automatic music transcription: An overview,” IEEE Signal Processing Magazine, vol. 36, no. 1, pp. 20–30, 2019, doi: 10.1109/MSP.2018.2869928.
  2. J. Gardner, I. Simon, E. Manilow, C. Hawthorne, and J. Engel, “MT3: Multi-task multitrack music transcription,” in Proc. Int. Conf. on Learning Representations (ICLR), 2022, arXiv: 2111.03017.
  3. S. Chang, E. Benetos, H. Kirchhoff, and S. Dixon, “YourMT3+: Multi-instrument music transcription with enhanced transformer architectures and cross-dataset stem augmentation,” in Proc. IEEE Int. Workshop on Machine Learning for Signal Processing (MLSP), 2024, pp. 1–6, doi: 10.1109/MLSP58920.2024.10734819.
  4. S. Rouard, M. Krause, A. Roebel, C.-J. Simon-Gabriel, and A. Défossez, “MuScriptor: An open model for multi-instrument music transcription,” in Proc. Int. Society for Music Information Retrieval Conf. (ISMIR), 2026, arXiv: 2607.08168. Code: MIT. Weights (muscriptor-medium): CC BY-NC 4.0, gated.
  5. R. M. Bittner, J. J. Bosch, D. Rubinstein, G. Meseguer-Brocal, and S. Ewert, “A lightweight instrument-agnostic model for polyphonic note transcription and multipitch estimation,” in Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 781–785, doi: 10.1109/ICASSP43922.2022.9746549. Software: Basic Pitch, Apache-2.0.
  6. C. Hawthorne, E. Elsen, J. Song, A. Roberts, I. Simon, C. Raffel, J. Engel, S. Oore, and D. Eck, “Onsets and Frames: Dual-objective piano transcription,” in Proc. Int. Society for Music Information Retrieval Conf. (ISMIR), 2018, pp. 50–57, doi: 10.5281/zenodo.1492341.
  7. J. W. Kim, J. Salamon, P. Li, and J. P. Bello, “CREPE: A convolutional representation for pitch estimation,” in Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2018, pp. 161–165, doi: 10.1109/ICASSP.2018.8461329.
  8. L. Nieradzik, “SwiftF0: Fast and accurate monophonic pitch detection,” 2025, arXiv: 2508.18440. Software: swift-f0, MIT.
  9. pitch-benchmark, software repository. [Online]. Available: https://github.com/lars76/pitch-benchmark. Accessed: 13 Nov. 2026. MIT. Published under the same account as SwiftF0.
  10. S. Rouard, F. Massa, and A. Défossez, “Hybrid transformers for music source separation,” in Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2023, pp. 1–5, doi: 10.1109/ICASSP49357.2023.10096956.
  11. W.-T. Lu, J.-C. Wang, Q. Kong, and Y.-N. Hung, “Music source separation with band-split RoPE transformer,” in Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2024, pp. 481–485, doi: 10.1109/ICASSP48485.2024.10446843.
  12. python-audio-separator, software repository. [Online]. Available: https://github.com/nomadkaraoke/python-audio-separator. Accessed: 13 Nov. 2026. Code: MIT. Distributes the BS-RoFormer SW checkpoint; no licence is stated for the weights.
  13. R. Solovyev, I. Kiselev, A. Stempkovskiy, and T. Gabruseva, “Music-Source-Separation-Training (MSST): A unified framework for training and evaluating music demixing models,” 2026, arXiv: 2607.23395. Code: MIT. The MVSep Mega-53 checkpoint is published with release v1.0.21 of the repository; no licence is stated for the weights.
  14. F. Krebs, S. Böck, and G. Widmer, “An efficient state-space model for joint tempo and meter tracking,” in Proc. Int. Society for Music Information Retrieval Conf. (ISMIR), 2015, pp. 72–78, doi: 10.5281/zenodo.1414966.
  15. S. Böck, F. Korzeniowski, J. Schlüter, F. Krebs, and G. Widmer, “madmom: A new Python audio and music signal processing library,” in Proc. ACM Int. Conf. on Multimedia, 2016, pp. 1174–1178, doi: 10.1145/2964284.2973795.
  16. F. Foscarin, J. Schlüter, and G. Widmer, “Beat this! Accurate beat tracking without DBN postprocessing,” in Proc. Int. Society for Music Information Retrieval Conf. (ISMIR), 2024, arXiv: 2407.21658. Code and published weights: MIT.
  17. D. Meredith, “The ps13 pitch spelling algorithm,” Journal of New Music Research, vol. 35, no. 2, pp. 121–159, 2006, doi: 10.1080/09298210600834961.
  18. C. Cancino-Chacón, S. D. Peter, E. Karystinaios, F. Foscarin, M. Grachten, and G. Widmer, “Partitura: A Python package for symbolic music processing,” in Proc. Music Encoding Conf. (MEC), 2022, arXiv: 2206.01071. Apache-2.0.
  19. C. L. Krumhansl, Cognitive Foundations of Musical Pitch. New York, NY, USA: Oxford University Press, 1990.
  20. W3C Music Notation Community Group, “MusicXML version 4.0,” Final Community Group Report, 2021. [Online]. Available: https://www.w3.org/2021/06/musicxml40/. Accessed: 13 Nov. 2026.
  21. S. Balke, A. Berndt, and M. Müller, “ChoraleBricks: A modular multitrack dataset for wind music research,” Transactions of the International Society for Music Information Retrieval, vol. 8, no. 1, pp. 39–54, 2025, doi: 10.5334/tismir.252. Data: version 1.1.0, doi: 10.5281/zenodo.20849469, CC BY 4.0.
  22. B. Li, X. Liu, K. Dinesh, Z. Duan, and G. Sharma, “Creating a multitrack classical music performance dataset for multimodal music analysis: Challenges, insights, and applications,” IEEE Transactions on Multimedia, vol. 21, no. 2, pp. 522–535, 2019, doi: 10.1109/TMM.2018.2856090. The dataset page states no licence.
  23. E. Manilow, G. Wichern, P. Seetharaman, and J. Le Roux, “Cutting music source separation some Slakh: A dataset to study the impact of training data quality and quantity,” in Proc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2019, pp. 45–49, doi: 10.1109/WASPAA.2019.8937170. Data: doi: 10.5281/zenodo.4599666, CC BY 4.0.
  24. University of Iowa Electronic Music Studios, “Musical instrument samples.” [Online]. Available: https://theremin.music.uiowa.edu/MIS.html. Accessed: 13 Nov. 2026. The page permits use “for any projects, without restrictions”.
  25. Versilian Studios, “VSCO 2 Community Edition,” sample library. [Online]. Available: https://github.com/sgossner/VSCO-2-CE. Accessed: 13 Nov. 2026. CC0 1.0.
  26. OpenAIR, the Open Acoustic Impulse Response Library, University of York. [Online]. Available: https://www.openair.hosted.york.ac.uk/. Accessed: 13 Nov. 2026. Impulse response of St Margaret’s Church (National Centre for Early Music), York. The repository’s notices record the licence as CC BY 4.0; the library’s site could not be reached to confirm it when this list was checked.
  27. C. Raffel, B. McFee, E. J. Humphrey, J. Salamon, O. Nieto, D. Liang, and D. P. W. Ellis, “mir_eval: A transparent implementation of common MIR metrics,” in Proc. Int. Society for Music Information Retrieval Conf. (ISMIR), 2014, pp. 367–372, doi: 10.5281/zenodo.1416528.
  28. M. Bay, A. F. Ehmann, and J. S. Downie, “Evaluation of multiple-F0 estimation and tracking systems,” in Proc. Int. Society for Music Information Retrieval Conf. (ISMIR), 2009, pp. 315–320, doi: 10.5281/zenodo.1418241.
  29. B. Efron and R. J. Tibshirani, An Introduction to the Bootstrap. Boston, MA, USA: Springer, 1993, doi: 10.1007/978-1-4899-4541-9.
  30. F. Foscarin, F. Korzeniowski, and R. Vogl, “Masked diffusion enables coherent beat tracking,” in Proc. Int. Society for Music Information Retrieval Conf. (ISMIR), 2026, arXiv: 2608.04624. Software and pretrained models: md_beat_this, CC BY-NC 4.0.

Appendix A: Values that are not gated

Table 4 lists the values on this page that were measured once and are not held by the regression gate, with their sources.

Table 4. Values measured once. Each is recorded with its source and command in site/shared/research/measured/measured.toml.
ValueAs reportedSource
Notes per agreement class (Figure 10)193, 110, 578, 297, 14docs/shared/research/10-benchmark-results.md, three-way vote
Whole jobs (Section 5.9)30 s solo: 21.4–23.2 s; 247 s band: 96 sdocs/shared/research/13-performance-audit.md, M5
Model calls of a solo job (Figure 18)1.8 s, 0.30 s, 2.5 s, 9.1 s, 0.30 sSame document, M6
Quantiser before penalty scaling (Section 5.2)89 % position, 89 % within beat, 76 % durationdocs/shared/research/10-benchmark-results.md, rhythm
Fast notes before and after (Figure 12)Figure recall 26 % → 44 % on 306 clipsdocs/brasscribe/plan/fast-notes.md, Section 7.2
Triplets on computed onsets (Section 6.1)97 %Same document, Section 8.2
Beat F-measure of the tracked beats (Section 5.2)0.82 and 0.78site/shared/research/measured/per-piece.json
Recall per voice (Figure 8)Means over pieces, computed with the gate's scorersite/shared/research/measured/per-piece.json
Model speed, memory and conversion parity (Figure 17, Figure 19)See the figuresconvert/reports/*.json

Back to top