Fretscribe: Transcribing Recordings into Tablature with Pretrained Models and a Fingering Search

1 Introduction

Tablature shows where a note is played on a fretted instrument: one line per string and a fret number on it. Transcribing tablature from audio adds two problems to automatic music transcription (AMT) [1]. First, a pitch can usually be played at several positions on the neck, and a microphone recording does not identify the one that was used. Second, the instrument is rarely alone: within a song it is mixed with voice, percussion and other instruments, and a guitar plays chords whose notes overlap.

Fretscribe shares its listening models and its rules for rhythm with Brasscribe, which is described in a companion paper. It runs on the user's own devices; every tablature is produced on the user's computer, and the phone app carries no listening models. This paper covers what is specific to tablature. Figure 1 summarises note detection for each instrument, alone and within a song.

The contributions of this paper are:

• a description of an audio-to-tablature pipeline built from pretrained models, deterministic note rules and a fingering search (Section 3);
• an evaluation on real guitar recordings with annotated strings, with a separation between the players used to tune the rules and those used to report results, and on synthesised material for bass, ukulele and mandolin (Sections 4 and 5);
• an analysis of the errors and of the threats to validity, including a case in which the tuned and the reported players differ markedly (Sections 5.6 and 6).

2 Related work

Note transcription and f0 estimation. Basic Pitch is a lightweight, instrument-agnostic note transcriber [2]; SwiftF0 estimates the fundamental frequency (f0) of a monophonic line [3]. Beat This! tracks beats and downbeats [4], and a band-split RoPE Transformer (BS-RoFormer) [5] separates a mixture into stems.

Tablature transcription. Kehling et al. estimate string and fret together with score parameters for electric guitar [6]. Wiggins and Kim estimate tablature frame by frame with a convolutional network (TabCNN) and define tablature precision, recall and a tablature disambiguation rate [7]; GuitarSet provides recordings with a pickup per string, so that every note has an annotated string [8]. In contrast to these learned approaches, Fretscribe does not estimate the string from the audio: it detects notes and then chooses positions by a search over playable fingerings, as in the optimum-path formulation of Sayegh [9] and the A*-based arrangement of Burlet and Fujinaga [10].

Gap. We found no annotated recordings of ukulele or mandolin, and no dataset of bass recordings with aligned notes that we could use; the evaluation for those instruments therefore rests on synthesised audio (Section 4.1).

3 Method

Figure 2 shows the pipeline.

3.1 Listening models

BS-RoFormer SW [11] separates a song into stems, and the stem of the instrument is analysed. It has stems for bass and for guitar and none for ukulele or mandolin, which are read from the guitar stem; the step is omitted when the recording contains the instrument alone. Beat This! provides beats and downbeats. Basic Pitch provides the notes, and SwiftF0 a second estimate that is used only where one note sounds at a time.

3.2 From notes to quantised notes

Rules remove overtones that the note model reported as notes, discard notes outside the range of the instrument and, for a bass, test whether a line detected an octave too high was played there. The notes are then quantised to the beat grid with written durations by the rules described in the companion paper. Each note then receives a confidence between 0 and 1. It is set by rule, not by a fitted model as in the companion paper: 0.5 plus half of the amplitude that Basic Pitch reports for the note. A note is placed in doubt, and marked with a ?question mark, when its amplitude is low and, in a monophonic line, SwiftF0 did not detect it at that pitch; or, within a chord, when its amplitude is low, or moderately low and the pitch is not detected again within a short time. The thresholds were set by hand.

3.3 Fingering

The fingering is a search and not a learned model, in the tradition that treats fingering as an optimum path through candidate positions [9], extended to chords as a graph search by Burlet and Fujinaga [10]; the cost terms follow the observation that guitarists limit hand span and favour low positions [12]. Notes that start together are placed together, each on its own string, within the span of a hand. Each candidate position has a cost: the stretch of the fingers, the height of the hand on the neck, and the distance the hand moves from the preceding notes, measured in millimetres along the neck, so that the same rule applies to a bass and a ukulele. The path of least total cost through the passage, found by dynamic programming, is the tablature. Open strings and known chord shapes also enter the cost. The search is deterministic, and a separate check confirms that every position is playable. A note that cannot be fingered together with the others is omitted until the tablature is playable, so playability holds by construction: 0.1–0.4 % of the notes are omitted on GuitarSet, and 16–30 % on the rendered strummed passages for ukulele and mandolin. Tuning, capo and playing style are parameters of this stage only, so a different tuning does not require the audio to be analysed again.

4 Experimental setup

4.1 Datasets

Table 1 lists the evaluation sets. Annotated recordings of real instruments were available for guitar only.

4.2 Splits and data leakage

No model was trained. The rules have thresholds; for guitar these were chosen on three of the six GuitarSet players and results are reported for the other three, so that a threshold is not evaluated on the recordings on which it was chosen. Section 5.6 shows the value of that separation. For bass there is no such split: the Slakh set is development data, and one threshold of the octave test was chosen on the set that scores it. The rendered passages have held-out groups, with exceptions that eval/README.md lists rule by rule.

4.3 Metrics

All measures lie between 0 and 1; higher is better. Note-level measures are computed with mir_eval.transcription, onset-only, and all of them use the standard ±50 ms onset tolerance. An octave error is a reference note that remains unmatched while an estimated note one or two octaves away starts within the tolerance.

• Note-level precision, recall and F-measure

  Notes are evaluated as in mir_eval [16], which implements the MIREX note-tracking convention [17]. An estimated note matches a reference note when its onset lies within ±50 ms of the reference onset and its pitch within ±50 cents; each note is matched at most once. With TP matched notes, FP unmatched estimated notes and FN unmatched reference notes:

  P = TP / (TP + FP),   R = TP / (TP + FN),   F = 2PR / (P + R) (1)

  Offsets are ignored unless stated. Where they are used (onset+offset F-measure), the offset must also lie within max(50 ms, 20 % of the reference note's duration). All results in this paper use the ±50 ms tolerance.

• String agreement and string recall

  On a fretted instrument a pitch can usually be played at several positions. Let M be the matched notes of (1), S ⊆ M those written on the string the player used (and hence at the same fret), and N_ref the number of reference notes:

  string agreement = |S| / |M|,   string recall = |S| / N_ref (2)

  These are note-level counterparts of the frame-level tablature disambiguation rate and tablature recall of Wiggins and Kim [7]; values at the two levels are not directly comparable.

• Precision and recall of the confidence marks

  A note whose confidence, set by the rule of Section 3.2, falls below a threshold is marked ?question mark. Let W be the written notes that are wrong (unmatched under (1)) and Q the marked notes:

  P_? = |Q ∩ W| / |Q|,   R_? = |Q ∩ W| / |W| (3)

  The false-alarm rate is the share of correct notes that are marked. In the drawing three notes are wrong and two are marked, one of them correctly: R_? = 1/3, P_? = 1/2.

• Metre accuracy

  No time signature is annotated by the beat tracker, so metre is scored by bar length. For a set of parts, with L the bar length in quarter notes:

  M = |{parts : L_written = L_score}| / |parts| (4)

  A tracker that follows the beat at half or double speed is not penalised by this measure.

In addition we report tempo accuracy, the share of recordings whose estimated tempo is within 4 % of the reference tempo; chord precision and recall, (1) restricted to notes that start together in groups of three or more; and playability, which is a requirement and not a score: a tablature with an unplayable position fails the benchmark.

4.4 Protocol

Model outputs are computed once and stored; scribe bench rescores each suite and fails when a value has moved by more than the tolerance in eval/baselines.json. Note-level precision, recall and F-measure, the octave-error rate, and tempo and metre accuracy are means over recordings. Chord precision and recall, string agreement and string recall, and the precision and recall of the confidence marks are ratios of counts pooled over all recordings of a condition. Per-recording values are not stored for the tablature suites, so no dispersion or confidence interval is reported, and no significance tests were carried out; differences are reported descriptively. None of the tablature suites runs in continuous integration, because they require data or models that are not in the repository; they are run on one computer.

5 Results

5.1 Note detection

Figure 1 (Section 1) gives the onset F-measure per instrument. A bass line, which is monophonic and occupies its own register, is detected almost as well within a mixture (0.86) as alone (0.89); at the bottom of a five-string bass the value falls to 0.73–0.75. A guitar recorded alone reaches 0.85 on single lines and 0.72 on chords. Within a song the guitar reaches 0.43. The rendered passages of the second view score highly; they are the weakest evidence in this paper, and their held-out groups contain melodies and picked passages only. The third view gives the strummed and chord-melody passages of the same groups, on which one rule was chosen: the ukuleles reach 0.74–0.86, and the mandolin 0.72 alone and 0.65 within a song, with octave-error rates of 13–15 %. No held-out rendered passage tests chords.

Table 2 gives precision and recall beside the F-measure for the recorded and Slakh conditions.

5.2 String and fret

Figure 3 compares the assigned strings with the annotated ones. In chordal playing, where the hand has few alternatives, 70 % of the correctly detected notes are on the player's string. For single lines the value is 46 %: the search minimises movement, whereas players also choose positions for timbre and for the phrase that follows. Relative to all notes played, 38 % of the notes of a single line are both detected and on the player's string. The tablature is playable in every case, but it is not a record of the performance.

5.3 Tempo and metre

Figure 4 reports tempo and metre. Nearly all of these recordings are in quadruple time, and the number of beats in the bar is correct for 92–100 % of them; on the 8 rendered passages in other metres it is correct for 50 %. Tempo is wrong more often, in part by a factor of two: accepting half and double tempo raises the share for single guitar lines from 61 % to 71 %.

5.4 Confidence marks

Figure 5 gives the precision and recall of the marks. A marked note is usually wrong, but most wrong notes are not marked: on a bass the precision is 92–94 % and the recall 27–36 %. A tablature without marks has therefore not been verified.

5.5 A bass without its low frequencies

Small microphones pass little of the lowest octave of a bass. Figure 6 simulates this by high-pass filtering the bass before analysis. Up to a 200 Hz cut-off the line is still detected, with a lower F-measure. With a steeper filter at 300 Hz the F-measure collapses and octave errors rise: 10 % of the reference notes are missed while a note one or two octaves away is produced, against 2.4 % without filtering. At that cut-off the octave test shifted 63 % of the lines as a whole, and none up to 200 Hz; that shift accounts for the collapse. The unfiltered row covers the Slakh basses only, whereas the filtered rows also include the low bass lines. The test filters synthesised basses; no phone was used.

5.6 Tuned and reported players

Figure 7 compares the players on whom the rules were tuned with those reported. Most measures are similar in the two groups. One is not: the recall of chord notes within single-line playing, which would have been reported as 88 % without the split. The results for bass, for the low bass lines and for most rendered passages have no such split.

6 Discussion

6.1 Error analysis

• An instrument within a song. A guitar in a mixture reaches F = 0.43, with a chord recall of 39 %. The value is no higher for songs with a single guitar (0.36).
• String assignment. Fewer than half of the correctly detected notes of a single line are on the player's string (Section 5.2).
• Octave errors are infrequent for a bass within a song (1.3 % of reference notes) and more frequent at the bottom of a five-string bass (4.7–5.4 %) and when the low frequencies are missing (Section 5.5). For guitar chords the rate is 2.9 %.
• Chords. In chordal playing the chord recall is 72 % and the chord precision 72 %.
• Tempo is wrong for up to four recordings in ten (Section 5.3).
• Tremolo. A mandolin melody in fast repeated notes reaches F = 0.64–0.66 alone, and 0.52 and 0.17 within a song, on one rendered passage each.
• Notes removed in cleaning. The overtones and low-amplitude notes that the cleaning removes amount to 31–42 % of the written notes in the tremolo passages and 15–17 % on IDMT-SMT-Guitar, against 2.4–4.6 % for the reported GuitarSet players.

6.2 Threats to validity

• Synthesised audio. All bass results, and all results for ukulele and mandolin, come from synthesised or rendered audio; the latter from one sampled instrument each and a few passages.
• Tuning on the test data. Only the guitar rules were tuned on players other than those reported (Section 4.2).
• No dispersion. Results are means without a measure of spread (Section 4.4).
• Comparability. The string measures are note-level and are not comparable with frame-level tablature metrics in the literature; several results of the companion paper exist at a ±100 ms tolerance only, and values should be compared only at the same tolerance.
• Simulation. The microphone experiment is a filter, not a recording.

7 Limitations

• No recording of a real bass, ukulele or mandolin with reference notes was evaluated.
• No recording made with a phone was evaluated.
• No tablature benchmark runs in continuous integration; the values are gated only when the benchmark is run on a computer that holds the data.
• String agreement is known for guitar only, because only the guitar sets annotate the string.
• The confidence marks cover pitch and onset only, not the string, the rhythm or the tempo.
• Playing techniques such as bends, slides and hammer-ons are not evaluated.
• Nearly all recordings are in quadruple time; other metres rest on 8 rendered passages.
• The computational cost of a tablature job was not measured; the models are those of the companion paper, which reports their cost on one laptop.

8 Conclusion and future work

With pretrained models and deterministic rules, a bass line and a guitar recorded alone are transcribed well enough to serve as a draft, whereas a guitar within a song is not. The fingering search always yields playable tablature, but for single lines it agrees with the player's choice of string for fewer than half of the notes. The confidence marks are precise and have low recall.

Future work includes recordings of real basses, ukuleles and mandolins with reference notes, and recordings made with phones; transcription of a guitar within a mixture; a fingering cost that reflects how guitarists choose positions, evaluated against the annotated strings of GuitarSet; a tablature benchmark in continuous integration on redistributable data; and measurements of computational cost.

9 Reproducibility statement

• Code. The system, the benchmarks and this page are in the public repository github.com/181192/brasscribe; the page was built from the commit named under the title.
• Commands. pixi run scribe bench fretted rescores the tablature suites from stored model outputs and compares them with eval/baselines.json; eval/README.md gives the commands that build each set and run the models. site/build.sh builds this page and fails if a stated finding no longer holds.
• Reproducible from the repository alone. None of the results: no tablature suite runs in continuous integration, and every suite needs a dataset or model outputs that are not in the repository. The page itself can be rebuilt from the stored baselines.
• Requiring a local download. All results: GuitarSet, IDMT-SMT-Guitar and Slakh must be downloaded and the models run locally; the rendered passages are generated locally with FluidSynth and the SoundFont.
• Data availability. GuitarSet, IDMT-SMT-Guitar and Slakh are available from their publishers. The rendered passages are generated by scripts in the repository.
• Method. The fingering search is documented in core/targets/fretted/README.md.

10 Ethics and licence statement

• Model licences. Basic Pitch is Apache-2.0; SwiftF0 and Beat This! are MIT. No licence is stated for the BS-RoFormer SW weights; they are downloaded by the user's computer from their publisher and are not redistributed by the project.
• Dataset licences are given in Table 1. IDMT-SMT-Guitar is licensed for non-commercial use without derivatives; it is used for evaluation only and nothing derived from it is distributed.
• Privacy. No recording, and no tablature derived from one, is sent to a server.
• Telemetry. ONNX Runtime, the Microsoft library that executes models in parts of the engine, reports its own usage to Microsoft with a device identifier; THIRD_PARTY_NOTICES.md documents what is sent.

Acknowledgements

This work depends on the authors of the models and datasets cited above, who made them openly available. Parts of the software and of this text were written with the assistance of an AI coding assistant and reviewed by the author.

How to cite

K.-A. Kalliainen, “Fretscribe: Transcribing recordings into tablature with pretrained models and a fingering search,” The Fretscribe project, 2026. [Online]. Available: https://github.com/181192/brasscribe

@misc{kalliainen2026fretscribe,
  author       = {Kalliainen, Kristoffer-Andre},
  title        = {Fretscribe: Transcribing Recordings into Tablature
                  with Pretrained Models and a Fingering Search},
  year         = {2026},
  howpublished = {The Fretscribe project},
  url          = {https://github.com/181192/brasscribe}
}

References

1. E. Benetos, S. Dixon, Z. Duan, and S. Ewert, “Automatic music transcription: An overview,” IEEE Signal Processing Magazine, vol. 36, no. 1, pp. 20–30, 2019, doi: 10.1109/MSP.2018.2869928.
2. R. M. Bittner, J. J. Bosch, D. Rubinstein, G. Meseguer-Brocal, and S. Ewert, “A lightweight instrument-agnostic model for polyphonic note transcription and multipitch estimation,” in Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 781–785, doi: 10.1109/ICASSP43922.2022.9746549. Software: Basic Pitch, Apache-2.0.
3. L. Nieradzik, “SwiftF0: Fast and accurate monophonic pitch detection,” 2025, arXiv: 2508.18440. Software: swift-f0, MIT.
4. F. Foscarin, J. Schlüter, and G. Widmer, “Beat this! Accurate beat tracking without DBN postprocessing,” in Proc. Int. Society for Music Information Retrieval Conf. (ISMIR), 2024, arXiv: 2407.21658. Code and published weights: MIT.
5. W.-T. Lu, J.-C. Wang, Q. Kong, and Y.-N. Hung, “Music source separation with band-split RoPE transformer,” in Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2024, pp. 481–485, doi: 10.1109/ICASSP48485.2024.10446843.
6. C. Kehling, J. Abeßer, C. Dittmar, and G. Schuller, “Automatic tablature transcription of electric guitar recordings by estimation of score- and instrument-related parameters,” in Proc. Int. Conf. on Digital Audio Effects (DAFx), 2014. [Online]. Available: https://www.dafx.de/paper-archive/2014/. Accessed: 13 Nov. 2026.
7. A. Wiggins and Y. Kim, “Guitar tablature estimation with a convolutional neural network,” in Proc. Int. Society for Music Information Retrieval Conf. (ISMIR), 2019, pp. 284–291, doi: 10.5281/zenodo.3527800.
8. Q. Xi, R. M. Bittner, J. Pauwels, X. Ye, and J. P. Bello, “GuitarSet: A dataset for guitar transcription,” in Proc. Int. Society for Music Information Retrieval Conf. (ISMIR), 2018, pp. 453–460, doi: 10.5281/zenodo.1492449. Data: version 1.1.0, doi: 10.5281/zenodo.3371780, CC BY 4.0.
9. S. I. Sayegh, “Fingering for string instruments with the optimum path paradigm,” Computer Music Journal, vol. 13, no. 3, 1989, doi: 10.2307/3680014.
10. G. Burlet and I. Fujinaga, “Robotaba guitar tablature transcription framework,” in Proc. Int. Society for Music Information Retrieval Conf. (ISMIR), 2013. [Online]. Available: https://archives.ismir.net/ismir2013/paper/000217.pdf. Accessed: 13 Nov. 2026.
11. python-audio-separator, software repository. [Online]. Available: https://github.com/nomadkaraoke/python-audio-separator. Accessed: 13 Nov. 2026. Code: MIT. Distributes the BS-RoFormer SW checkpoint; no licence is stated for the weights.
12. H. Heijink and R. G. J. Meulenbroek, “On the complexity of classical guitar playing: Functional adaptations to task constraints,” Journal of Motor Behavior, vol. 34, no. 4, pp. 339–351, 2002, doi: 10.1080/00222890209601952.
13. C. Kehling, A. Männchen, and A. Eppler, “IDMT-SMT-Guitar dataset,” version 1.0.0, Zenodo, doi: 10.5281/zenodo.7544110. CC BY-NC-ND 4.0.
14. E. Manilow, G. Wichern, P. Seetharaman, and J. Le Roux, “Cutting music source separation some Slakh: A dataset to study the impact of training data quality and quantity,” in Proc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2019, pp. 45–49, doi: 10.1109/WASPAA.2019.8937170. Data: doi: 10.5281/zenodo.4599666, CC BY 4.0.
15. MuseScore General SoundFont, version 0.2.0. [Online]. Available: https://ftp.osuosl.org/pub/musescore/soundfont/MuseScore_General/. Accessed: 13 Nov. 2026. MIT. Rendered with FluidSynth (LGPL-2.1).
16. C. Raffel, B. McFee, E. J. Humphrey, J. Salamon, O. Nieto, D. Liang, and D. P. W. Ellis, “mir_eval: A transparent implementation of common MIR metrics,” in Proc. Int. Society for Music Information Retrieval Conf. (ISMIR), 2014, pp. 367–372, doi: 10.5281/zenodo.1416528.
17. M. Bay, A. F. Ehmann, and J. S. Downie, “Evaluation of multiple-F0 estimation and tracking systems,” in Proc. Int. Society for Music Information Retrieval Conf. (ISMIR), 2009, pp. 315–320, doi: 10.5281/zenodo.1418241.