Harmonic Representations in Symbolic Music Transformers are Localized and Controllable
Using Aria, a LLaMA-based symbolic music transformer evaluated on Bach chorales, we find that harmonic decisions are causally localized progressively from middle layers onward.
ProofPaper ↗
Key points
- Transformer models trained on symbolic music have become remarkably capable.
- However, it is yet unclear how they represent music structure, and whether understanding their internal representations can be used to unlock methods for model control.
- We examine this through a case study of harmonic analysis: when a music transformer navigates a cadence, what does it internally "know" about its harmonic context, and can that knowledge be used to affect the generation?
- We then confirm these representations are actionable for music generation control: steering via latent directions corresponding to harmonic changes reliably produces a deceptive or minor resolution, while preserving structural coherence and generation quality.
Sources (1)
- [1]Harmonic Representations in Symbolic Music Transformers are Localized and ControllablearXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 11, 10:24 PM
Using Aria, a LLaMA-based symbolic music transformer evaluated on Bach chorales, we find that harmonic decisions are causally localized progressively from middle layers onward.
Transformer models trained on symbolic music have become remarkably capable.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 11, 2026Channel Match: Calibrating Shape Latents for Few-Step 3D Generation
- Oct 10, 2026Understanding Unified VLMs: From Attention Probing to Guidance Tuning for Multimodal Generation
- Oct 10, 2026vllm-sr/d3
- Oct 9, 2026Image Diffusion Models Are Good 3D Human Motion Generators
- Oct 9, 2026LiquidAI/LFM2.5-24B-A2B
- Oct 9, 2026Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation