AKUSPACE LTX-2.5 Audio LoRA v0.5 — checkpoint 11500, card, config, six A/B examples

.gitattributes

@@ -33,3 +33,11 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+examples/beat-dual-delay.mp3 filter=lfs diff=lfs merge=lfs -text
+examples/beat-empty-club.mp3 filter=lfs diff=lfs merge=lfs -text
+examples/beat-medium-room.mp3 filter=lfs diff=lfs merge=lfs -text
+examples/beat-outdoor-day.mp3 filter=lfs diff=lfs merge=lfs -text
+examples/dry-beat.mp3 filter=lfs diff=lfs merge=lfs -text
+examples/dry-voice.mp3 filter=lfs diff=lfs merge=lfs -text
+examples/voice-cathedral.mp3 filter=lfs diff=lfs merge=lfs -text
+examples/voice-small-room.mp3 filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,199 @@
+---
+license: other
+license_name: ltx-2-community-license-agreement
+license_link: https://github.com/Lightricks/LTX-2/blob/main/LICENSE.md
+base_model: Lightricks/LTX-2.5
+pipeline_tag: audio-to-audio
+tags:
+  - ltx
+  - ltx-2.5
+  - lora
+  - audio-to-audio
+  - audio
+  - spatial-audio
+  - room-acoustics
+  - reverb
+  - ambience
+  - comfyui
+---
+
+# AKUSPACE — LTX-2.5 Audio LoRA v0.5
+
+**Acoustic space control for LTX-2.5.** Give it a dry recording and a prompt naming
+an acoustic treatment, and it re-renders that recording as though it were captured
+in that space — while the performance stays put on the timeline.
+
+Trigger word: `AKUSPACE`
+
+- Website and A/B examples: https://akuspace.pages.dev
+- ComfyUI nodes: https://github.com/koshimazaki/ComfyUI-Koshi-Nodes
+
+## What it does
+
+Three kinds of treatment, each with a text-controlled amount:
+
+| Mode | What it is | Options |
+|---|---|---|
+| **Space** | rooms a sound sits *in* | small room (0.67 s), medium room (1.9 s), empty club (1.2 s), cathedral |
+| **Place** | environments a sound sits *among* | outdoor day, outdoor night |
+| **Sound effects** | processing a sound goes *through* | modular granular delay |
+
+Each treatment has a level axis, and the level word is part of the trained caption:
+
+- **gentle** — subtle inclusion of the effect
+- **moderate** — the sweet spot, and the level to showcase
+- **heavy** — pushes the dry signal into sound-effect territory. Shipped as
+  **experimental**: it is the edge of the trained range, not a general-purpose setting.
+
+Rooms and sound effects carry all three amounts. **Outdoor places carry only
+`gentle` and `heavy`** — an ambience bed is a separate recording rather than a
+tail, so it survives being scaled down but not up.
+
+The level word occupies the same position in every caption, which is what makes it
+a consistent semantic control rather than an arbitrary number.
+
+## It holds sync with picture
+
+Envelope cross-correlation of each published example against its own dry reference,
+measured on this checkpoint (step 11500), 2 ms resolution:
+
+| Example | Offset | Peak correlation |
+|---|---|---|
+| Voice · small room | −20 ms | 0.96 |
+| Voice · cathedral | −22 ms | 0.97 |
+| Beat · medium room | −16 ms | 0.34 |
+| Beat · empty club | −20 ms | 0.30 |
+| Beat · outdoor day | −24 ms | 0.32 |
+| Beat · dual delay | −22 ms | 0.34 |
+
+The offset is **16–24 ms across every mode and both source types** — a consistent,
+small lead rather than per-clip drift, and inside lip-sync tolerance. So a
+performance can be moved from a studio into a cathedral in post, by describing it,
+without re-shooting or re-syncing.
+
+**Read the correlation column carefully.** It measures how much the amplitude
+envelope *changed*, and reverb changes it by design — most of all on
+transient-dense material, where the tail fills the gaps between hits. The low
+figures on percussion are the expected signature of the effect working, not a
+defect, and they are reported here rather than omitted. Envelope correlation is a
+timing measure, not a quality measure; judge quality by ear.
+
+## Use
+
+```text
+AKUSPACE female spoken voice through synthetic cathedral reverb, moderate
+```
+
+Supply the corresponding dry audio as the **audio-to-audio reference**.
+
+Suggested inference settings: **24 steps, CFG 4**.
+
+> **Turn prompt enhancement OFF.** The rewriter paraphrases away the trigger word
+> and the level word, which are exactly the tokens the control surface depends on.
+
+**This adapter is reference-conditioned.** It was trained only to transform a
+supplied reference, so plain text-to-audio with the LoRA loaded and no reference
+present generates near-silence at any strength. That is a usage constraint, not a
+defect. For video, the working chain is: dry voice → audio-to-audio space pass →
+image+audio-to-video, with the audio held fixed and the base model doing lip-sync.
+
+## Training
+
+- Base checkpoint: `Lightricks/LTX-2.5` (22B dev transformer, bf16)
+- LoRA rank / alpha: 32 / 32, dropout 0.0
+- Target modules: `audio_attn1`, `audio_attn2`, `audio_ff` — the audio branches
+  only. The video branch and both cross-modal attention paths are untouched, which
+  is why the spatial effect survives inside joint audio-video renders.
+- Optimiser: AdamW, learning rate 2e-4, batch size 1, bf16, seed 42
+- Steps: trained to 12,300; **this release ships step 11,500**, chosen by listening
+  across a 228-clip sweep of four candidate checkpoints against every trained cell.
+- Dataset: 266 paired clips — 228 train / 38 validation
+- Clip length 6.000 s, 48 kHz, −3 dBFS ceiling
+- Held out from training: one beat and one male voice source
+
+## How the dataset was built
+
+Every item is **the same performance twice** — once dry, once through a real
+acoustic treatment — so the model learns a transformation rather than an
+association.
+
+The material is my own, recorded and produced over several years: multiple speaking
+voices, beats and electronic music, and acoustic instruments (piano, saxophone,
+citola, handclaps). Treatments come from digital reverbs, analog and modular
+patches, and original field recordings for the outdoor beds — captured as spaces,
+as places, as sound effects, and across different room sizes. Nothing is scraped or
+third-party licensed.
+
+Sources were rendered as a deliberate grid — 14 sources × (4 rooms × 3 levels + one
+effect chain × 3 levels + 2 places × 2 levels) — so that no space is defined by one
+voice and no voice is defined by one space. Captions name the space and its decay
+time, putting the physical quantity into the text, which is what makes a continuous
+control surface reachable.
+
+Two decisions did more for quality than any hyperparameter:
+
+**Source coverage per space.** An earlier version trained on 8 sources produced
+reverb that read as a lo-fi proxy. Raising per-space coverage to 14, with no other
+change, is what made the rooms believable. Coverage, not capacity.
+
+**Not fading the tails.** Clips are cut at 6 seconds without a fade, leaving the
+reverb tail faintly ringing at the boundary (−44 to −76 dB). Fading to silence
+teaches the model that reverb *stops* at the window edge; leaving it ringing
+teaches that it continues, which is what lets a 6-second-trained adapter generate a
+coherent 15-second tail.
+
+A verification script gates the pipeline before every training run: grid
+completeness, byte-level file integrity, sample-rate and channel consistency,
+per-source level spread, and per-space processing depth. Each of those checks
+caught a real failure. The integrity check compares declared frame counts against
+audio data actually present — a truncated WAV keeps its original header, so every
+duration-based check reports a healthy file while the encoder silently skips it.
+
+## Limitations
+
+- **Sung vocals are out of distribution.** Every trained voice is *spoken*.
+- **Brass is out of distribution.** Trumpet appears only inside a single combined
+  piano-and-saxophone source. Results on solo brass are generalisation, not a
+  trained capability.
+- Requires an audio reference; it will not generate a space from text alone.
+- This is generative transformation, not physically accurate acoustic simulation.
+- It may alter wording, timing, pitch, or timbre — audio is regenerated, not
+  filtered.
+- Outdoor ambience can mask a quiet source, and beds are recordings rather than
+  reverb tails, so they scale down more gracefully than up.
+- The level axis is a learned caption control, not a calibrated wet/dry percentage.
+
+## Files
+
+| Path | What it is |
+|---|---|
+| `akuspace-ltx25-v0.5.safetensors` | The adapter — step 11500, bf16, rank 32 |
+| `config/a2a_v5_ltx25.yaml` | The training config this run used |
+| `examples/` | The six A/B pairs, 320k MP3 |
+
+The examples share two dry references. `dry-voice.mp3` pairs with
+`voice-small-room.mp3` and `voice-cathedral.mp3`; `dry-beat.mp3` pairs with
+`beat-medium-room.mp3`, `beat-empty-club.mp3`, `beat-outdoor-day.mp3` and
+`beat-dual-delay.mp3`. Rooms are at `moderate`, outdoor at `gentle`, dual delay at
+`heavy`. No gain or normalisation was applied on export, so the level relationship
+between dry and processed is the one the model produced.
+
+An interactive crossfade between each pair is at https://akuspace.pages.dev
+
+## Examples and provenance
+
+The published examples use a synthetic TTS voice and an AI-generated beat as dry
+sources. That is deliberate rather than a compromise: a replacement or synthetic
+voice track that needs to be seated in a scene is the realistic use case for this
+adapter, and it is the case the audio-dub category is about.
+
+Training data contains no synthetic-TTS-vendor material; the voices in the training
+grid are my own recordings.
+
+## Licence
+
+Released under the **LTX-2.x Community License** terms that govern the base model.
+Confirm the current terms at
+https://github.com/Lightricks/LTX-2/blob/main/LICENSE.md before commercial use —
+the community licence covers entities under $10M annual revenue, with separate
+commercial agreements above that threshold.

akuspace-ltx25-v0.5.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:623a32970c0984a80af5364d5257d1d3a7fa0c8fc112caee427d7aa631b3e361
+size 163714520

config/a2a_v5_ltx25.yaml

@@ -0,0 +1,112 @@
+model:
+  model_path: /workspace/models/ltx-2.5/diffusion_models/ltx-2.5-22b-dev-transformer-bf16.safetensors
+  text_encoder_path: /workspace/models/ltx-2.5/text_encoders/gemma4-12b-with-proj-ltx-2.5-bf16.safetensors
+  training_mode: lora
+  load_checkpoint: null
+  video_vae_path: /workspace/models/ltx-2.5/vae/ltx-2.5-video-vae-bf16.safetensors
+  audio_vae_path: /workspace/models/ltx-2.5/vae/ltx-2.5-audio-vae-bf16.safetensors
+lora:
+  rank: 32
+  alpha: 32
+  dropout: 0.0
+  target_modules:
+  - audio_attn1.to_k
+  - audio_attn1.to_q
+  - audio_attn1.to_v
+  - audio_attn1.to_out.0
+  - audio_attn2.to_k
+  - audio_attn2.to_q
+  - audio_attn2.to_v
+  - audio_attn2.to_out.0
+  - audio_ff.net.0.proj
+  - audio_ff.net.2
+training_strategy:
+  name: flexible
+  audio:
+    is_generated: true
+    latents_dir: audio_latents
+    conditions:
+    - type: reference
+      latents_dir: reference_audio_latents
+      probability: 1.0
+optimization:
+  learning_rate: 0.0002
+  steps: 12300
+  batch_size: 1
+  gradient_accumulation_steps: 1
+  max_grad_norm: 1.0
+  optimizer_type: adamw
+  scheduler_type: linear
+  scheduler_params: {}
+  enable_gradient_checkpointing: true
+acceleration:
+  mixed_precision_mode: bf16
+  quantization: null
+  load_text_encoder_in_8bit: false
+  offload_optimizer_during_validation: false
+data:
+  preprocessed_data_root: /workspace/Demos/data/acoustic-space-v5/.precomputed
+  num_dataloader_workers: 2
+validation:
+  samples:
+  - prompt: AKUSPACE male spoken voice in a small bathroom-like room, moderate bright close reflections and a short 0.67-second reverb decay, no background ambience
+    conditions:
+    - type: reference
+      audio: /workspace/Demos/data/acoustic-space-v5/audio/references/male_voice_02_small_room_0_67_mid.wav
+  - prompt: AKUSPACE male spoken voice through synthetic cathedral reverb, moderate wide diffuse reflections and a long decaying tail, no background ambience
+    conditions:
+    - type: reference
+      audio: /workspace/Demos/data/acoustic-space-v5/audio/references/male_voice_02_synthetic_cathedral_mid.wav
+  - prompt: AKUSPACE electronic rhythm loop outdoors in daytime, heavy open-air acoustics with continuous birdsong ambience
+    conditions:
+    - type: reference
+      audio: /workspace/Demos/data/acoustic-space-v5/audio/references/beat_02_outdoor_day_birds_high.wav
+  - prompt: AKUSPACE electronic rhythm loop outdoors at night, heavy open-air acoustics with crickets and distant car ambience
+    conditions:
+    - type: reference
+      audio: /workspace/Demos/data/acoustic-space-v5/audio/references/beat_02_outdoor_night_high.wav
+  - prompt: AKUSPACE male spoken voice through a modular granular delay, moderate scattered grains and unpredictable modulated echoes, no background ambience
+    conditions:
+    - type: reference
+      audio: /workspace/Demos/data/acoustic-space-v5/audio/references/male_voice_02_modular_granular_delay_mid.wav
+  negative_prompt: distorted, clipped, metallic, unintelligible speech, changed words, changed rhythm
+  video_dims:
+  - 512
+  - 512
+  - 153
+  frame_rate: 25.0
+  seed: 42
+  inference_steps: 24
+  interval: 250
+  guidance_scale: 4.0
+  stg_scale: 1.0
+  stg_blocks:
+  - 29
+  stg_mode: stg_av
+  generate_audio: true
+  generate_video: false
+  skip_initial_validation: true
+checkpoints:
+  interval: 100
+  keep_last_n: -1
+  precision: bfloat16
+flow_matching:
+  timestep_sampling_mode: shifted_logit_normal
+  timestep_sampling_params: {}
+hub:
+  push_to_hub: false
+  hub_model_id: null
+wandb:
+  enabled: false
+  project: ltx-acoustic-space-control
+  entity: null
+  tags:
+  - ltx2.3
+  - a2a
+  - ic-lora
+  - audio
+  - acoustic-space
+  - ableton
+  log_validation_videos: true
+seed: 42
+output_dir: /workspace/Demos/runs/acoustic-space-v5-ltx25

examples/beat-dual-delay.mp3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:de2a1a421a63edf1e20a1e973d6f5b769fce3cd934c5b075004f7bde39733110
+size 604844

examples/beat-empty-club.mp3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b8c74cf6a899cf51a1b7be9164c4c00048daf0d0694319163bf8bb4f7898dc78
+size 604844

examples/beat-medium-room.mp3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd11d77cb89b59dafabee0adb1ad252191fc0a4832a986f0df6a49b0dc3940e0
+size 604844

examples/beat-outdoor-day.mp3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:34057000f8d7235b8c027303bc7d59592c834a9fdc23a028e00dae54f719b98f
+size 604844

examples/dry-beat.mp3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d08c33c6d978c4577b7189ede5c4d3881381afda7eca3e2adaa9bfcc383df2ca
+size 605804

examples/dry-voice.mp3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36c1680c582f02ef8507278c0ae7f9aaf847cea97573a7be7cc450f937642f86
+size 605804

examples/voice-cathedral.mp3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4b1203f9b50f931e2c9e2b1c492dcb4ce3c499da0c4546c0bc0427146ff9e6f0
+size 604844

examples/voice-small-room.mp3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8919b5fbcfe1ad55b34329b306ef5d58d51250ea66125cfcda22aff55b056d1
+size 604844