TytanMikJas/DART-Degradation-Aware-Recurrent-Transformer

DART: A Degradation-Aware Recurrent Transformer for Archival Film Restoration

ACCV 2026 · Mikołaj Jastrzębski, Wojciech Kozłowski, Kamil Adamczewski · Wrocław University of Science and Technology

Paper (arXiv:2607.21219) · Code · Project page · AbsoluteDegradation pipeline and benchmark · Datasets (Kaggle)

_{Real archival footage → DART restoration → predicted soft defect mask (bright = damaged).}

Pretrained weights of DART, the model reported as "DART (Ours)" in Tables 2–4 of the paper. Archival film carries compound damage (scratches, dust, blur, noise, flicker, photometric aging) and has no clean reference. DART predicts a continuous defect mask under direct supervision, propagates it through time, and uses it both to gate temporal fusion and to condition the restoration backbone on how damaged each frame is.

Model details

• Architecture: bidirectional recurrent network. Each step warps the previous hidden state, soft mask and frame with a frozen RAFT optical flow. A Dilation Pyramid MaskNet (dilation rates 1, 2, 3) predicts the soft defect mask M_t, which gates the fusion of the encoded frame with the aligned state. An AdaLN-Zero condition encoder turns the mask and the temporal residual into a vector that modulates every block of a Swin restoration transformer. A fusion convolution merges the forward and backward states.
• Parameters: 6.58M, including the frozen RAFT-Sintel flow estimator (5.26M) and 1.32M trained DART parameters.
• Input: a clip of RGB frames (monochrome footage, grey replicated to 3 channels), normalised to [-1, 1]. Any resolution works: frames are padded to a multiple of 16. Clips are restored in 7-frame windows with stride 3, as in training and evaluation.
• Output: restored frames in [-1, 1] plus per-frame soft defect masks in [0, 1] (the MaskNet logits of both propagation directions).
• Efficiency (1×7×3×256×256 clip, NVIDIA A100, fp32): 372 GFLOPs per frame, 0.35 GB of memory, 29.5 FPS.

Files

File	Format	SHA-256
dart.safetensors	safetensors, fp32	d78a388c76ffb2d71cd0fee55f44e3dde695112a40c540a7b67cc0590cfa73d3
dart.pth	PyTorch state dict, fp32	dcdb10f4a9b047af557eb415a6999c9ea0258c252807c18e368e5a2100926010

Both files hold the same state dict of the DART network (VideoBackbone, 423 tensors). The training discriminator and optimizer state are not included. The bundled RAFT weights are identical to the official raft-sintel.pth.

Usage

git clone https://github.com/TytanMikJas/DART-Degradation-Aware-Recurrent-Transformer.git DART && cd DART
uv sync
uv run python scripts/download_weights.py      # GitHub release, falls back to this repository

# Restore a video file or a folder of frames
uv run python scripts/inference.py --input my_clip.mp4 --output outputs/my_clip --save-masks --save-video

Or download the weights directly from this repository:

hf download TytanMikJas/DART-Degradation-Aware-Recurrent-Transformer dart.safetensors --local-dir pretrained_models

From Python (run from the repository root):

import torch
from scripts.inference import load_dart

net = load_dart("pretrained_models/dart.safetensors", config_name="train", device=torch.device("cuda"))
clip = torch.rand(1, 7, 3, 256, 256, device="cuda") * 2 - 1   # (B, T, 3, H, W) in [-1, 1], H, W divisible by 16
with torch.no_grad():
    restored = net(clip)                                         # (B, T, 3, H, W) in [-1, 1]
mask = torch.sigmoid(net.last_gate_mask_logits_forward)          # (B, T-1, 1, H, W), frames 1..T-1

Inference runs on a GPU and also on the CPU (slowly). The weights load with strict=True into the network built from configs/train.yaml.

Training data

• Clean source: REDS train_sharp (720p, 24 fps; CC BY 4.0).
• Degradations: generated online by the AbsoluteDegradation physics-inspired film-degradation pipeline (NeurIPS 2026, Evaluations and Datasets Track). For every 256×256 crop it yields a sharp ground-truth frame, its degraded version and a continuous artifact mask. The mask is the target of DART's mask supervision.

Training procedure

• Objective: L1 + VGG19 perceptual loss (λ_rec = 1), a Temporal-PatchGAN hinge loss (λ_adv = 0.01), and mask supervision with class-balanced BCE + Dice on the MaskNet logits of both propagation directions (λ_mask = 0.5, λ_dice = 0.5).
• Optimisation: Adam (β = 0.9, 0.99), learning rate 2·10⁻⁴ with linear decay, 20 epochs, 7-frame windows, 256×256 crops, FP16 mixed precision, frozen RAFT.
• Hardware: 2× NVIDIA A100, 1 clip per GPU (PLGrid, Athena cluster).

Evaluation data

Real archival footage has no clean reference, so evaluation on it uses no-reference metrics learned from human quality ratings: CLIPIQA+, MUSIQ and MANIQA (via pyiqa).

Benchmark	Content	Source
AbsoluteDegradation — Real Scene-Split Eval	13,252 lossless PNG frames of public-domain archival films (1896–1918, Library of Congress); compact subset of the AbsoluteDegradation real benchmark. Primary target domain.	Kaggle
SRWOV	216 lower-resolution, JPEG-compressed real-world old-film clips. Secondary benchmark.	MambaOFR repository
Synthetic Degraded Eval	The 30 REDS val_sharp clips degraded by AbsoluteDegradation, with ground truth; full-reference proxy (PSNR, SSIM, LPIPS) used for the ablation study.	Kaggle

The evaluation protocol (scripts/test.py + scripts/evaluate.py) scores the frames of any model, so new methods can be compared directly with the table below.

Results

All architectures were trained from scratch on the same AbsoluteDegradation data with their published hyperparameters (Table 2 of the paper; higher is better, bold = best).

Model	AbsoluteDegradation CLIPIQA+	MUSIQ	MANIQA	SRWOV CLIPIQA+	MUSIQ	MANIQA
Input frames	0.2483	26.8418	0.1619	0.3890	42.8591	0.2307
BasicVSR	0.4049	49.5797	0.2844	0.4086	57.6442	0.3104
BasicVSR++	0.3685	42.9144	0.2375	0.4428	58.0442	0.3231
ShiftNet	0.4309	46.7724	0.2764	0.4578	57.9334	0.3336
DeepRemaster	0.3574	44.7540	0.2542	0.3722	53.5473	0.2637
RTN	0.4220	46.8737	0.2661	0.4441	58.4585	0.3321
MambaOFR	0.4190	47.2436	0.2777	0.4402	57.3551	0.3210
DART	0.4334	51.2304	0.3011	0.4488	59.4263	0.3427

Ablations (Table 4): removing the AdaLN conditioning, the Dilation Pyramid MaskNet, the recurrent mask or the mask supervision each lowers every no-reference metric on AbsoluteDegradation. Removing AdaLN alone drops MANIQA from 0.301 to 0.263.

Only the DART weights are released here. The baselines' weights are not part of this release; see their official repositories.

Intended use and limitations

DART is intended for research on the restoration of monochrome archival film, and for restoring such footage for viewing and study. Known limitations (Sec. 5 of the paper):

• It targets monochrome footage; colour film is not supported.
• Like all flow-based propagators, it relies on optical-flow accuracy, so extreme occlusions or highly persistent defects are challenging.
• It has no internal shot-boundary handling: across a cut the propagated state still carries the previous shot. Split clips at cuts, or reset the state at detected cuts.
• Defects more severe than those produced by the synthesis pipeline are localised, but their attenuation may be limited.
• The reported quality relies on no-reference metrics, which do not fully substitute for human evaluation.

Restoration models can alter historical footage. Keep the original material and disclose restoration when publishing restored archival content.

License

The DART weights and code are released under the MIT License. The bundled RAFT-Sintel weights come from princeton-vl/RAFT (BSD-3-Clause, Copyright (c) 2020, princeton-vl). The training data REDS is licensed CC BY 4.0.

Citation

@inproceedings{jastrzebski2026dart,
  title     = {{DART}: A Degradation-Aware Recurrent Transformer for Archival Film Restoration},
  author    = {Jastrz{\k{e}}bski, Miko{\l}aj and Koz{\l}owski, Wojciech and Adamczewski, Kamil},
  booktitle = {Asian Conference on Computer Vision (ACCV)},
  year      = {2026}
}

@inproceedings{jastrzebski2026absolutedegradation,
  title     = {{AbsoluteDegradation}: A Physics-Inspired Synthetic Film-Degradation Pipeline and Archival Film Restoration Benchmark},
  author    = {Jastrz{\k{e}}bski, Miko{\l}aj and Glinkowski, Dawid and Zieli{\'n}ski, Dawid and Borkowski, Daniel and Koz{\l}owski, Wojciech and Adamczewski, Kamil},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS), Evaluations and Datasets Track},
  year      = {2026}
}

Acknowledgements

We gratefully acknowledge Polish high-performance computing infrastructure PLGrid (HPC Center: ACK Cyfronet AGH) for providing computer facilities and support within computational grant no. PLG/2025/018640.