erjui/CheXagent-2-3b-srrg-findings

CheXagent-2-3b: Structured Radiology Report Generation (Findings)

This model is a fine-tuned version of StanfordAIMI/CheXagent-2-3b for generating the FINDINGS section of structured chest X-ray radiology reports. It was trained using LoRA (Low-Rank Adaptation) on the csrrg_ift_dataset containing instruction-following examples from MIMIC-CXR and CheXpert+ datasets.

Model Description

This model performs Structured Radiology Report Generation (SRRG) for chest X-rays, specifically generating detailed findings sections that describe anatomical observations organized by body regions (lungs, heart, mediastinum, bones, etc.).

Key characteristics:

• Generates the FINDINGS section of radiology reports
• Trained on single chest X-ray examinations
• Produces structured, clinically relevant observations
• Fine-tuned with LoRA for parameter-efficient adaptation

Intended Use

Primary Use Cases

• Research on automated radiology report generation
• Development of clinical decision support systems
• Medical AI and multimodal model research
• Educational tools for radiology training

Intended Users

• Medical AI researchers
• Healthcare technology developers
• Clinical informatics specialists
• Radiology departments (research use only)

Out-of-Scope Use

• NOT intended for clinical diagnosis without physician review
• Should not replace human radiologists in clinical practice
• Requires validation before any clinical deployment

Training Details

Training Data

• Dataset: csrrg_ift_dataset (srrg_ift_dataset_findings subset)
• Training samples: ~181,874 instruction-following examples
• Data sources: MIMIC-CXR and CheXpert+ chest X-ray datasets
• Task format: Instruction fine-tuning with system-user-assistant conversations

Training Procedure

Fine-tuning method: LoRA (Low-Rank Adaptation)

LoRA Configuration:

• Rank (r): 32
• Alpha: 64
• Dropout: 0.1
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

Training hyperparameters:

• Learning rate: 2e-4
• Batch size: 4 per device
• Gradient accumulation steps: 32 (effective batch size: 128)
• Epochs: 1
• Optimizer: AdamW
• Learning rate scheduler: Cosine with 3% warmup
• Precision: bfloat16
• Attention implementation: Flash Attention 2
• Max sequence length: 2048
• Max images per sample: 1

Hardware:

• GPU: NVIDIA H100
• Training framework: HuggingFace Transformers + PEFT

Usage

Loading the Model

from transformers import AutoProcessor, AutoModelForVision2Seq
from PIL import Image
import torch

# Load model and processor
model_name = "erjui/CheXagent-2-3b-srrg-findings"
model = AutoModelForVision2Seq.from_pretrained(
    model_name,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
processor = AutoProcessor.from_pretrained("StanfordAIMI/CheXagent-2-3b", trust_remote_code=True)

# Load chest X-ray image (single image for SRRG)
image = Image.open("chest_xray.jpg")

# Prepare input
messages = [
    {
        "role": "system",
        "content": [{"type": "text", "text": "You are an expert radiologist."}]
    },
    {
        "role": "user",
        "content": [
            {"type": "text", "text": "Analyze the chest X-ray images and write the FINDINGS section of a radiology report. Use standard medical terminology and organize findings by anatomical regions."},
            {"type": "image"}
        ]
    }
]

# Process and generate (max_images_per_sample: 1)
inputs = processor(images=image, text=messages, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
generated_text = processor.decode(outputs[0], skip_special_tokens=True)

print(generated_text)

Expected Output Format

FINDINGS:
Lungs and Airways:
- No pleural effusion or pneumothorax detected
- Bibasilar atelectasis present

Cardiovascular:
- Mild left ventricular enlargement

Musculoskeletal and Chest Wall:
- Bilateral rib fractures noted

Citation

If you use this model, please cite:

@article{kang2025automated,
  title={Automated Structured Radiology Report Generation with Rich Clinical Context},
  author={Kang, Seongjae and Lee, Dong Bok and Jung, Juho and Kim, Dongseop and Kim, Won Hwa and Joo, Sunghoon},
  journal={arXiv preprint arXiv:2510.00428},
  year={2025}
}

Also cite the base model:

@article{chen2024chexagent,
  title={Chexagent: Towards a foundation model for chest x-ray interpretation},
  author={Chen, Zhihong and Varma, Maya and Delbrouck, Jean-Benoit and Paschali, Magdalini and Blankemeier, Louis and Van Veen, Dave and Valanarasu, Jeya Maria Jose and Youssef, Alaa and Cohen, Joseph Paul and Reis, Eduardo Pontes and others},
  journal={arXiv preprint arXiv:2401.12208},
  year={2024}
}

Model Card Authors

Seongjae Kang (erjui)

Model Card Contact

For questions or issues, please open an issue on the model repository.