---
license: other
license_name: apache-2.0-and-sam-license
license_link: LICENSE
library_name: transformers
pipeline_tag: image-text-to-text
tags:
  - multimodal
  - scientific
  - protein
  - rna
  - dna
  - molecule
  - weather
  - medical-imaging
base_model:
  - Qwen/Qwen3-VL-8B-Instruct
extra_gated_heading: You need to agree to Meta's SAM License to use the medical-image segmentation weights
extra_gated_description: >-
  The bulk of this model is Apache-2.0. The medical-image segmentation branch
  embeds SAM 3 weights, which are governed by Meta's SAM License (see
  SAM_LICENSE.txt). By accessing these weights you agree to that license,
  including its acceptable-use restrictions.
---

<div align="center">

[🤗 Model](https://huggingface.co/sais-org/MKB) &nbsp;•&nbsp; [💻 GitHub](https://github.com/Shanghai-Academy-of-AI-For-Science/MKB) &nbsp;•&nbsp; [📜 Technical Report](https://arxiv.org/pdf/2607.20557) &nbsp;•&nbsp; [⚖️ License: Apache-2.0 + SAM License](https://github.com/Shanghai-Academy-of-AI-For-Science/MKB/blob/main/LICENSE)

</div>

# 神珍 · Monkey King Bang (MKB)

**神珍 is a unified scientific multimodal foundation model** that
supports scientific **understanding and generation** across Earth science,
proteins, RNA, DNA, and small molecules. Native scientific encoders/decoders
wrap a shared **Qwen3-VL-8B-Instruct** backbone (about **11B** parameters in
total), so heterogeneous scientific data (sequences, molecular graphs, gridded
physical fields, medical images) are reasoned about and generated in one
representation space — natural language in and out, no per-task fine-tuning.

> 📜 Read the **[technical report](https://arxiv.org/pdf/2607.20557)** for architecture, training, and full benchmarks.

## Key features

- **Unified understanding *and* generation** across 7 modalities through one
  natural-language interface.
- **Seven modalities, one 8B backbone** (protein / RNA / DNA / molecule /
  weather / medical-image / text) via a modality router.
- **Native scientific encoders/decoders** (ESM-2, RNA/DNA ConvFormers, molecular
  graph encoder, Swin-ViT weather tower, SAM-based image path) preserve domain
  structure a generic tokenizer would destroy.

## Capabilities

| Modality      | Understanding | Generation |
|:--------------|:-------------:|:----------:|
| Protein       | ✅            | —          |
| RNA           | ✅            | ✅         |
| DNA           | ✅            | —          |
| Molecule      | ✅            | ✅         |
| Weather       | —             | ✅         |
| Medical image | —             | ✅         |
| Text          | ✅            | ✅         |

<sub>**Understanding** = classification / regression / scientific QA. **Generation**: RNA sequence design · Molecule text → SMILES · Weather 10-day global ERA5 0.25° forecast · Medical-image text-prompted segmentation (SAM 3-based; Meta SAM License).</sub>

## Benchmarks

**神珍** (8B backbone, ~11B total) vs **Biology-Instructions** (Llama-3.1-**8B**,
text-token, no scientific encoders) and **Intern-S1-Pro** (**~1T** MoE scientific
model). **Bold** = best; <u>underline</u> = second-best.

### Biological sequence understanding

| Task | Metric | 神珍 (~11B) | Biology-Instructions (8B) | Intern-S1-Pro (~1T) |
|:-----|:------:|:----------------:|:-------------------------:|:-------------------:|
| DNA · Epigenetic marks (EMP) | MCC | **71.99** | 3.64 | <u>14.02</u> |
| DNA · Promoter det. 300bp (PD300) | MCC | **91.17** | 58.18 | <u>82.65</u> |
| DNA · Core-promoter (CPD) | MCC | **66.35** | 44.54 | <u>54.60</u> |
| DNA · Enhancer activity (EA) | PCC | 52.64 | <u>53.28</u> | **55.16** |
| RNA · ncRNA function | Acc | **91.46** | <u>63.09</u> | 34.50 |
| RNA · Modification | AUC | **96.03** | <u>59.06</u> | 57.77 |
| RNA · APA isoform | R² | <u>79.87</u> | 59.01 | **82.95** |
| RNA · CRISPR on-target | Spearman ρ | **28.76** | -0.02 | <u>15.69</u> |
| Protein · Stability | Spearman ρ | **70.63** | 60.25 | <u>60.82</u> |
| Protein · Fluorescence | Spearman ρ | <u>70.12</u> | 2.57 | **78.14** |
| Protein · Enzyme Commission | Fmax | <u>68.65</u> | 19.79 | **72.70** |
| Protein · Solubility | Acc | <u>67.26</u> | 63.02 | **67.60** |
| Cross-modal · RPI (RNA–protein) | MCC | **76.49** | <u>74.26</u> | 58.51 |
| Cross-modal · AAN (antibody–antigen) | MCC | <u>42.96</u> | 1.06 | **44.76** |
| Cross-modal · EPI (enhancer–promoter) | MCC | <u>-0.03</u> | **3.37** | -1.30 |

<sub>Aggregate over 20 biological-understanding benchmarks: 神珍 matches or beats the ~1T Intern-S1-Pro on 10/20 and the same-scale 8B text-token baseline on 16/20.</sub>

### Molecule understanding (SMolInstruct)

| Task | Metric | 神珍 (~11B) | LlaSMol |
|:-----|:------:|:----------------:|:-------:|
| BBBP | Acc | **96.95** | 74.60 |
| HIV | Acc | **97.00** | 96.70 |
| SIDER | Acc | **71.00** | 70.70 |
| ClinTox | Acc | 92.36 | **93.10** |
| ESOL | RMSE ↓ | **0.550** | 1.150 |
| Lipophilicity | RMSE ↓ | **0.628** | 1.010 |

### Earth-science forecasting — vs ECMWF HRES (day-10, global ERA5 0.25°)

| Variable | Metric | 神珍 (~11B) | ECMWF HRES (NWP) |
|:---------|:------:|:----------------:|:----------------:|
| Z500 | RMSE ↓ | **≈680** | ≈800 |
| T2M | RMSE ↓ (K) | **≈2.5** | ≈2.9 |
| MSL | RMSE ↓ (Pa) | **≈625** | ≈740 |

<sub>神珍 tracks or beats the operational physics-based HRES system, with the advantage growing at longer lead times.</sub>

### Medical-image segmentation

Mean Dice (%) on the BiomedParse test splits, 102,855 image–prompt pairs across
nine imaging modalities, versus six modality-native segmentation specialists.

| Modality | # Samples | 神珍 | BiomedParse | MedSAM | SAM | SAM3 | DINO+MedSAM | DINO+SAM |
|:---------|----------:|:-----------:|:-----------:|:------:|:---:|:----:|:-----------:|:--------:|
| **All**    | 102,855 | **91.20** | <u>90.73</u> | 83.55 | 71.29 | 35.40 | 15.37 | 15.10 |
| CT         |  45,306 | **93.36** | <u>92.25</u> | 83.87 | 74.10 | 28.93 |  9.59 | 10.34 |
| MRI        |  30,990 | **85.29** | <u>85.25</u> | 75.90 | 68.34 | 53.64 | 13.28 | 12.39 |
| OCT        |     283 | <u>85.31</u> | **86.63** | 56.26 | 55.99 |  8.69 |  6.68 |  6.98 |
| X-ray      |  13,840 | <u>98.02</u> | **98.28** | 97.75 | 81.35 | 39.96 | 37.22 | 30.63 |
| Dermoscopy |      65 | **98.08** | 97.11 | <u>97.35</u> | 88.23 | 51.47 | 81.28 | 78.29 |
| Endoscopy  |     410 | **97.39** | 96.77 | <u>97.05</u> | 92.88 | 38.82 | 25.01 | 24.54 |
| Fundus     |     800 | <u>91.33</u> | **91.50** | 88.06 | 57.16 | 18.58 |  3.19 |  2.73 |
| Pathology  |     977 | **87.29** | <u>81.57</u> | 43.44 | 42.06 | 26.08 | 25.38 | 24.69 |
| Ultrasound |  10,184 | <u>90.54</u> | **91.03** | 89.76 | 57.47 |  5.23 | 17.12 | 22.91 |

<sub>Best overall Dice (All), and best on CT, MRI, pathology, dermoscopy, and endoscopy; on X-ray, Fundus, and Ultrasound the gap to BiomedParse is ≤ 0.5 Dice, and on the smallest split (OCT) it is 1.3.</sub>

## Usage

Runs via the accompanying code repository (custom multimodal architecture).

```bash
git clone https://github.com/Shanghai-Academy-of-AI-For-Science/MKB && cd MKB
pip install -r requirements.txt          # Python 3.10; transformers==5.0.0
hf download sais-org/MKB --local-dir ./model

export PYTHONPATH=$PWD/code
python code/inference.py --model_path model --greedy --max_new_tokens 64 \
  --rna "GGATGCGATCATGTCTGCACTAACACACCGGATCCCATCAGAACTCCGAAGTTAAGCGTGCTTGGGCGGGAGTAGTACTAGGATGGGCGACCCCTTAGGAAGTACTCGTGTTGCATCCC" \
  --system "You are a non-coding RNA family classifier. Output only the family name, no other text." \
  --prompt $'<rna>\nWhich family does this non-coding RNA sequence belong to?'
```

All weights are contained in `model.safetensors`: the scientific
encoders/decoders (ESM-2, the Suiren molecular graph encoder, the RNA/DNA
ConvFormers, the Swin-ViT weather tower) and the fine-tuned SAM 3 branch used
for medical-image segmentation.

Each task has a specific `--system` prompt that fixes the output format; see
`run_examples.sh` in the repository for per-task examples, weather, and segmentation.

## License

**Composite license.** 神珍's own components — the code, and all weights
except the SAM 3 branch — are **Apache-2.0**, built on Qwen3-VL (Apache-2.0) and
including merged ESM-2 (MIT) and Polaris/Suiren-derived encoders.

The **medical-image segmentation branch embeds SAM 3 weights**, which are
governed by **Meta's SAM License** (`SAM_LICENSE.txt`, shipped alongside these
weights). SAM 3 use is subject to that license, including its acceptable-use
restrictions (no military / weapons / illegal uses; Trade-Control compliance).
See `THIRD_PARTY_LICENSES.md` / `NOTICE` for the full third-party breakdown.

## Citation

```bibtex
@misc{chen2026monkeykingbangunified,
  title         = {Monkey King Bang: A Unified Scientific Multimodal Foundation Model},
  author        = {Hesen Chen and Xinyu Su and Xiaomeng Yang and Yuetan Lin and Zixiong Yang and Junyi An and Fenglei Cao and Yifeng Jiao and Yunqi Zhang and Yuan Cheng and Zhiyu Tan and Hao Li and Libo Wu and Yuan Qi},
  year          = {2026},
  eprint        = {2607.20557},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url           = {https://arxiv.org/abs/2607.20557}
}
```