Instructions to use IPEC-COMMUNITY/spatialvla-4b-224-sft-fractal with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use IPEC-COMMUNITY/spatialvla-4b-224-sft-fractal with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="IPEC-COMMUNITY/spatialvla-4b-224-sft-fractal", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("IPEC-COMMUNITY/spatialvla-4b-224-sft-fractal", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use IPEC-COMMUNITY/spatialvla-4b-224-sft-fractal with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "IPEC-COMMUNITY/spatialvla-4b-224-sft-fractal"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IPEC-COMMUNITY/spatialvla-4b-224-sft-fractal",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/IPEC-COMMUNITY/spatialvla-4b-224-sft-fractal

SGLang

How to use IPEC-COMMUNITY/spatialvla-4b-224-sft-fractal with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "IPEC-COMMUNITY/spatialvla-4b-224-sft-fractal" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IPEC-COMMUNITY/spatialvla-4b-224-sft-fractal",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "IPEC-COMMUNITY/spatialvla-4b-224-sft-fractal" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IPEC-COMMUNITY/spatialvla-4b-224-sft-fractal",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use IPEC-COMMUNITY/spatialvla-4b-224-sft-fractal with Docker Model Runner:
```
docker model run hf.co/IPEC-COMMUNITY/spatialvla-4b-224-sft-fractal
```

Question about used dataset for training

by phandat128 - opened May 19, 2025

Discussion

phandat128

May 19, 2025

Hi, thanks for sharing this project and checkpoints.
I am trying to reproduce the results of your work and also want to adapt to another variants further. Can i ask you if the training dataset for this checkpoint that you mentioned in the paper is here: https://huggingface.co/datasets/IPEC-COMMUNITY/fractal20220817_data_lerobot or something else.
Thank you!

delinqu

IPEC at Shanghai AI Laboratory org May 27, 2025

Hi! We use the original fractal dataset in RLDs format: https://robotics-transformer-x.github.io/.
https://huggingface.co/datasets/IPEC-COMMUNITY/fractal20220817_data_lerobot is the converted Lerobot version, which is more flexible and efficient in training.
Hence, the Lerobot dataset is recommended if you're reproducing SpatialVLA results or developing your model. I'll also release all the code in the Lerobot training in the next few weeks.

phandat128

May 28, 2025

Thank you so much for your answer!

phandat128 changed discussion status to closed May 28, 2025

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment