Instructions to use h2oai/h2o-lightning-4b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use h2oai/h2o-lightning-4b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="h2oai/h2o-lightning-4b") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("h2oai/h2o-lightning-4b") model = AutoModelForMultimodalLM.from_pretrained("h2oai/h2o-lightning-4b", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use h2oai/h2o-lightning-4b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "h2oai/h2o-lightning-4b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "h2oai/h2o-lightning-4b", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/h2oai/h2o-lightning-4b
- SGLang
How to use h2oai/h2o-lightning-4b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "h2oai/h2o-lightning-4b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "h2oai/h2o-lightning-4b", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "h2oai/h2o-lightning-4b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "h2oai/h2o-lightning-4b", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use h2oai/h2o-lightning-4b with Docker Model Runner:
docker model run hf.co/h2oai/h2o-lightning-4b
SO-Arena evaluation results for H2O Lightning
We recently released SO-Arena, a benchmark for System One decision models, and H2O Lightning is among the 21 models we evaluated. We'd like to share the results here, as they may be useful to you and to others using the model.
SO-Arena asks whether models are right, and reliably right: it spans 11 major domains with difficulty tiers (the leaderboard is scored on the hardest tier of each), and tests whether decisions stay consistent under reordered options or a missing correct answer.
SO-Arena at a glance: verified labels, difficulty tiers, decision reliability, and interactive evaluation.
H2O Lightning pairs speed with strong accuracy: it sits on our performance–latency Pareto frontier and outscores OpenAI Decisions at a fraction of its latency.
| Metric | Result | Standing |
|---|---|---|
| Mean latency | 26.5 ms | on the performance–latency Pareto frontier |
| Leaderboard score | 54.9 | above OpenAI Decisions (53.6), more than 10× faster |
| Among 4B models | — | 3rd of 8 4B configurations |

Performance vs. latency on SO-Arena. H2O Lightning is on the Pareto frontier.
Evaluated release: h2oai/h2o-lightning-4b @ 08e1da5. Rankings are among the 19 System One configurations in the paper (reliability: the 18 without StartLux Auto). Open-weight latency is the mean per task on one H100; closed APIs are timed end to end.
Full per-domain and per-tier results are on our leaderboard, with details in the paper. We'd be delighted if SO-Arena could help showcase H2O Lightning's strengths. Everything needed to run it is in our GitHub repo, and we're glad to help.
