Xylolabs

CLIP ViT-B/32 text operator components for Tenstorrent Tensix

Base model Formats License Code Tenstorrent Tensix

openai/clip-vit-base-patch32, the text encoder's weights prepared for Tenstorrent Tensix NPUs (Wormhole, Blackhole).

  • Done for Tensix: Twelve transformer blocks plus embedding, position, norm, projection and selector operands, with no value changed: 160 entries in FP32, and the embedding high and low parts and end-token selector (3 entries) in bf16. The visual encoder is not included.
  • Size: about 0.26 GB stored.
  • Checked: All 149 original text tensors match OpenAI's JIT file exactly, and all 163 prepared tensors match the existing preparation formulas; every value is finite.
  • Code: load_components.py, which checks hashes, names, shapes, dtypes and finite values; tensix-bfp-gptq v0.1.0 (quantization and format checks), tensix-kernels v0.1.0 and tensix-runner v0.1.0 (kernels and runner for Tensix).

Prepared text-encoder operator tensors from OpenAI CLIP ViT-B/32, used by the OmniVLA-Edge integration.

Scope

Inputs contain up to 77 CLIP tokens, padded to 96 positions, with exactly one highest-vocabulary end token. Visual encoding and the image-text logit scale are excluded. A tokenizer, architecture implementation and integration adapter are required.

Formats

Tensor group Stored dtype
160 block/position/norm/projection/mask entries FP32
Embedding high/low parts and end-token selector, 3 entries BF16

Masks are allowed-attention indicators, rather than additive 0/negative-infinity masks.

Quickstart

Use PyTorch, safetensors and huggingface_hub. Stream one prepared block:

from pathlib import Path
import importlib.util
from huggingface_hub import snapshot_download

folder = Path(snapshot_download("Xylolabs/CLIP-ViT-B-32-Text-Tenstorrent-Tensix-Components"))
spec = importlib.util.spec_from_file_location("clip_components", folder / "load_components.py")
components = importlib.util.module_from_spec(spec)
spec.loader.exec_module(components)
for name, tensor in components.iter_component(folder, "block_00"):
    print(name, tuple(tensor.shape), tensor.dtype)

The existing port uses BF16 dense products with FP32 residuals, normalization, attention scores/softmax and pooling. The embedding path adds high and low BF16 lookups in FP32.

License

MIT for the original OpenAI model (LICENSE), with the OmniVLA integration/reference MIT notice retained in OMNIVLA_MIT_LICENSE. NOTICE records the source identities, prepared representation and excluded components. The Xylolabs loader alone uses Apache 2.0 (CODE_LICENSE). Not affiliated with or endorsed by Tenstorrent.

Citation

Cite the original OpenAI CLIP project and paper and the OmniVLA integration.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Xylolabs/CLIP-ViT-B-32-Text-Tenstorrent-Tensix-Components

Finetuned
(163)
this model

Collection including Xylolabs/CLIP-ViT-B-32-Text-Tenstorrent-Tensix-Components