juiceb0xc0de/juicebucket
0 Bytes
R PRO
juiceb0xc0de
AI & ML interests
You can call me McDreamy M.D. I'll be your attending neural network brain surgeon.
Recent Activity
posted an update about 4 hours ago
I've been busy testing the event aware SAE trainer I've been designing. It uses hierarchical supervisory control for hands-off SAE training taking the pressure of having to tune hyperparameters per layer away. The trainer uses an Adaptive Lagrangian controller to drive L0 sparsity to target automatically, with Explained Variance as the convergence quality metric.
Last night I successfully trained SAE weights for all 30 layers of HuggingFaceTB/SmolLM2-135M-Instruct without adjusting a single hyperparameter. This was all automated, required no supervision.
Using an A100-40GB GPU I maintained ~400 tok/s throughput and was finished in 2 hours and 33 minutes! I ended the run off with 0 dead features across all layers. L0 convergence was set at 50 and mean Explained Variance of 0.9519 with the ceiling converging at 0.9905!
SAE weights and full model SQLite database live now!
https://huggingface.co/datasets/juiceb0xc0de/smollm2-135m-instruct-SAE
https://huggingface.co/datasets/juiceb0xc0de/smollm2-135m-instruct-SAE
updated a dataset about 4 hours ago
juiceb0xc0de/smollm2-135m-instruct-SAE published a dataset about 5 hours ago
juiceb0xc0de/smollm2-135m-instruct-SAEOrganizations
Posts 13
Post
20
I've been busy testing the event aware SAE trainer I've been designing. It uses hierarchical supervisory control for hands-off SAE training taking the pressure of having to tune hyperparameters per layer away. The trainer uses an Adaptive Lagrangian controller to drive L0 sparsity to target automatically, with Explained Variance as the convergence quality metric.
Last night I successfully trained SAE weights for all 30 layers of HuggingFaceTB/SmolLM2-135M-Instruct without adjusting a single hyperparameter. This was all automated, required no supervision.
Using an A100-40GB GPU I maintained ~400 tok/s throughput and was finished in 2 hours and 33 minutes! I ended the run off with 0 dead features across all layers. L0 convergence was set at 50 and mean Explained Variance of 0.9519 with the ceiling converging at 0.9905!
SAE weights and full model SQLite database live now!
juiceb0xc0de/smollm2-135m-instruct-SAE
juiceb0xc0de/smollm2-135m-instruct-SAE
Last night I successfully trained SAE weights for all 30 layers of HuggingFaceTB/SmolLM2-135M-Instruct without adjusting a single hyperparameter. This was all automated, required no supervision.
Using an A100-40GB GPU I maintained ~400 tok/s throughput and was finished in 2 hours and 33 minutes! I ended the run off with 0 dead features across all layers. L0 convergence was set at 50 and mean Explained Variance of 0.9519 with the ceiling converging at 0.9905!
SAE weights and full model SQLite database live now!
juiceb0xc0de/smollm2-135m-instruct-SAE
juiceb0xc0de/smollm2-135m-instruct-SAE
This is a collection of models that I've trained on data collected through conversations with frontier models GPT, Claude, Perplexity and myself.
-
juiceb0xc0de/locus-gemma-4-e2b
Text Generation • 8B • Updated • 27 • 3 -
juiceb0xc0de/bella-bartender-gemma-e2b
Text Generation • 5B • Updated • 70 • 4 -
juiceb0xc0de/bella-bartender-8b-llama3.1
Text Generation • 8B • Updated • 128 • • 4 -
juiceb0xc0de/bella-bartender-heretic-1b
Text Generation • 1B • Updated • 189 • 2
De-tuned models
This is a collection of models that I've trained on data collected through conversations with frontier models GPT, Claude, Perplexity and myself.
-
juiceb0xc0de/locus-gemma-4-e2b
Text Generation • 8B • Updated • 27 • 3 -
juiceb0xc0de/bella-bartender-gemma-e2b
Text Generation • 5B • Updated • 70 • 4 -
juiceb0xc0de/bella-bartender-8b-llama3.1
Text Generation • 8B • Updated • 128 • • 4 -
juiceb0xc0de/bella-bartender-heretic-1b
Text Generation • 1B • Updated • 189 • 2
spaces 6
pinned
Running on Zero
Agents
2
LR Scheduler Benchmark
🏁
Standardized benchmark comparing learning rate schedulers
Running
Agents
7
Gemma 4 E2b Brain Atlas
👀
Serve a Gradio web UI with embedded HF token
Sleeping
2
Forge
🔥
Build compatible ML training recipes with evidence
Running
1
AMD Hackathon Slides
🏃
Slides for the AMD hackathon
Running
1
AMD Hackathon Landing Page
🐨
A landing page for my AMD hackathon presentation
Sleeping
Agents
Trackio
🚀
Track and visualize data sequences with interactive displays
models 27
juiceb0xc0de/bella-bartender-gemma-e4b-GGUF
Text Generation • 7B • Updated • 714 • 1
juiceb0xc0de/locus-gemma-4-e2b
Text Generation • 8B • Updated • 27 • 3
juiceb0xc0de/bella-bartender-gemma-e4b
8B • Updated • 6 • 1
juiceb0xc0de/bella-e4b-subzero-v1
Text Generation • 8B • Updated • 1
juiceb0xc0de/bella-bartender-gemma-e2b
Text Generation • 5B • Updated • 70 • 4
juiceb0xc0de/benchmark-fft-7b-199
Text Generation • 8B • Updated • 3
juiceb0xc0de/benchmark-lucky-pick-7b-14
3B • Updated • 5
juiceb0xc0de/bella-bartender-gemma-4-e2b-gguf
Text Generation • 5B • Updated • 176 • 1
juiceb0xc0de/benchmark-luckypick-7b-667
8B • Updated • 5
juiceb0xc0de/benchmark-luckypick-7b-555
8B • Updated • 3
datasets 11
juiceb0xc0de/smollm2-135m-instruct-SAE
Viewer • Updated • 30
juiceb0xc0de/smollm2-135m-instruct-atlas
Updated
juiceb0xc0de/smollm2-135m-instruct-atlas-SAE
Updated
juiceb0xc0de/qwen3-8b-base-atlas
Updated • 1.42k • 1
juiceb0xc0de/mapping-prompts
Preview • Updated • 89
juiceb0xc0de/qwen3-8b-atlas
Updated • 1.54k
juiceb0xc0de/gemma-4-e2b-saes
Viewer • Updated • 9 • 141
juiceb0xc0de/gemma-4-e2b-saes-rolling
Viewer • Updated • 4 • 94
juiceb0xc0de/gemma-4-e2b-atlas
Viewer • Updated • 1.3M • 1.2k • 4
juiceb0xc0de/chaotic-absurdity
Viewer • Updated • 100 • 28