Spaces:
Running on Zero
Running on Zero
Commit ·
45c6078
1
Parent(s): 1a45ca6
Clarify PyPI usage
Browse files
README.md
CHANGED
|
@@ -25,7 +25,7 @@ count is, and how many characters each tokenizer buys per token.
|
|
| 25 |
| | |
|
| 26 |
|---|---|
|
| 27 |
| **Source** | [github.com/sanderland/ctok](https://github.com/sanderland/ctok) |
|
| 28 |
-
| **Python package** | [`
|
| 29 |
| **The research** | [On the biology of Claude's tokenizer](https://tokencontributions.substack.com/p/on-the-biology-of-claudes-tokenizer) |
|
| 30 |
|
| 31 |
## What is compared
|
|
|
|
| 25 |
| | |
|
| 26 |
|---|---|
|
| 27 |
| **Source** | [github.com/sanderland/ctok](https://github.com/sanderland/ctok) |
|
| 28 |
+
| **Python package** | [`ctok` on PyPI](https://pypi.org/project/ctok/) — count any string yourself with `uvx ctok "your text"`, offline, no API call |
|
| 29 |
| **The research** | [On the biology of Claude's tokenizer](https://tokencontributions.substack.com/p/on-the-biology-of-claudes-tokenizer) |
|
| 30 |
|
| 31 |
## What is compared
|
app.py
CHANGED
|
@@ -488,7 +488,7 @@ from **ctok**, a 99.9%+ accurate offline reconstruction — text only, no per-me
|
|
| 488 |
LINKS = f"""
|
| 489 |
<div class='links'>
|
| 490 |
<a href='{REPO_URL}' target='_blank' rel='noopener'>◆ Source</a>
|
| 491 |
-
<a href='{PYPI_URL}' target='_blank' rel='noopener'>▼ <code>
|
| 492 |
<a href='{POST_URL}' target='_blank' rel='noopener'>✎ The research behind it</a>
|
| 493 |
</div>
|
| 494 |
"""
|
|
|
|
| 488 |
LINKS = f"""
|
| 489 |
<div class='links'>
|
| 490 |
<a href='{REPO_URL}' target='_blank' rel='noopener'>◆ Source</a>
|
| 491 |
+
<a href='{PYPI_URL}' target='_blank' rel='noopener'>▼ <code>ctok</code> on PyPI</a>
|
| 492 |
<a href='{POST_URL}' target='_blank' rel='noopener'>✎ The research behind it</a>
|
| 493 |
</div>
|
| 494 |
"""
|