Fix README formatting
Browse files
README.md
CHANGED
|
@@ -30,7 +30,7 @@ The model contains exactly one trainable parameter: `w`.
|
|
| 30 |
|
| 31 |
The parameter is trained using gradient descent.
|
| 32 |
|
| 33 |
-
During generation:
|
| 34 |
|
| 35 |
**Input character β Fixed feature function β w β Character probabilities β Generated character**
|
| 36 |
|
|
@@ -40,92 +40,91 @@ The vocabulary and feature function are fixed and are not trainable parameters.
|
|
| 40 |
|
| 41 |
## Download
|
| 42 |
|
| 43 |
-
###
|
| 44 |
|
| 45 |
-
|
| 46 |
|
| 47 |
-
``
|
| 48 |
-
wget "https://huggingface.co/KtiyaKK/SpTn-Model-only-1P/resolve/main/sp_tn_1p.npz?download=true"
|
| 49 |
|
| 50 |
-
Download chat.py
|
| 51 |
|
| 52 |
-
|
| 53 |
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
python chat.py
|
| 57 |
|
| 58 |
-
|
| 59 |
|
| 60 |
-
|
| 61 |
|
| 62 |
-
|
| 63 |
|
| 64 |
-
Installation
|
| 65 |
|
| 66 |
Install NumPy:
|
| 67 |
|
| 68 |
-
pip install numpy
|
| 69 |
|
| 70 |
-
Quick Start
|
| 71 |
|
| 72 |
-
Download
|
| 73 |
|
| 74 |
-
|
| 75 |
-
- chat.py
|
| 76 |
|
| 77 |
-
|
|
|
|
|
|
|
| 78 |
|
| 79 |
Run:
|
| 80 |
|
| 81 |
-
python chat.py
|
| 82 |
|
| 83 |
Example:
|
| 84 |
|
| 85 |
-
You: hello
|
| 86 |
-
|
|
|
|
| 87 |
|
| 88 |
The generated text is experimental and is not intended to be meaningful natural language.
|
| 89 |
|
| 90 |
-
Load the Model
|
| 91 |
|
| 92 |
To verify and load the model:
|
| 93 |
|
| 94 |
-
python load_model.py
|
| 95 |
|
| 96 |
-
Expected output:
|
| 97 |
|
| 98 |
-
SpTn-Model-only-1P
|
| 99 |
-
Trainable parameters: 1
|
| 100 |
-
Parameter: ...
|
| 101 |
-
Vocabulary size: ...
|
| 102 |
-
Model loaded successfully.
|
| 103 |
|
| 104 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 105 |
|
| 106 |
The model can be trained with:
|
| 107 |
|
| 108 |
-
python save_model.py
|
| 109 |
|
| 110 |
This trains the single parameter and saves the resulting model to:
|
| 111 |
|
| 112 |
-
sp_tn_1p.npz
|
| 113 |
|
| 114 |
-
Architecture
|
| 115 |
|
| 116 |
-
SpTn-Model-only-1P contains exactly one trainable scalar: w.
|
| 117 |
|
| 118 |
The model uses a fixed character vocabulary and a fixed feature function.
|
| 119 |
|
| 120 |
-
Only w is updated during training.
|
| 121 |
|
| 122 |
-
The generation process uses the trained value of w to calculate character probabilities.
|
| 123 |
|
| 124 |
-
Important Limitations
|
| 125 |
|
| 126 |
SpTn-Model-only-1P is an experimental research project.
|
| 127 |
|
| 128 |
-
It is not a general-purpose language model.
|
| 129 |
|
| 130 |
The model does not understand natural language or the semantic meaning of the user's input.
|
| 131 |
|
|
@@ -135,21 +134,25 @@ The extremely small parameter count severely limits the model's capacity.
|
|
| 135 |
|
| 136 |
The project is intended to explore parameter efficiency and extremely low-parameter generative systems.
|
| 137 |
|
| 138 |
-
Roadmap
|
| 139 |
|
| 140 |
The project will investigate progressively larger parameter counts:
|
| 141 |
|
| 142 |
-
1P β 100P β 1K β 10K β 100K β 1M β 10M
|
| 143 |
|
| 144 |
The goal is to study how generative behavior changes as the number of trainable parameters increases.
|
| 145 |
|
| 146 |
-
Repository
|
|
|
|
|
|
|
|
|
|
|
|
|
| 147 |
|
| 148 |
-
|
| 149 |
|
| 150 |
-
|
| 151 |
|
| 152 |
-
Disclaimer
|
| 153 |
|
| 154 |
SpTn-Model-only-1P is an experimental project.
|
| 155 |
|
|
|
|
| 30 |
|
| 31 |
The parameter is trained using gradient descent.
|
| 32 |
|
| 33 |
+
During generation, the process is:
|
| 34 |
|
| 35 |
**Input character β Fixed feature function β w β Character probabilities β Generated character**
|
| 36 |
|
|
|
|
| 40 |
|
| 41 |
## Download
|
| 42 |
|
| 43 |
+
### Download the Model
|
| 44 |
|
| 45 |
+
You can download the model directly with:
|
| 46 |
|
| 47 |
+
`wget "https://huggingface.co/KtiyaKK/SpTn-Model-only-1P/resolve/main/sp_tn_1p.npz?download=true"`
|
|
|
|
| 48 |
|
| 49 |
+
### Download chat.py
|
| 50 |
|
| 51 |
+
Download the chat program directly with:
|
| 52 |
|
| 53 |
+
`wget "https://raw.githubusercontent.com/parhamtaheri453-crypto/SpTn-Model-only-1P/main/chat.py"`
|
|
|
|
|
|
|
| 54 |
|
| 55 |
+
Then run:
|
| 56 |
|
| 57 |
+
`python chat.py`
|
| 58 |
|
| 59 |
+
Both files should be placed in the same directory.
|
| 60 |
|
| 61 |
+
## Installation
|
| 62 |
|
| 63 |
Install NumPy:
|
| 64 |
|
| 65 |
+
`pip install numpy`
|
| 66 |
|
| 67 |
+
## Quick Start
|
| 68 |
|
| 69 |
+
Download the model:
|
| 70 |
|
| 71 |
+
`wget "https://huggingface.co/KtiyaKK/SpTn-Model-only-1P/resolve/main/sp_tn_1p.npz?download=true"`
|
|
|
|
| 72 |
|
| 73 |
+
Download `chat.py`:
|
| 74 |
+
|
| 75 |
+
`wget "https://raw.githubusercontent.com/parhamtaheri453-crypto/SpTn-Model-only-1P/main/chat.py"`
|
| 76 |
|
| 77 |
Run:
|
| 78 |
|
| 79 |
+
`python chat.py`
|
| 80 |
|
| 81 |
Example:
|
| 82 |
|
| 83 |
+
`You: hello`
|
| 84 |
+
|
| 85 |
+
`Model: helloenemts aonddsoc xroxoe`
|
| 86 |
|
| 87 |
The generated text is experimental and is not intended to be meaningful natural language.
|
| 88 |
|
| 89 |
+
## Load the Model
|
| 90 |
|
| 91 |
To verify and load the model:
|
| 92 |
|
| 93 |
+
`python load_model.py`
|
| 94 |
|
| 95 |
+
Expected output includes:
|
| 96 |
|
| 97 |
+
`SpTn-Model-only-1P`
|
|
|
|
|
|
|
|
|
|
|
|
|
| 98 |
|
| 99 |
+
`Trainable parameters: 1`
|
| 100 |
+
|
| 101 |
+
`Model loaded successfully.`
|
| 102 |
+
|
| 103 |
+
## Training
|
| 104 |
|
| 105 |
The model can be trained with:
|
| 106 |
|
| 107 |
+
`python save_model.py`
|
| 108 |
|
| 109 |
This trains the single parameter and saves the resulting model to:
|
| 110 |
|
| 111 |
+
`sp_tn_1p.npz`
|
| 112 |
|
| 113 |
+
## Architecture
|
| 114 |
|
| 115 |
+
SpTn-Model-only-1P contains exactly one trainable scalar: `w`.
|
| 116 |
|
| 117 |
The model uses a fixed character vocabulary and a fixed feature function.
|
| 118 |
|
| 119 |
+
Only `w` is updated during training.
|
| 120 |
|
| 121 |
+
The generation process uses the trained value of `w` to calculate character probabilities.
|
| 122 |
|
| 123 |
+
## Important Limitations
|
| 124 |
|
| 125 |
SpTn-Model-only-1P is an experimental research project.
|
| 126 |
|
| 127 |
+
It is **not** a general-purpose language model.
|
| 128 |
|
| 129 |
The model does not understand natural language or the semantic meaning of the user's input.
|
| 130 |
|
|
|
|
| 134 |
|
| 135 |
The project is intended to explore parameter efficiency and extremely low-parameter generative systems.
|
| 136 |
|
| 137 |
+
## Roadmap
|
| 138 |
|
| 139 |
The project will investigate progressively larger parameter counts:
|
| 140 |
|
| 141 |
+
**1P β 100P β 1K β 10K β 100K β 1M β 10M**
|
| 142 |
|
| 143 |
The goal is to study how generative behavior changes as the number of trainable parameters increases.
|
| 144 |
|
| 145 |
+
## Repository
|
| 146 |
+
|
| 147 |
+
GitHub:
|
| 148 |
+
|
| 149 |
+
https://github.com/parhamtaheri453-crypto/SpTn-Model-only-1P
|
| 150 |
|
| 151 |
+
Hugging Face:
|
| 152 |
|
| 153 |
+
https://huggingface.co/KtiyaKK/SpTn-Model-only-1P
|
| 154 |
|
| 155 |
+
## Disclaimer
|
| 156 |
|
| 157 |
SpTn-Model-only-1P is an experimental project.
|
| 158 |
|