# Token per second calculations **URL:** \ **Category:** Intermediate\ **Created:** [November 10, 2023, 12:31pm UTC](https://discuss.huggingface.co/t/token-per-second-calculations/61801 "2023-11-10T12:31:54Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![almersawi](https://avatars.discourse-cdn.com/v4/letter/a/ee7513/32.png) [@almersawi](https://discuss.huggingface.co/u/almersawi)\ **Post date:** [November 10, 2023, 12:31pm UTC](https://discuss.huggingface.co/t/token-per-second-calculations/61801/1 "2023-11-10T12:31:54Z")
Hi! I’m trying to calculate the number of token per second that I expect to get from “llama 7b” model deployed on A10G (31.52 TFLOPS for FP16). I know that the number of tokens = (TFLOPS / (2 \* number of model parameters)) When I do the calculations I found that no\_of\_tokens = (31.52 \* 10e12) / (2 \* 7 \* 10e9) = 2251.4285714285716 tokens / second but what I get from the model is approximately 32 token / second. Am I missing something? ---
**Author:** ![Diogo](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/diogo/32/34237_2.png) [@Diogo](https://discuss.huggingface.co/u/Diogo)\ **Post date:** [November 8, 2024, 9:27pm UTC](https://discuss.huggingface.co/t/token-per-second-calculations/61801/2 "2024-11-08T21:27:44Z")
Dude I am at the same spot… Did you found out what were you missing? ---
**Author:** ![wenxi](https://avatars.discourse-cdn.com/v4/letter/w/e47774/32.png) [@wenxi](https://discuss.huggingface.co/u/wenxi)\ **Post date:** [April 20, 2025, 5:57am UTC](https://discuss.huggingface.co/t/token-per-second-calculations/61801/3 "2025-04-20T05:57:33Z")
It could be due to Memory Bandwidth Limitations. A simple operation for the 7B model might require loading all 14GB of parameters. At 600 GB/s memory bandwidth, just loading all parameters once would take: 14GB ÷ 600GB/s = 0.023 seconds. This translates to a theoretical maximum of ~43 operations per second involving all parameters