Repartage — contenu de Anurag Daram
Benchmark I ran on TwIL-LM3-Pro, our 3.6B reasoning model from The webAI Intelligence Lab made with Cutter Dalton Paul J. Maykish and David Stout
I tested it against Qwen3.5-9B and Qwen3.8-27B on a CPU-only machine (Intel Xeon, 15 GiB RAM, no GPU).
TwIL-LM3-Pro generated tokens 98x faster than Qwen3.8-27B.
Results:
– TwIL-LM3-Pro (2.24 GB weights): 10.47 tokens per second
– Qwen3.5-9B (5.67 GB weights): 4.36 tokens per second
– Qwen3.8-27B (16.45 GB weights): 0.107 tokens per second
Speed on CPU comes down to how fast you can read the weights. TwIL and Qwen 9B both run at the RAM's memory limit of about 24 GB/s. Qwen 9B is 2.4x slower only because its weights are 2.5x larger.
Qwen 27B's 16.45 GB of weights don't fit in the 15 GiB of RAM. Parts have to be re-read from disk for every token. Size alone would make it 7.3x slower. The disk reads add another 13x. Total: 98x slower than TwIL.
With enough RAM, the gap would shrink to about 7x. But most people don't have unlimited RAM or a top-tier GPU. The model you can actually fit matters more than the one you can't.
Download TwIL-LM3-Pro on Hugging Face : https://lnkd.in/gTHZYUSV
250
3
8
250 reactions
Sources LinkedIn
- Conversation LinkedIn : voir la publication
- Profil : Pierre Jean Duvivier dit Sage (LinkedIn)
- Auteur partagé : Anurag Daram (LinkedIn)
Laisser un commentaire