/models

Models

160 models, from the original Transformer to this year’s releases. Open a model to see every value with its source. Totals are the figures the labs state; an asterisk marks a count modelled from the configuration where no figure is stated.

SearchWeightsShow reported estimates for closed models

Must have

160 of 160 models

									Attention
Kolibri-1	Aleph Alpha	2026-10	78B	3.78B*	50	2,560	20 KiB	1M	40× GQA 48q/4kv, window 513 + 10× GQA 48q/4kv
Claude Fable 5.1closed	Anthropic	2026-09	not disclosed	—	—	—	—	1,000,000	not disclosed
Claude Opus 5.5closed	Anthropic	2026-09	not disclosed	—	—	—	—	1,000,000	not disclosed
DeepSeek-V4.1-Flash	DeepSeek	2026-09	552B	16B	40	5,120	3.13 KiB	1M	compressed 64h, window 128
Ember-1closed	Fireworks AI	2026-09	2.78T*	106B*	93	7,168	27 KiB	—	69× Kimi Delta Attention + 24× MLA 96h, latent 512
GPT-6 Astraclosed	OpenAI	2026-09	not disclosed	—	—	—	—	1,050,000	not disclosed
MiMo-V2.6-Flash-RL	Xiaomi	2026-09	309B	15B	48	4,096	22.5 KiB	1M	9× GQA 64q/4kv + 39× GQA 64q/8kv, window 128
MiMo-V2.6-Pro-RL	Xiaomi	2026-09	1.02T	42B	70	6,144	50 KiB	1M	10× GQA 128q/8kv + 60× GQA 128q/8kv, window 128
Naive-N0.5-Flash	NaiveAI	2026-09	309B	15.5B	48	4,096	22.5 KiB	1M	9× GQA 64q/4kv + 39× GQA 64q/8kv, window 128
Step 5 Previewclosed	StepFun	2026-09	600B	27B	—	—	—	1,000,000	not disclosed
GLM-5.3-Flash	Z.ai	2026-08	320B	18B	45	4,096	13.8 KiB	1M	34× Kimi Delta Attention + 11× MLA 64h, latent 512, top-2048
Hy4 preview	Tencent	2026-08	770B	49B	78	6,144	93 KiB	1M	MLA 64h, latent 512, top-2048
Ling 3.0 Flash	InclusionAI	2026-08	124B*	5.2B*	42	2,560	7.88 KiB	256K	35× Kimi Delta Attention + 7× MLA 32h, latent 512
Muse Glimmer 30B	Meta	2026-08	29.6B	—	52	6,656	13 KiB	128K	39× GQA 32q/2kv, window 2048 + 13× GQA 32q/2kv
Nemotron 3.5 Lightning	NVIDIA	2026-08	30B	3B	52	2,688	6 KiB	1M	23× Mamba-2 + 6× GQA 32q/2kv
Ornith 1.5 35B-A3B	Ornith	2026-08	35B	3B	40	2,048	20 KiB	256K	30× Gated DeltaNet + 10× GQA 16q/2kv
Qwen3.8 27B	Alibaba Cloud (Qwen)	2026-08	27B	—	64	5,120	64 KiB	256K	48× Gated DeltaNet + 16× GQA 24q/4kv
Qwen3.8-Flash-Next	Alibaba Cloud (Qwen)	2026-08	125B	6B	48	2,560	24 KiB	256K	36× Gated DeltaNet + 12× GQA 24q/2kv
Antares 1B	Cisco	2026-07	1.84B*	1.84B*	40	2,048	80 KiB	128K	GQA 16q/4kv
BTL-3	Bad Theory Labs	2026-07	27B	—	64	5,120	64 KiB	256K	48× Gated DeltaNet + 16× GQA 24q/4kv
Claude Opus 5closed	Anthropic	2026-07	not disclosed	—	—	—	—	—	not disclosed
Inkling	Thinking Machines Lab	2026-07	975B	41B	66	6,144	44 KiB	—	55× GQA 64q/16kv, window 512 + 11× GQA 64q/8kv
Laguna S 2.1	Poolside	2026-07	118B	8B	48	3,072	48 KiB	1M	12× GQA 48q/8kv + 36× GQA 72q/8kv, window 512
Motif 3 Beta	Motif Technologies	2026-07	314B	13B	53	4,096	14.6 KiB	256K	40× MLA 80h, latent 512, window 128 + 13× MLA 80h, latent 512
Nanbeige4.2 3B	BOSS Zhipin	2026-07	4B	—	22	3,072	88 KiB	256K	GQA 48q/8kv
Solar Open 2	Upstage	2026-07	250B	15B	48	4,096	48 KiB	1M	12× GQA 64q/8kv + 36× Kimi Delta Attention
Soofi S	SOOFI Consortium	2026-07	31.6B	3.2B	52	2,688	6 KiB	256K	23× Mamba-2 + 6× GQA 32q/2kv
GLM-5.2	Z.ai	2026-06	744B*	41.8B*	78	6,144	93 KiB	1M	MLA 64h, latent 512, top-2048
Kimi K2.7 Code	Moonshot AI	2026-06	1T	32.9B*	61	7,168	68.6 KiB	256K	MLA 64h, latent 512
Kimi K3	Moonshot AI	2026-06	2.8T	106B*	93	7,168	27 KiB	1M	69× Kimi Delta Attention + 24× MLA 96h, latent 512
Laguna XS 2.1	Poolside	2026-06	33B	3B	40	2,048	40 KiB	256K	10× GQA 48q/8kv + 30× GQA 64q/8kv, window 512
MiniMax-M3	MiniMax	2026-06	428B	23B	60	6,144	134 KiB	1M	3× GQA 64q/4kv + 57× GQA 64q/4kv, top-2176
Nemotron 3 Ultra	NVIDIA	2026-06	550B	55B	108	8,192	12 KiB	1M	48× Mamba-2 + 12× GQA 64q/2kv
North Mini Code 1.0	Cohere	2026-06	30B	3B	49	2,048	26 KiB	256K	13× GQA 32q/4kv + 36× GQA 32q/4kv, window 4096
VibeThinker-3B	Weibo	2026-06	3B	—	36	2,048	36 KiB	128K	GQA 16q/2kv
Command A+	Cohere	2026-05	218B	25B	32	4,096	32 KiB	128K	24× GQA 128q/8kv, window 4096 + 8× GQA 128q/8kv
Gemma 4 12B	Google	2026-05	11.9B	—	48	3,840	8 KiB	256K	40× GQA 16q/8kv, window 1024 + 8× MQA 16q/1kv
LFM2.5 8B-A1B	Liquid AI	2026-05	8.3B	1.5B	24	2,048	12 KiB	125K	18× Short convolution + 6× GQA 32q/8kv
Mellum2 12B-A2.5B Thinking	JetBrains	2026-05	12B	2.5B	28	2,304	14 KiB	128K	21× GQA 32q/4kv, window 1024 + 7× GQA 32q/4kv
ZAYA1-8B	Zyphra	2026-05	8.4B	760M	40	2,048	40 KiB	128K	GQA 8q/2kv
DeepSeek-V4-Flash	DeepSeek	2026-04	284B	13.7B*	43	4,096	6.72 KiB	1M	compressed 64h, window 128
DeepSeek-V4-Pro	DeepSeek	2026-04	1.6T	49B	61	7,168	9.62 KiB	1M	compressed 128h, window 128
GLM-5.1	Z.ai	2026-04	744B*	41.8B*	78	6,144	107 KiB	198K	MLA 64h, latent 512, top-2048
Granite 4.1 30B	IBM	2026-04	30B	—	64	4,096	256 KiB	128K	GQA 32q/8kv
Hy3 preview	Tencent	2026-04	295B	21B	80	4,096	320 KiB	256K	GQA 64q/8kv
Kimi K2.6	Moonshot AI	2026-04	1T	32.9B*	61	7,168	68.6 KiB	256K	MLA 64h, latent 512
Laguna XS.2	Poolside	2026-04	33B	3B	40	2,048	40 KiB	256K	10× GQA 48q/8kv + 30× GQA 64q/8kv, window 512
Ling 2.6 1T	InclusionAI	2026-04	1T	63.6B*	80	8,192	11.3 KiB	256K	70× Linear attention + 10× MLA 64h, latent 512
MiMo-V2.5	Xiaomi	2026-04	310B	15B	48	4,096	22.5 KiB	1M	9× GQA 64q/4kv + 39× GQA 64q/8kv, window 128
MiMo-V2.5-Pro	Xiaomi	2026-04	1.02T	42B	70	6,144	50 KiB	1M	10× GQA 128q/8kv + 60× GQA 128q/8kv, window 128
MiniMax-M2.7	MiniMax	2026-04	229B*	11B*	62	3,072	248 KiB	200K	GQA 48q/8kv
Qwen3.6 27B	Alibaba Cloud (Qwen)	2026-04	27B	—	64	5,120	64 KiB	256K	48× Gated DeltaNet + 16× GQA 24q/4kv
Qwen3.6 35B-A3B	Alibaba Cloud (Qwen)	2026-04	35B	3B	40	2,048	20 KiB	256K	30× Gated DeltaNet + 10× GQA 16q/2kv
Gemma 4 26B-A4B	Google	2026-03	26B	4B	30	2,816	10 KiB	256K	25× GQA 16q/8kv, window 1024 + 5× GQA 16q/2kv
Gemma 4 31B	Google	2026-03	30.7B	—	60	5,376	40 KiB	256K	50× GQA 32q/16kv, window 1024 + 10× GQA 32q/4kv
Gemma 4 E2B	Google	2026-03	5.1B	—	35	1,536	6 KiB	128K	28× MQA 8q/1kv, window 512 + 7× MQA 8q/1kv
Gemma 4 E4B	Google	2026-03	8B	—	42	2,560	16 KiB	128K	35× GQA 8q/2kv, window 512 + 7× GQA 8q/2kv
LFM2.5 350M	Liquid AI	2026-03	350M	—	16	1,024	12 KiB	32K	10× Short convolution + 6× GQA 16q/8kv
Nemotron 3 Nano 4B	NVIDIA	2026-03	4B	—	42	3,136	16 KiB	256K	21× Mamba-2 + 4× GQA 40q/8kv
Nemotron 3 Super	NVIDIA	2026-03	120B	12B	88	4,096	8 KiB	1M	40× Mamba-2 + 8× GQA 32q/2kv
Sarvam 105B	Sarvam AI	2026-03	105B	10.3B	32	4,096	36 KiB	128K	MLA 64h, latent 512
Sarvam 30B	Sarvam AI	2026-03	30B	4.52B*	19	4,096	19 KiB	128K	GQA 64q/4kv
GLM-5	Z.ai	2026-02	744B	40B	78	6,144	107 KiB	198K	MLA 64h, latent 512, top-2048
Ling 2.5 1T	InclusionAI	2026-02	1T	63B	80	8,192	11.3 KiB	128K	70× Linear attention + 10× MLA 64h, latent 512
MiniMax-M2.5	MiniMax	2026-02	229B*	11B*	62	3,072	248 KiB	192K	GQA 48q/8kv
Nanbeige4.1 3B	BOSS Zhipin	2026-02	3B	—	32	2,560	64 KiB	256K	GQA 20q/4kv
Qwen3.5 397B-A17B	Alibaba Cloud (Qwen)	2026-02	397B	17B	60	4,096	30 KiB	256K	45× Gated DeltaNet + 15× GQA 32q/2kv
Step 3.5 Flash	StepFun	2026-02	196B	11B	48	4,096	48 KiB	256K	12× GQA 64q/8kv + 36× GQA 96q/8kv, window 512
Tiny Aya	Cohere	2026-02	3.35B	—	36	2,048	18 KiB	8K	27× GQA 16q/4kv, window 4096 + 9× GQA 16q/4kv
Kimi K2.5	Moonshot AI	2026-01	1T	32.9B*	61	7,168	68.6 KiB	256K	MLA 64h, latent 512
LFM2.5 1.2B	Liquid AI	2026-01	1.17B	—	16	2,048	12 KiB	32K	10× Short convolution + 6× GQA 32q/8kv
LongCat-Flash-Lite	Meituan	2026-01	68.5B	3B	42	3,072	31.5 KiB	320K	MLA 32h, latent 512
Mistral Small 4	Mistral AI	2026-01	119B	6.5B	36	4,096	22.5 KiB	256K	MLA 32h, latent 256
Trinity Large	Arcee AI	2026-01	398B	13.4B*	60	3,072	60 KiB	8K	45× GQA 48q/8kv, window 4096 + 15× GQA 48q/8kv
DeepSeek-V3.2	DeepSeek	2025-12	672B*	38.4B*	61	7,168	83.9 KiB	160K	MLA 128h, latent 512, top-2048
GLM-4.7	Z.ai	2025-12	353B*	33.6B*	92	5,120	368 KiB	198K	GQA 96q/8kv
MiMo-V2-Flash	Xiaomi	2025-12	309B	15B	48	4,096	22.5 KiB	256K	9× GQA 64q/4kv + 39× GQA 64q/8kv, window 128
Nemotron 3 Nano 30B-A3B	NVIDIA	2025-12	30B	3.5B	52	2,688	6 KiB	256K	23× Mamba-2 + 6× GQA 32q/2kv
Gemini 3 Proclosed	Google	2025-11	not disclosed	—	—	—	—	1,000,000	not disclosed
INTELLECT-3	Prime Intellect	2025-11	107B*	13.4B*	46	4,096	184 KiB	128K	GQA 96q/8kv
Mistral Large 3	Mistral AI	2025-11	673B	39B	61	7,168	68.6 KiB	288K	MLA 128h, latent 512
Olmo 3 32B	Ai2	2025-11	32B	—	64	5,120	64 KiB	64K	48× GQA 40q/8kv, window 4096 + 16× GQA 40q/8kv
Kimi Linear 48B-A3B	Moonshot AI	2025-10	48B	3B	27	2,304	7.88 KiB	—	20× Kimi Delta Attention + 7× MLA 32h, latent 512
MiniMax-M2	MiniMax	2025-10	230B	10B	62	3,072	248 KiB	192K	GQA 48q/8kv
Ouro 2.6B Thinking	ByteDance	2025-10	2.6B	—	48	2,048	384 KiB	64K	MHA 16q/16kv
Olmo 3 7B	Ai2	2025-09	7B	—	32	4,096	128 KiB	64K	24× MHA 32q/32kv, window 4096 + 8× MHA 32q/32kv
Qwen3-Next 80B-A3B	Alibaba Cloud (Qwen)	2025-09	80B	3B	48	2,048	24 KiB	256K	36× Gated DeltaNet + 12× GQA 16q/2kv
Gemma 3 270M	Google	2025-08	270M	—	18	640	3 KiB	32K	15× MQA 4q/1kv, window 512 + 3× MQA 4q/1kv
GPT-5closed	OpenAI	2025-08	not disclosed	—	—	—	—	400,000	not disclosed
gpt-oss-120b	OpenAI	2025-08	117B	5.1B	36	2,880	36 KiB	128K	18× GQA 64q/8kv, window 128 + 18× GQA 64q/8kv
gpt-oss-20b	OpenAI	2025-08	21B	3.6B	24	2,880	24 KiB	128K	12× GQA 64q/8kv, window 128 + 12× GQA 64q/8kv
Grok 2.5	xAI	2025-08	270B*	115B*	64	8,192	256 KiB	128K	GQA 64q/8kv
GLM-4.5	Z.ai	2025-07	355B	32B	92	5,120	368 KiB	128K	GQA 96q/8kv
GLM-4.5-Air	Z.ai	2025-07	106B	12B	46	4,096	184 KiB	128K	GQA 96q/8kv
Grok 4closed	xAI	2025-07	not disclosed	—	—	—	—	250K	not disclosed
Kimi K2	Moonshot AI	2025-07	1T	32B	61	7,168	68.6 KiB	128K	MLA 64h, latent 512
Qwen3-Coder 30B-A3B	Alibaba Cloud (Qwen)	2025-07	30.5B	3.3B	48	2,048	96 KiB	256K	GQA 32q/4kv
SmolLM3 3B	Hugging Face	2025-06	3B	—	36	2,048	72 KiB	64K	GQA 16q/4kv
Llama 4 Maverick	Meta	2025-04	400B	17B	48	5,120	48 KiB	1,000,000	36× GQA 40q/8kv, chunks of 8192 + 12× GQA 40q/8kv
Qwen3 0.6B	Alibaba Cloud (Qwen)	2025-04	600M	—	28	1,024	112 KiB	32K	GQA 16q/8kv
Qwen3 235B-A22B	Alibaba Cloud (Qwen)	2025-04	235B	22B	94	4,096	188 KiB	32K	GQA 64q/4kv
Qwen3 30B-A3B	Alibaba Cloud (Qwen)	2025-04	30.5B	3.3B	48	2,048	96 KiB	32K	GQA 32q/4kv
Qwen3 32B	Alibaba Cloud (Qwen)	2025-04	32.8B	—	64	5,120	256 KiB	32K	GQA 64q/8kv
Qwen3 4B	Alibaba Cloud (Qwen)	2025-04	4B	—	36	2,560	144 KiB	32K	GQA 32q/8kv
Qwen3 8B	Alibaba Cloud (Qwen)	2025-04	8.2B	—	36	4,096	144 KiB	32K	GQA 32q/8kv
Gemini 2.5 Proclosed	Google	2025-03	not disclosed	—	—	—	—	—	not disclosed
Gemma 3 27B	Google	2025-03	27B	—	62	5,376	80 KiB	128K	52× GQA 32q/16kv, window 1024 + 10× GQA 32q/16kv
Mistral Small 3.1	Mistral AI	2025-03	24B	—	40	5,120	160 KiB	128K	GQA 32q/8kv
DeepSeek-R1	DeepSeek	2025-01	671B	37B	61	7,168	68.6 KiB	160K	MLA 128h, latent 512
MiniMax-Text-01	MiniMax	2025-01	456B	45.9B	80	6,144	40 KiB	10000K	70× Linear attention + 10× GQA 64q/8kv
DeepSeek-V3	DeepSeek	2024-12	671B	37B	61	7,168	68.6 KiB	160K	MLA 128h, latent 512
OLMo 2 7B	Ai2	2024-12	7B	—	32	4,096	512 KiB	4K	MHA 32q/32kv
Phi-4	Microsoft	2024-12	14B	—	40	5,120	200 KiB	16K	GQA 40q/10kv
xLSTM 7B	NXAI	2024-12	7B	—	32	4,096	0 B	—	mLSTM
Llama 3.2 1B	Meta	2024-09	1.23B	—	16	2,048	32 KiB	128K	GQA 32q/8kv
Llama 3.2 3B	Meta	2024-09	3.21B	—	28	3,072	112 KiB	128K	GQA 24q/8kv
o1-previewclosed	OpenAI	2024-09	not disclosed	—	—	—	—	125K	not disclosed
Qwen2.5 72B	Alibaba Cloud (Qwen)	2024-09	72.7B	—	80	8,192	320 KiB	128K	GQA 64q/8kv
GPT-4o miniclosed	OpenAI	2024-07	not disclosed	—	—	—	—	125K	not disclosed
Llama 3.1 405B	Meta	2024-07	405B	—	126	16,384	504 KiB	128K	GQA 128q/8kv
OLMoE-1B-7B	Ai2	2024-07	7B	1B	16	2,048	128 KiB	4K	MHA 16q/16kv
RWKV-6 Finch 14B	RWKV	2024-07	14B	—	61	4,096	0 B	—	RWKV
Claude 3.5 Sonnetclosed	Anthropic	2024-06	not disclosed	—	—	—	—	200,000	not disclosed
Gemma 2 27B	Google	2024-06	27B	—	46	4,608	184 KiB	8K	23× GQA 32q/16kv, window 4096 + 23× GQA 32q/16kv
GPT-4oclosed	OpenAI	2024-05	not disclosed	—	—	—	—	125K	not disclosed
Mamba-2 2.7B	Carnegie Mellon / Princeton	2024-05	2.7B	—	64	2,560	0 B	—	Mamba-2
DeepSeek-V2	DeepSeek	2024-04	236B	21B	60	5,120	67.5 KiB	160K	MLA 128h, latent 512
Llama 3 8B	Meta	2024-04	8B	—	32	4,096	128 KiB	8K	GQA 32q/8kv
Mixtral 8x22B	Mistral AI	2024-04	141B*	39.2B*	56	6,144	224 KiB	64K	GQA 48q/8kv
Phi-3-mini	Microsoft	2024-04	3.8B	—	32	3,072	384 KiB	4K	MHA 32q/32kv
Claude 3 Opusclosed	Anthropic	2024-03	not disclosed	—	—	—	—	200,000	not disclosed
Grok-1	xAI	2024-03	314B	83.8B*	64	6,144	256 KiB	8K	GQA 48q/8kv
Jamba v0.1	AI21 Labs	2024-03	52B	12B	32	4,096	16 KiB	256K	28× Mamba + 4× GQA 32q/8kv
Gemini 1.5 Proclosed	Google	2024-02	not disclosed	—	—	—	—	—	not disclosed
Qwen1.5-MoE-A2.7B	Alibaba Cloud (Qwen)	2024-02	14.3B	2.7B	24	2,048	192 KiB	8K	MHA 16q/16kv
DeepSeekMoE 16B	DeepSeek	2024-01	16B	2.83B*	28	2,048	224 KiB	4K	MHA 16q/16kv
Gemini 1.0 Ultraclosed	Google	2023-12	not disclosed	—	—	—	—	—	not disclosed
Mamba 2.8B	Carnegie Mellon / Princeton	2023-12	2.8B	—	64	2,560	0 B	—	Mamba
Mixtral 8x7B	Mistral AI	2023-12	47B	13B	32	4,096	128 KiB	32K	GQA 32q/8kv
Mistral 7B	Mistral AI	2023-09	7B	—	32	4,096	0 B	32K	GQA 32q/8kv, window 4096
Llama 2 70B	Meta	2023-07	70B	—	80	8,192	320 KiB	4K	GQA 64q/8kv
Falcon-40B	TII	2023-05	40B	—	60	8,192	120 KiB	—	GQA 128q/8kv
Falcon-7B	TII	2023-05	7B	—	32	4,544	8 KiB	—	MQA 71q/1kv
RWKV-4 14B	RWKV	2023-05	14B	—	40	5,120	0 B	1K	RWKV
GPT-3.5 Turbo (ChatGPT)closed	OpenAI	2023-03	not disclosed	—	—	—	—	16,385	not disclosed
GPT-4closed	OpenAI	2023-03	not disclosed	—	—	—	—	8K	not disclosed
LLaMA 65B	Meta	2023-02	65.2B	—	80	8,192	2.5 MiB	2K	MHA 64q/64kv
BLOOM	BigScience	2022-11	176B	—	70	14,336	3.83 MiB	—	MHA 112q/112kv
OPT-66B	Meta	2022-05	66B	—	64	9,216	2.25 MiB	2K	MHA 72q/72kv
GPT-NeoX-20B	EleutherAI	2022-04	20B	—	44	6,144	1.03 MiB	2K	MHA 64q/64kv
PaLM 540Bclosed	Google	2022-04	540B	—	118	18,432	118 KiB	2K	MQA 48q/1kv
Chinchilla 70Bclosed	Google DeepMind	2022-03	70B	—	80	8,192	2.5 MiB	2K	MHA 64q/64kv
GLaM (64B/64E)closed	Google	2021-12	1.2T	96.6B	64	8,192	4 MiB	—	MHA 128q/128kv
GPT-J 6B	EleutherAI	2021-06	6B	—	28	4,096	448 KiB	2K	MHA 16q/16kv
Switch-C	Google	2021-01	1.57T	1.73B*	15	2,080	240 KiB	—	MHA 32q/32kv
GPT-3 175Bclosed	OpenAI	2020-05	175B	—	96	12,288	4.5 MiB	2K	MHA 96q/96kv
T5 11B	Google	2019-10	11B	—	24	1,024	3 MiB	512	MHA 128q/128kv
GPT-2 XL	OpenAI	2019-02	1.5B	—	48	1,600	300 KiB	1K	MHA 25q/25kv
BERT-Large	Google	2018-10	340M	—	24	1,024	96 KiB	512	MHA 16q/16kv
Transformer (base)	Google	2017-06	65M	—	6	512	24 KiB	—	MHA 8q/8kv

* modelled from the configuration where the lab states no figure. KV / token: bytes added per token in BF16 by the layers whose cache grows with context (windowed and recurrent layers excluded).