Skip to content

Commit bc16288

Browse files
committed
add back
1 parent 8324f55 commit bc16288

File tree

1 file changed

+19
-1
lines changed

1 file changed

+19
-1
lines changed

src/MaxText/utils/ckpt_scripts/convert_qwen3_moe.py

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,25 @@
4646
"head_dim": 128,
4747
"num_experts": 128,
4848
"moe_intermediate_size": 1536,
49-
}
49+
},
50+
"qwen3-30b-a3b": {
51+
"num_hidden_layers": 48,
52+
"num_attention_heads": 32,
53+
"num_key_value_heads": 4,
54+
"hidden_size": 2048,
55+
"head_dim": 128,
56+
"num_experts": 128,
57+
"moe_intermediate_size": 768,
58+
},
59+
"qwen3-480b-a35b": {
60+
"num_hidden_layers": 62,
61+
"num_attention_heads": 96,
62+
"num_key_value_heads": 8,
63+
"hidden_size": 6144,
64+
"head_dim": 128,
65+
"num_experts": 160,
66+
"moe_intermediate_size": 2560,
67+
},
5068
}
5169

5270

0 commit comments

Comments
 (0)