Serendipity/CTI-Inference-Opt
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: 默认开真稀疏MoE cap=2.0(本地4.77->4.05s -15%,AUC微降,PCOC1.105区间内)

Browse Source 
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
OwnerSunshine530
2026-06-17 21:22:31 +08:00
parent b397c142fa
commit 2cf7f185fc
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
代码/code/infer.py
+2 -2
View File
@@ -144,8 +144,8 @@ CONFIG = {
# 须靠提交验证。AUC中性、MoE仅占2%算力故风险极低。 # 须靠提交验证。AUC中性、MoE仅占2%算力故风险极低。
"vectorize_moe": True, # True=稠密向量化MoE(无同步点)；False=原逐expert循环(.nonzero同步) "vectorize_moe": True, # True=稠密向量化MoE(无同步点)；False=原逐expert循环(.nonzero同步)
"moe_baddbmm": True, # MoE FFN 用 baddbmm(cutlass GEMM+bias epilogue融合),省 bias add kernel "moe_baddbmm": True, # MoE FFN 用 baddbmm(cutlass GEMM+bias epilogue融合),省 bias add kernel
"moe_sparse": False, # True=真稀疏MoE(只算top-k,capacity分组),减GEMM~3x;风险:开销/容量丢弃AUC "moe_sparse": True, # 真稀疏MoE(只算top-k,capacity分组),本地4.77->4.05s(-15%),AUC微降无碍
"moe_capacity": 1.25, # 每expert容量 = ceil(Nk/E*factor);越大越不丢token但计算越多 "moe_capacity": 2.0, # 每expert容量 = ceil(Nk/E*factor);cap=2.0 PCOC1.105在区间(1.25会炸到1.418)
"skip_moe_loss": True, # 推理跳过 moe_loss(load-balance,推理无用),省 importance/std/mean kernel "skip_moe_loss": True, # 推理跳过 moe_loss(load-balance,推理无用),省 importance/std/mean kernel
# PCOC 校准:本地拟合-0.1067(本地PCOC1.109),但评测PCOC稳定1.059,按斜率换算评测最优≈-0.059。 # PCOC 校准:本地拟合-0.1067(本地PCOC1.109),但评测PCOC稳定1.059,按斜率换算评测最优≈-0.059。
"logit_bias": -0.06, # logit 加常数偏移使评测 PCOC→~1.0(单调,AUC不变,免费+~0.33分) "logit_bias": -0.06, # logit 加常数偏移使评测 PCOC→~1.0(单调,AUC不变,免费+~0.33分)