Serendipity/CTI-Inference-Opt
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

perf: chunk_users 默认 3(本地6.2->4.13s,减块对角浪费;AUC不变) — A第一步冲70

Browse Source 
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
OwnerSunshine530
2026-06-16 22:57:29 +08:00
parent df65b3659d
commit 4c7cbcd9b1
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
代码/code/infer.py
+1 -1
View File
@@ -44,7 +44,7 @@ CONFIG = {
# sdpa 是评测端验证最快(89.96s/58.86)。flex/compile/小batch/varlen 在评测端都更差。 # sdpa 是评测端验证最快(89.96s/58.86)。flex/compile/小batch/varlen 在评测端都更差。
# attn: "chunked"(按用户分块SDPA,降O(S²),本地14.25->7.92s) / "sdpa"(稠密mask) / 其它对照 # attn: "chunked"(按用户分块SDPA,降O(S²),本地14.25->7.92s) / "sdpa"(稠密mask) / 其它对照
"attn": "chunked", "attn": "chunked",
"chunk_users": 4, # chunked 每块用户数（本地 4 最快 6.18s；再小收益递减） "chunk_users": 3, # chunked 每块用户数（扫描 1/2/3/4：chunk=3 最优 4.13s，块对角浪费vs块数开销平衡）
# 稠密MoE去掉了 model(batch) 内唯一的同步点(MoE循环的.nonzero())。若评测计时不 # 稠密MoE去掉了 model(batch) 内唯一的同步点(MoE循环的.nonzero())。若评测计时不
# synchronize，去掉同步点可能让被计时的 model(batch) 大幅缩短。本地force-sync看不出， # synchronize，去掉同步点可能让被计时的 model(batch) 大幅缩短。本地force-sync看不出，
# 须靠提交验证。AUC中性、MoE仅占2%算力故风险极低。 # 须靠提交验证。AUC中性、MoE仅占2%算力故风险极低。