Serendipity/CTI-Inference-Opt
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

experiment: triton_block_m=128(块数减半=launch减半);消同步赚-1.64s证评测对launch敏感→块大试

Browse Source 
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
OwnerSunshine530
2026-06-20 01:11:59 +08:00
parent 69d49cd282
commit 292a021679
1 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
代码/code/infer.py
+2 -1
View File
@@ -145,7 +145,8 @@ CONFIG = {
# sdpa 是评测端验证最快(89.96s/58.86)。flex/compile/小batch/varlen 在评测端都更差。 # sdpa 是评测端验证最快(89.96s/58.86)。flex/compile/小batch/varlen 在评测端都更差。
# attn: "chunked"(按用户分块SDPA,降O(S²),本地14.25->7.92s) / "sdpa"(稠密mask) / 其它对照 # attn: "chunked"(按用户分块SDPA,降O(S²),本地14.25->7.92s) / "sdpa"(稠密mask) / 其它对照
"attn": "triton", # Triton varlen flash(单kernel,消逐块调用/mask构造开销);无triton回退chunked "attn": "triton", # Triton varlen flash(单kernel,消逐块调用/mask构造开销);无triton回退chunked
"triton_block_m": 64, # Triton query 块大小(可调 32/64/128;块大=调用少) # 本地 64 最快(4.86);但评测对launch敏感(消同步刚赚-1.64s),块大=launch少→评测试128
"triton_block_m": 128, # Triton query 块大小;128 块数减半、launch减半(评测可能更快)
"chunk_users": 4, # chunked 回退时用;评测扫描 3/4/8 中 4 最优(47.84s/67.998) "chunk_users": 4, # chunked 回退时用;评测扫描 3/4/8 中 4 最优(47.84s/67.998)
# 稠密MoE去掉了 model(batch) 内唯一的同步点(MoE循环的.nonzero())。若评测计时不 # 稠密MoE去掉了 model(batch) 内唯一的同步点(MoE循环的.nonzero())。若评测计时不
# synchronize，去掉同步点可能让被计时的 model(batch) 大幅缩短。本地force-sync看不出， # synchronize，去掉同步点可能让被计时的 model(batch) 大幅缩短。本地force-sync看不出，