Code

Topk#37

Code

Open

huangzl19 wants to merge 2 commits intofeifeibear:mainfeifeibear/LLMSpeculativeSampling:mainfrom huangzl19:topkhuangzl19/LLMSpeculativeSampling:topkCopy head branch name to clipboard

+8,803-101

Commits

Commits on Oct 10, 2024

• 完成了topk的测试和改动，发现topk的性能更高。k还没有做成参数，如果想要改动需要再kvcache_model.py 86行中改变topk的参数。不管怎么说，实验的效果说明topk是有效的

  huangzl19

  committed
  7e9b9d1

  Copy full SHA for 7e9b9d1

Commits on Oct 26, 2024

• add the argparse of topk (--topk_approx)

  huangzl19

  committed
  c0248db

  Copy full SHA for c0248db