Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

You can use 0.6B for speculative decoding on the larger models. It'll speed up 32B, but slows down 30B-A3B dramatically.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: