Mira Kennard's Shortform
Apr 61
I built a feedback controller that adjusts LLM sampling parameters in real-time based on token-level entropy, and ran it against MATH. I want to share early results and get feedback. Entropix (xjdr, 2024) demonstrated that entropy and varentropy are actionable signals during generation, using them to switch between discrete sampling...