Essays
Notes on technology and life.
- Speculation and a slower clock
Speculative decoding and a locked GPU clock both cut decode energy. At small batches they combine to save 62%. At large batches they get in each other's way.
- What a power cap does to inference
Cut an H100 from 700 W to 415 W and decode keeps 96% of its speed. Prefill keeps 74%. Here's why.
- Breaking the silence
Starting a blog. Not sure why.