samankeon.comPapers and technical reports. Full list on Google Scholar.
A layer-by-layer mathematical description of the Gemma-3 decoder, written alongside a from-scratch reference implementation in MintEngine and used to check numerical parity of every layer against production inference engines. Covers embeddings and scaling, the pre-normalized transformer layer with explicit residual handling, multi-query attention, RoPE, the gated MLP, and Gemma's RMSNorm variant.
Evaluating efficiency techniques such as quantization, pruning, and distillation requires a rigorous, standardized way to establish model quality parity, and quantization in particular has effects on quality that scaling laws do not capture. This work proposes an Elo-based scoring framework that ranks optimized model variants through automated competitive matchups on real-world queries from LMSYS chat, judged by LLM preference, so quality can be compared across tasks without task-specific ground truth. Backed by over 2,000 H100 GPU hours, the framework offers a scalable, reproducible protocol for assessing the trade-offs of efficiency techniques.