Saman Keon

Writing · 1 posts

rope

  1. RoPE: The Phase Geometry Behind Long-Context TransformersHow rotating query and key vectors turns absolute position into relative attention