This argument works better for state space models. A transformer would still ste... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

rar00 on June 25, 2025 | parent | context | favorite | on: The bitter lesson is coming for tokenization

This argument works better for state space models. A transformer would still steps context one token at a time, not maintain an internal 1e18 state.

mgraczyk on June 25, 2025 [–]

That doesn't matter, are you familiar with any theoretical results in which the computation is somehow limited in ways that practically matter when the context length is very long? I am not

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact