i thought mixture of experts didn't update itself with new sets of weights and w... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		mnky9800n on Jan 15, 2025 \| parent \| context \| favorite \| on: Transformer^2: Self-Adaptive LLMs i thought mixture of experts didn't update itself with new sets of weights and was just a collection of already trained networks/weights? I could be wrong.

QuadmasterXLII on Jan 15, 2025 [–]

Well, that depends in whether you keep training it

mnky9800n on Jan 15, 2025 | [–]

perhaps they should always be training and never static. haha. i allegedly grow wiser in my age, why not neural networks?

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact