I think there is a lot of potential for transformers in RL. And I do think that AGI is mostly a matter of scale. But I do not agree at all that AGI is just transformers + RL.
I don’t think we have a concrete idea at all, and I don’t agree at all that 10 years ago people thought we had no path forward. 10 years ago is exactly AlexNet, aka the birth of deep learning.
I think it's fair to say that no one knows if AGI is even possible at this point, but RL+transformer is definitely the most promising approach imo.
10 years ago no one was talking about AGI, researchers didn't even call it "AI" due to its association with the previous AI winter. There were certainly advances in ML but I think if you asked anyone whether those approaches would lead to real AI it would be an unequivocal no.
I'd say AGI is very very much possible because it already exists (for example: humans) and unless there exists a soul or some other kind of "literally magic" (and I'd be doubtful even then) there is no reason we couldn't one day build AGI.
It also seems like unless something goes very very wrong with the exponential growth curve in computational capacity humanity has had, we'll be able to straight up simulate a human brain (probably well before) 2100.
