My intuition that got confirmed is that GPT fails at anything visual. Letters, shapes. It's trying but failing every time.

It succeeds only if the thing was drilled diwn hard in learning like american flag or implementing tictactoe (but not predicting best move on the fly).