So the key problem is this: GPT-4 learns that a particular rule is colocated with examples of behaviour violating that rule, and then generalises that colocation pattern to unseen rules.
This is conversation starter. If you don't like the maths then ignore it and focus on the key insight.
This is conversation starter. If you don't like the maths then ignore it and focus on the key insight.