This is a common question and the authors answer this in their paper by checking the training set when evaluating it on particular tasks. E.g. for the addition task they search for existing samples of addition in the training data and find that not all are represented.