In python line continuations are a pre-processing step as far as I'm aware (together with inserting INDENT and DEDENT tokens). In the same vein you could just do a simple loop over the token list and remove the appropriate NEWLINE tokens before going to the actual parsing.