Yes, perhaps the computer needs information on which pixels belong to which buildings in order to apply perspective-per-object, rather than perspective-to-the-entire-image?