The big caveat to this is that an attacker can generally get a browser to include a cross-site resource in their process. For example, `<img src="https://sensitive.com/myprofilepic.png">` will cause the image to be loaded in the attacker's process where they can then potentially steal it. The article "Post-Spectre Web Development" goes into details on how sites can defend against this (and other vectors).