Check the link in my top comment. Most problems are related to the privacy.
A website could always tell if you loaded the resource or had it in cache. If you had it in cache, it means you have previously visited a website that used the same library. Now, if multiple such inferences can be made, it could be used for many things like fingerprinting, for marketing purposes, for identifying a user's identity, etc.
