I think that it’s pretty hard to stop with the current state of PWA installation.

You could try the manifest data, (the data for the PWA app) tied more to the html and dns. Making it harder to impersonate other sites.

You could also go a more extreme route and have something like PWA app signing like other kinds of apps.