The problem with having those APIs in the browser is that it increases the attacker surface area, which makes the browser less secure for everyone, including those who do not use PWAs.
The only saving grace is that you have to accept the permission box (I hope so at least...), which, for the average user, may not be much protection.
Simply existing in the world increases your attack surface; everything is a trade off between usability and security. Given the pressures browsers are under, they have incentives built into their business model to provide very good security which is a departure from most other software where security is just a nuisance at best and totally ignored at worst.
