Know-how is not the same as particular inventions. Besides which, confidentiality agreements would be much more aggressively enforced than they are now, to the point that they would make it unattractive to hire someone who's worked for a direct competitor.

Know-how may not be the same as inventions, but it is also not a super relevant distinction. This is because know-how begets invention, as it leads to knowing what tools are available for solving problems, and also to knowing what problems needs solving. Removing patents would change some economics around, but in the end it will not change the primary driver towards improvement, which is that people want better and/or cheaper stuff. Companies don't need any other incentives to improve the stuff they do - if they don't, the customers will desert them.

Confidentiality agreements for employees and former employees that doesn't actually give the employees something in exchange for that confidentiality, is not legal in my jurisdiction.