Micrometer-scale dust particles would in fact hit you all the time. And they’d absolutely mess up your ship over time without a lot of shielding.