It is certainly not absurd to not like being bombarded with a cloud of smoke or vapor, at the very least. The vast majority of people who use e-cigs follow the same etiquette as smokers, but there's always asshats in society who care about nobody but themselves.
Not sure how driving is relevant, but yes, I do on occasion drive (though usually it's the wife behind the wheel). Unfortunately I do not live in a public transit friendly or walkable city.
