Why have any capitalism? Because the alternatives like communism are disastrous. If we want a complex tax system tax prep companies serve a useful function.
If their product is better than the free government app, and users find the government app not good enough, they will win anyway. If it's not better, then they will not. I don't see a problem here.
The government intervening to keep an industry afloat, particularly an industry that only exists because of complex government regulations, is about as far from capitalism as you can get.
Being forced into paying the federal government a portion of wages because a person happened to be born within the federal government land mass is taking the fruits of a person's work. Slaves are born into the slave owner's plantation. Don't pay and be punished by the slave master.
Going to school, at no charge to student with free lunch, is probably preferrable to working in the mines or factory at age ten. If you are concerned about government-run education, you can home school your children.
You're not forced to work. Go to any big city, you'll see plenty of people who don't. You can just leech off everyone else's taxes, plenty of people do.