There's also zero conclusive evidence that ZFC is consistent. And even worse: if you found a proof (within ZFC or a weaker system) that ZFC was consistent, you would immediately know (by Gödel's second theorem) that it is actually inconsistent. The most we could hope for is that we couls prove its consistency in another system (one that hopefully convinces us more of its evident truth?).
ZFC is weird (especially choice). It's not implausible, but there's little a priori reason to assume that it describes some phyiscal reality. It just happens to give a foundation to a lot of really useful mathematics.
You could take a theory such as Peano Arithmetic and argue that that one is self-evident. But unfortunately, again by the second theorem, you can't use PA to prove ZFC consistent. That's, roughly, what Hilbert wanted to do in order to convince his critics, and he failed.
Human thought is paraconsistent - relevance/relevant logic best models how implication works in natural language, and that is paraconsistent; I think the intelligibility of inconsistent fiction such as Graham Priest’s Sylvan’s Box [0] is also evidence of that. If one believes mathematics is ultimately grounded in human thought, and if human thought is ultimately paraconsistent, that suggests paraconsistent logic may be a better foundation for mathematics than classical logic. It also suggests that maybe we should seriously consider taking the inconsistency horn of Godel’s trilemma (incomplete or inconsistent or weak), given the paraconsistent rejection of the principle of explosion means that doing so is non-trivial. Inconsistent theories can be strong, complete and non-trivial.
I would bet that, even if ZFC is not consistent, there's another set of axioms which is, and in which all the stuff we've proven in ZFC still holds. That is, ZFC just happens to be a useful framework for the mathematics we're interested in. Even if ZFC collapses it seems very unlikely that all the stuff we've proven within it will; instead, we'll fix ZFC, like ZFC "fixed" naive set theory.
ZFC is weird (especially choice). It's not implausible, but there's little a priori reason to assume that it describes some phyiscal reality. It just happens to give a foundation to a lot of really useful mathematics.
You could take a theory such as Peano Arithmetic and argue that that one is self-evident. But unfortunately, again by the second theorem, you can't use PA to prove ZFC consistent. That's, roughly, what Hilbert wanted to do in order to convince his critics, and he failed.