It's worth to point that "3% probability we're just seeing a pattern by accident" is only right when you understand it as "in the world here our hypothesis is wrong the same experiment would give such pattern in 3% cases", not as "given such result probability that we are wrong is 3%".
It seems like an education fail to me. Most people don't know the very basics of stats and we live in a world that's highly probabilistic.
It seems like something that should be taught alongside math from elementary school, not something you can maybe get an elective in in high school or college.
I never learned statistics until my second year in university, it was really mind opening, especially with showing me just how much poorly done statistics with tiny sample sizes or misrepresented data are quoted and used as facts.
I actually found statistics pretty fun and eventually we learned to use R for it. Though I ended up missing 1 lecture and it almost screwed me over for the rest of the class. I found a Kahn academy video though that covered it and was able to catch up. There really is a ton of information to learn. It would have been nice to have at least touched on it a little bit in highschool beyond learning about means and the tiniest bit about standard deviations.
They're nuanced concepts that are easily misunderstood, but in a typical US public school, statistical significance (or anything beyond basic descriptive statistics) is not typically part of the curriculum[1]. It's a hard problem to effectively educate out of existence but right now we're not even really trying.
[1] source: I never saw it as a student, and I've consulted with math teachers (well, administrators, about math) in multiple states, and I've never had it come up there either.
