Percentage probabilities are meaningless for an event like this.
Can you run the 2016 election 1,000 times and count the times Hillary wins? Of course you can't.
Unlike estimating "what's the chance it rains tomorrow?" you can't even find 1,000 elections in the past with roughly similar conditions.
Unlike a baseball season, you don't have enough samples for the true probabilities to come out.
So, yes, despite all your downvotes, you're right: he pandered to the people who want a percentage number, despite the fact that it's meaningless and they will misunderstand it.
This is a lot like saying, "fractional exponents are meaningless, because you can't multiply something by itself half of a time." This intuitive model has broken down, but the problem is that the model wasn't sufficient to capture the complexity of the problem; it was a pedagogic stepping stone.
Similarly, probability is not the result of conducting repeated experiments (though the results of repeated experiments can be used to estimate probability). It's the measure of an event relative to the measure of the set of all events, eg, it's how much of the total volume in the space is taken up by a given event.
If you have a question I'm happy to clarify. If a restatement is helpful, the problem is that this definition of probability is limited and fails to capture the complexity of the problem domain, not that the problem domain is meaningless.
You're misunderstanding the terminology, "events" refers to sets of potential outcomes (eg, when rolling a die, the outcomes are 1-6 and the event "even" is the set of 2,4, and 6).
(I don't know 538's actual methodology, consider the below an illustration rather than a literal description of 538's model.)
The outcomes here would be each way the electoral vote could possibly be allocated, based on the number of states and the idiosyncracies of each state constitution. The events would be the division of these outcomes by who wins the election in that instance. The model then seeks to attach a weight to each outcome based on how likely it is.
You can think of these weights as the volume of each outcome. We use some kind of model to assign them. The most straightforward model would be to repeat an experiment many times and then assume the rate at which the outcome appears in your sample is approximately the same as it is globally. Other times that isn't possible and you need to use more sophisticated modeling techniques. I'm not going to offer a defense of statistics and machine learning writ large, but I think looking around it should be pretty clear these things can work well even if there's a lot of art to getting it right.
You can imagine comparing the probabilities of each event by pouring their constituent outcomes into separate graduated cylinders so that you can compare their cumulative volume. If you divide their volume by the total volume of all outcomes, that's the probability of that event. Or rather, that's the estimate of the probability, given our model and the data we have.
This is more or less my drunk history version of measure theory and probability theory. I imagine I've gotten some details wrong, so I'd encourage anyone interested to check the subjects out for themselves. Here's some YouTube videos:
> Percentage probabilities are meaningless for an event like this.
No, they aren't.
(They can’t be validated for a single event in isolation, but 538 predicts lots of election events, so their models can, generally, be assessed, and are historically quite accurate.)
Yes, I have. I have even explained thr contradiction.
> You just said you like 538.
Not in this subthread.
> Their "lots of election events" are not different samples of a single variable -- they're different variables with different methodologies.
No, its lots of events with the same methodology (within an election cycle, within any class of election events.)
Now, in principal the Presidential general election prediction is a unique (per cycle) event, but the rules for assigning electoral votes based on lower-level electoral results are known in advance, and the national Presidential election is (equivalent to) a mechanical composite of the state level predictions via a model accounting for their degree of interdependence; both the state level prediction and the degree of independence of their variation are testable, and the model does not purport to predict variations due to faithless electors or interventions to assign electors other than by the state election rules in place prior to the state elections.
EDIT: Actually, there’s lots of predictions to evaluate even for the overall Presidential prediction, because there is a new predictiom every time they update based on new polling data, and you can evaluate the accuracy of the model using all of those predictions, not just the final one.
> > You haven't contradicted anything.
Yes, I have. I have even explained the contradiction.
Let's dissect this:
> No, its lots of events with the same methodology (within an election cycle, within any class of election events.)
So if that were accurate, it would require that he have an algorithm which is exactly the same from election to election, and requires no human tweaking for each one. Is that the case? Does his methodology never change, which would require that the inputs are always done the same way, too?
I think you see the problem here. The "inputs" are the polls which get done mostly by third-party polling firms, using methods that are always evolving.
Secondly, I don't believe 538 has an algorithm that's free of human intervention and never changes. What that means is, each election is a unique event.
Even if each election cycle was a unique event because of model tweaks between cycles, there are hundreds, perhaps thousands, of events and several orders of magnitude more of predictions at different times in each cycle.
No, there are not "hundreds, perhaps thousands, of events and several orders of magnitude"
There is one event. The US House, Senate, and state elections have completely different methodologies and no bearing on the Presidential election, which is enormously complicated by the Electoral College.
> The US House, Senate, and state elections have completely different methodologies
No, they don’t.
> and no bearing on the Presidential election
Even when considering the Presidential general election in isolation, there are currently 56 (51 statewide, including D.C., plus by-district elections for 2 electors in Maine and 3 in Nebraska) elections of presidential electors, which not only “have a bearing on” but strictly determine (absent irregularities that 538 doesn’t purport to address) the Presidential election.
OK, you're right, there are 56 elections, and what I meant by "no bearing" is, there has to be polling in most or all of the swing states. Furthermore, the swing states have weights. So it's a much more difficult problem than a single district, and a much noisier estimate since the per-state noise compounds.
I don't blame anyone for getting it wrong. It's a hard problem.
> Can you run the 2016 election 1,000 times and count the times Hillary wins?
I assume you mean "in real life", but otherwise this seems to be close to their actual methodology. You can see the outcome frequency charts under "What to expect from the Electoral College."[1]
Well yes, I mean, all simulations are, right? You put as much real-life data in as you can, but everything's an abstraction unless you're the team from the Devs tv series.
The actual election that happened is not a simulation, at least not the same kind of simulation that was run by Nate.
That qualitative difference is fundamental. It suffers from the same problem when results of simulations are applied to the "real world". The quality of the simulation is paramount, and there's no way to determine whether the simulation is good enough besides assertions that the simulations are really sophisticated and Nate is very smart.
If you apply the same statistically model to many discrete events, you can evaluate its accuracy. Intuitively, High confidence + Correct answer > Low confidence correct answer > low confidence wrong answer > high confidence wrong answer.
Burning my roulette wheel after winning doesn't invalidate the fact that there was a 1/38 chance of doing so.
You can stop there. Does 538 have a single model that never changes and never receives human tweaking? Does it in turn rely on third-party polls that also never change their methodology? No, of course it doesn't.
The last sentence illustrates why people's concept of probability is so off. Cards, dice, and roulette wheels are not a model for human events.
>Can you run the 2016 election 1,000 times and count the times Hillary wins? Of course you can't.
If you think that evaluating their model based on binary outcomes is the best way to do it, then, well, this argument makes sense.
If you want to evaluate the model based on all of the details available, then, well, it stops making nearly as much sense. What explanations were given for potential Trump victories? What information was available at the time? What information was available after? Do we have data around if those explanations did align with Trump's victory?
I've not done a deep dive on this - 538 is entertainment for me. But from what I remembered in 2016, what I read today in digging up some example articles for another comment, and looking at the actual outcomes... yes, those explanations were closely aligned with reality.
Humans have to come up with probabilities and make decisions based off of them all the time, even when we will never get a chance to get 1000 occurrences. These might not be provable in the same way as something we can run 1000 times is, but the idea that they are meaningless and without value is bizarre.
it's not surprising that so many people have trouble with this. Everyone studies card games, dice, and pulling a red ball out of a jar with N red balls and M white balls, because those are idealized problems.
Can you run the 2016 election 1,000 times and count the times Hillary wins? Of course you can't.
Unlike estimating "what's the chance it rains tomorrow?" you can't even find 1,000 elections in the past with roughly similar conditions.
Unlike a baseball season, you don't have enough samples for the true probabilities to come out.
So, yes, despite all your downvotes, you're right: he pandered to the people who want a percentage number, despite the fact that it's meaningless and they will misunderstand it.