This has been on my mind the last few years and I've thought about writing an academic paper about it in one domain.
I've sort of increasingly suspected that some domains might involve an intrinsic amount of stochasticity or uncertainty that is baked into things. As in, an amount that becomes nonignorable relative to what you're trying to predict.
There's some math and comp sci papers that basically argue (in a proof sense) you there must be a limit to modeling. It's been awhile since I read the papers but my recollection is that the modeling process itself creates (necessitates?) a certain gap between the model and the real world that creates hard boundaries.
For example, if the modeling step requires an amount of time A, some information is lost in that time.
Can I recommend looking at what people studying Experts at work and how they manage to do better than they "should" be able to have found? There are tons of domains looking at this empirically :)
I've sort of increasingly suspected that some domains might involve an intrinsic amount of stochasticity or uncertainty that is baked into things. As in, an amount that becomes nonignorable relative to what you're trying to predict.
There's some math and comp sci papers that basically argue (in a proof sense) you there must be a limit to modeling. It's been awhile since I read the papers but my recollection is that the modeling process itself creates (necessitates?) a certain gap between the model and the real world that creates hard boundaries. For example, if the modeling step requires an amount of time A, some information is lost in that time.
These are a couple of the papers:
https://arxiv.org/abs/0708.1362 https://www.jstor.org/stable/10.4169/amer.math.monthly.121.0...
I suspect as the system you're trying to model is more complex or has more dependencies it gets worse.