Is this an actual question? I’ll answer it anyway: because they had nothing to do with financial market risk shenanigans and just wanted to get somewhere.
I’m in the skeptical camp. Whatever theory that will eventually emerge will not be as solid as:
1. Theory of pattern recognition (as developed in 80s and 90s)
2. Theory of thermodynamics
3. Theory of gravity
4. Theory of electromagnetism
5. Theory of relativity
Etc. because of two reasons:
1. While half of deep learning is how humans construct the architecture of networks, the more important half relies on data. This data is a hodgepodge of scraped internet data (text and videos), books, user interactions etc., which really has no coherent structure
2. To extract meaningful insights from this much data, it takes models of enormous size like 10B+. The thing about random systems (in the mathematical sense) is that it takes “something” of order of magnitude bigger size to “understand” it, unless there is some concentration of measure type mathematical niceties (as in thermodynamics), which I don’t think is there in these models and data. This is the same reason I don’t think humans will ever be able to “understand” human consciousness. It will take something of an order of magnitude bigger than our own brains to do that.
Here is Terence Tao explaining this concentration stuff in another context: https://mathstodon.xyz/@tao/113873092369347147
I would love to be proven wrong though.
The whole point about theory, though, is that simple rules can define complex phenomena. I don’t think anything you wrote fundamentally rules out the idea that we could find a theory of deep learning.
I really couldn't have been more obscure, could I? :P
In 1932, "the first oil field in the Persian Gulf outside of Iran" was discovered in Bahrain [1]. (The same year Saudi Arabia announced unification [2].)
In the end, Saudi Arabia had larger reserves and wound up geopolitically dominating its first-moving rival. In commodities, the game tends to be scale in part through land grabbing. Less who got where first.
To close the analogy, if AI does wind up commoditised, the layers at which that commodity is held are probably between power and compute [3]. So if AI commoditises (commodifies?), Google selling computer (and indirectly power) to Anthropic and OpenAI is the smarter play than trying to advantage Gemini. (If AI doesn't commoditise, the opposite may be true–Google is supercharging a competitor.)
I should say it’s “mostly” a myth, there are some fleeting competitive advantages to first mover but a lot of them don’t apply well to tech companies and there isn’t strong historical evidence supporting it.
Why? Being a first mover only counts for something if it can yield exclusivity that is durable.. you should know this being a VC and all. Real options - hello?
If you want to benefit massively off being a first mover, you better do the work in figuring it out how you are going to acquire exclusivity that lasts long enough that keeps most firms out.
The vagueness is by design, it’s another dark pattern. “Delete all photos from icloud? [are we gonna delete the ones that we only keep compressed versions on your phone? Iono ¯\_(ツ)_/¯, you wanna find out? Yea, didn’t think so...]”
reply