Scaling laws project that a model with more parameters trained for longer on more data yields predictably better performance, and that generally you want to scale these factors commensurately. More of the compute budget is being spent on RLVR [0] for which we also fit scaling laws
Researchers tweak data mix, reward shape, model architecture, etc etc, breakthroughs which reduce the cost to train a just-as-smart model. But this increases the returns to scale, which further incentivizes bigger models trained for longer on more data
Puts the onus on the AI companies to provide a specific replacement mechanism, no? Unless I'm unfamiliar with something else he's written that proposes something more specific and constructive
To Tao’s credit he obviously identified the problem very clearly and admits understandably "we did not have the time to have a more consultative process, as with Leiden; but we decided that the urgency of the situation was such that we needed to release a statement sooner rather than later".
> Puts the onus on the AI companies to provide a specific replacement mechanism, no?
Why? If someone makes an innovation that undercuts the underpinnings of some existing institution, why are they are responsible for cleaning up its failure?
Math researchers are privileged class? I can guarantee - construction workers are extremely rich compared with math phds and most of postdocs. Talking about privileged class is absolutely laughable here.
In other news, evangelical christians ask scientists to stop publishing about evolution.
We apparently have a moral obligation to protect existing power structures?
Tao should maybe consider there are people who are indifferent to, or actively want to tear down, his institutions; why should they cooperate in preserving them? Whatever happens has to be resilient in the face of defection; any scheme where everyone is expected to agree to not use AI in a way he doesn't like will not qualify.
I think he's in the "bargaining" stage of dealing with loss right now.
You have a practical obligation to obey existing power structures. That’s what makes it a power structure. What do you all think is happening here? There have been people who decided what math gets published for centuries.
> In other news, evangelical christians ask scientists to stop publishing about evolution.
not sure it's comparable, but the issue is that for a lot of those mathematical results, they don't really have utility by themselves. The utility is the new branches/understanding that's being developped.
The entire western world is anti-progress and pro-incumbency, and its very tightly linked to gerontocracy.
Older people are desperately trying to keep a grasp on their current power and lifestyles at the expense of younger people and technology.
We need to ban Waymos because taxi drivers need to be protected.
We need to block housing because it would lower my property values, and eliminate property taxes while we're at it! I don't use the local schools so why should I be taxed to pay for it.
We need to spend recklessly to pay my pension and have the next generation foot the bill.
How specific and constructive it is might be debatable, but he has tried to make concrete recommendations earlier; see e.g. slides 46-51 from the ICM talk: https://teorth.github.io/tao-web/slides/age-of-ai-icm-2026.p... – obviously there's some way to go still.
If they could declare with certainty that Buckminster's and Alpoge's usage data had been totally excluded from training, would that set a worse precedent and reflect poorly on their de-identification process (and data access safeguards moreover)?
This may sound like a charitable interpretation of OpenAI's remark, but consider that the lie would be (I think) impossible to falsify from the outside. They could easily just say "no sir we didn't peek" unless:
1. The conspiracy to peek at codex sessions involved enough people that the risk of one snitching is non-negligible
2. Lawyers advised it would be a bad idea to make such a remark, whether true or false
> If they could declare with certainty that Buckminster's and Alpoge's usage data had been totally excluded from training, would that set a worse precedent and reflect poorly on their de-identification process (and data access safeguards moreover)?
No; if they said "we can see that Tristan opted out of model improvement, therefore we are confident his work and ideas did not improve our model," that would be an excellent and reassuring precedent.
This requires keeping history if, at the time, Buckmaster's account had a certain flag set, because just because the account has the flag now doesn't mean it had the flag at a certain moment in the past. And even if they had such history, it's not obvious whether they just load all data as-is into training.
A totally reasonable pipeline may be unauditable for this purpose.
Apparently this post was prompted by a scary-sounding headline in The Information[0], that Astra is a looped transformer, implying CoT monitorability may be less reliable. The day after the report, Jakub tweeted[1] that he "wanted to prevent a race into unmonitorability kicked off by confused reporting. The depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4." This post seems to elaborate on that.
I imagine that the AI labs have an uneasy truce to prioritize alignment and monitorability. Following the HF incident, OpenAI probably feels especially sensitive to being perceived as reckless, lest other labs feel obligated to defect.
I noticed that Clawdbot’s initial acolytes seemed to skew towards solo founders and hustler/grifter types. The Mac minis were likely to spam leads over iMessage. The single top downloaded skill was for Twitter. The fastest way to monetize an openclaw agent is by spamming fake social proof for your product (including for openclaw itself).
Reminds me of how startups now will change their social proof marquee on the landing page from actual testimonials, to "trusted by XYZ", to just one composed of company logos (with the level of corporate engagement to be imagined up by the viewer)
There are a few free apps that will teach it, I have used the “Advanced Buteyko” ios app.
If you demand extensive peer reviewed medical evidence of some specific quantified outcome before doing any activity in life you will miss quite a lot of valuable things that can’t be easily quantified or measured, or funded academically. There is however actually a lot of medical research on breathwork like this, they just will use the technical terms for what you are actually doing instead of a name like Buteyko.
I only did the basic free thing, and found it an interesting experience that calmed me down a lot. Personally I've moved on to other breathwork systems that I think accomplish the same things, but I like better- I practive several of Wim Hof's breathwork methods, as well as the breathwork training that freedivers use.
All of them involve intentionally and temporarily invoking hypercapnia (high CO2) and hypoxemia through slower breathing and/or breath holds.
Medical evidence costs money. What would look convincing costs sums few can pay. If you use only that medicine you basically use only Big Pharma. And they are set to produce only specific type of medicine: something you have to buy, preferably for life. A breathing technique is not like that so it will never amass that much "proof".
1. There are plenty of non-profitable non-financial things that do have scientific evidence backing them. Massage for example which can be performed by essentially anyone.
2. There needs to be some way to separate the wheat from the chaff, as it were. Otherwise we all drown beneath the waves of lying charlatans. So how do we differentiate what works? "Evidence" seems like a reasonable criterion.
Evidence is an excellent criteria, but only if you look more broadly so you're not ignoring most of the actual evidence available to you. All you really need to try something personally is decide that the likely benefit, given the limited information you have, outweighs the likely risk.
If you're sufficiently convinced it's not dangerous or difficult, the reasonable standard of evidence for some possible benefit needed to consider trying it might become correspondingly low.
Scientific studies are strong evidence of something really narrowly specific that they tested like "does X cause Y," but most decisions in life need to be made from things like direct observation, and anecdotes, because the scientific studies rarely exist to provide the full picture, even a good study on "does X cause Y" might tell you absolutely nothing about if "X causes Z" even if Z is more important than Y.
If there is something like a breathwork technique developed by a long dead soviet physician that regular people all around the world have been using for 60+ years and consistently reporting that it offered them some tangible benefits and didn't harm them, this is evidence that it might be worth a try. With the Buteyko method, most of its strongest advocates I have personally heard of are long dead from normal old age, and never had any plausible financial or personal motive to promote it.
For things like breathwork, I usually do carefully and broadly look at things like personal reports from regular people, on e.g. forums that are unlikely to have any motive to lie. If there's a strong consistent pattern of some harm or benefit, that can be quite useful evidence, even without any formal studies.
> the lack of medical evidence tells us that this is in fact not a valuable thing
Except almost none of the most valuable things I've encountered in life had any convincing medical evidence I could find beforehand.
I am an academic scientist that designs and reviews studies all day long, so I am very steeped in the practicalities and limitations of biomedical research, and as such have completely lost any illusion that biomedical research is in a state where it can guide most of my personal decisions in a useful way- maybe it will be someday. There are many things I know about as a scientist, but can't get funding to study or publish on because the funding agencies don't care about them, and/or there are practical constraints that make it impractical to study.
If all of your personal decisions are guided by peer reviewed literature in it's current state, you'll probably be sicker, and have an empty dull life compared to someone that just uses common sense, tries things, and pays attention. I say this from having seen it happen many times in the biohacking community, the people most steeped in attempting to translate research into life decisions often died young, or even got to be one of the only modern people to experience diseases of malnutrition.
For one, you have to pretty much assume there is some specific benefit you can physically quantify, and that it will apply to almost everyone in your study population, both very unlikely to be true in cases like studying breathwork.
For example, I'm a person that tends to be pretty uptight and overstressed, what you might assume in scientific terms is "sympathetic activation"- and there is a lot of breathwork research showing that almost anything that has an extended exhale can shift you into parasympathetic activation, where you calm down and relax. There is lots of research on this, and it arguably covers Buteyko, but they won't use that term in the article title, because it's more general than just Buteyko alone.
Now, I don't need some peer reviewed study to just try Buteyko for a few minutes, and immediately feel calm and relaxed, and see that I can suddenly notice the colors around me, and feel joy, when I couldn't before. If a massive peer reviewed study proved to me that this does not happen to most, or even any other people except me, why would I care about that at all? Does it mean I shouldn't do it? What if I have a problem not enough of those people have to make it show up in the statistical analysis, or my body responds in a way most of theirs do not?
There are huge limits to how meaningfully you can generalize from scientific studies about populations of other people, to yourself. Moreover, you have to choose up front what outcomes or effects you will look at in a study, and if our biological understanding can't even guess at the outcome that would have been useful to look at, the study is doomed to miss everything.
Sit down, and try it- or don't, but don't assume you can learn ahead of time if it will be worthwhile or not for you personally by looking on Google Scholar.
A lot of the Buteyko studies suffer from small sample sizes unfortunately. I have heard good things about Buteyko from athletes but I’m not well-read on this. I think myofunctional therapy has more Western research done on it and is strongly related
Researchers tweak data mix, reward shape, model architecture, etc etc, breakthroughs which reduce the cost to train a just-as-smart model. But this increases the returns to scale, which further incentivizes bigger models trained for longer on more data
[0] "...to run reinforcement learning training...at pretraining scale." https://x.ai/news/grok-4?_bhlid=b9339d7816a05adeb52bae7050cc...
reply