Hey yes we are actually currently running this as part of the next wave!

Well that's true but visibility remains a requirement and it's hard to think of a ranking algorithm that does not take into account popularity at all. Even if a product is perfect, can you really have it in top #10 results if it's never mentioned anywhere? But then if you take into account popularity / citation frequency / etc. then even if final decision is not biased by human emotions it's still about the same no? (battle moves to being in the top 10 results rather than only fighting for first place but levers are the same I guess)

We decided to use three different providers so we could verify that this choice doesn't impact the result of our experiments (E2B, Blaxel and Daytona)

If you want your Claude Code to search you can always tweak your own with a good skill, this should work perfectly! It's more a problem for vendors who can't tell all people in the world to download a specific skill first.

True but they'll probably never integrate ads into the model's thinking (for now it's only a separate display in the apps). When tokens are a commodity you'll just switch to the one you can trust and since all labs are soon going to become tokenmeter companies (Sam's own words), I don't think they can afford to do that.

We don't need to reproduce the same errors! Anyway I do think it is going to be different this time because generating content is becoming so easy today that the entire web would just become 99.99% slop very quickly if things don't change. That being said the solution isn't that trivial, curious if you have thoughts on it?

Youtube at least, is already 99% for new content, outside the channels you already subscribed. The amount of channels that are basically and AI animation with a voice-over of a curious wikipedia article is staggering, but those are the good ones.

The worse part is a deluge of stupid boomer fanfic (The ones about ungrateful kids, ungrateful employers that fired someone who secretly was a load-bearing (ha!) element for a contract, or HOA drama), self-help stuff, and red-pill incel fanfics.

You're talking to the CEO of the want-to-be-SEO company [1] that did that. It is very much content marketing for their services.

[1]: https://news.ycombinator.com/item?id=49557233

Indeed, we are helping growth teams get picked by coding agents, this is not a secret. But I do think this has the potential of getting 1000x worse than SEO so I don't imagine people letting that happen.

For example we just added code review here: https://armature.tech/leaderboards#app/code-review -> See my comment here about how much coding agents choose themselves. I don't expect people to let this last forever for instance, otherwise it's worrying for literally any software out there.

Why? We haven't benchmarked Data Warehouses yet, only prod databases where you wouldn't expect Redshift to be considered.

Azure database was mostly for enterprise use-cases. Rebuilding a lot of things in-house is a real trend, especially for Claude Code when you don't ask it explicitly to consider all solutions and avoid overhead of managing things yourself. Codex and Cursor seem to have this in mind more naturally (at least using GPT-5.6 Sol / Grok 4.6)

Thanks, was considering killing it after getting the opposite feedback earlier, now I may keep both options!

Hey, thanks! I'm wondering if it's clear from our website that this is the price of a fully managed service, not just access to a platform or reports. Think of an SEO agency model.