Fair points, but you are glossing over important issues:
1. You are using the current point-in-time cost for batteries as compared to a subsidy spent over years while battery costs were dropping.
2. You do not include transmission upgrades and costs to distribute energy from centralized batteries.
3. You ignore the benefits and efficiencies of having generation and batteries distributed and close to the source of demand. It also increases resilience of the grid.
4. You assume the political will was there for government and taxpayers to bear the full cost burden of development of the batteries.
1. The subsidy was fixed per kWh at a level that has always been vastly above utility scale BESS.
2. Power usage is flatlining. There is no reason to need transmission upgrades for BESS. Successful projects are colocated at retiring generators, or at existing major grid terminus.
3. Are those benefits worth 4x the price? Noting again, that's 4x the subsidy, not even the actual installed cost which is closer to 8-12x.
4. It would have been cheaper even at just the subsidy rate.
Where are you getting that? The 2026 AEMO plan says:
"Electricity consumption is expected to almost double over the coming decades, driven by population and economic growth, electrification across homes, businesses and industry and the emergence of new large loads such as data centres."
Are you really holding up Snowy 2 as a success story? We're past 20bn on Snowy 2 now. Run those numbers against grid scale BESS, on both capacity and peak generation and get back to me - 10 years ago it looked good, now it's not favourable.
lots of people confuse a $75-$120 / kWh cell price for grid scale battery installed price, which these days is about $300 / kWh.
1 gigawatt hour of installed BESS therefore at current prices is roughly $300 million.
Snowy 2 is 350 gigawatt hours, which would cost $105 billion in equivalent batteries which is about $150 billion in Australian dollars, or ~7x cheaper per gwh.
If you price it on throughput and not storage size, because Snowy 2 is very constrained on when it can charge and discharge then the realistic 8Twh per year you can get out of Snowy2 loses to batteries by a factor of 3 based on your costs.
But that's assuming the batteries can charge and discharge in full every day.
The truth is probably somewhere in the middle where Snowy can act as seasonal storage with a big capacity that isn't used every day.
I think it would then also lose on price if compared with a portfolio of batteries alongside solar and wind to directly supply electricity and continually recharge the batteries over seasonal scales but that is a more complex comparison.
In general I think even good hydro schemes (and Snowy 2 is not a good one) rely on non-electrical factors to be worthwhile now batteries have plummeted in price, e.g. managing seasonal water flows for agriculture, and the electricity can be seen as an additional benefit.
sigh This reminds me of all of the tedious arguments by people reminding us that solar panels don't work at night, implying that they are therefore useless.
Or the ones that remind you that batteries are useless for seasonal storage.
Or that the wind doesnt always blow.
Yes, individually all of them have crippling flaws and all of them work best when combined with technology that has complementary crippling flaws.
Snowy 2 won't be used instead of batteries it'll be used alongside batteries.
Right now nothing comes close to beating its price / gwh no matter how much the cost overruns or how many people dump on it for being 7x cheaper per gwh than batteries instead of 10x.
More countries should be building one as well as building out grid level batteries. They complement each other.
This is an interesting idea, without giving away your benchmarks specifically what kind of stuff do you test? I might try to assemble something myself, it's so hard to determine model quality from the system card these days.
I have actually been planning to open source the framework. Only benchmarks I care about are the ones that look like real work I do. So it makes it easy to trawl your own repos looking for benchmark task candidates from real bug fixes or features. Then as a bonus I can compare the agent’s solution to my own as a reference.
So most look like that but I did include a few one-shot “build an app that solves this problem” and some qualitative design tasks and a tough algorithmic optimization one.
Also working on a product to build tasks from your own work for testing coding agents. Main thing I would offer is to look carefully at the agent trajectories - they love to figure out ways to cheat. Additionally, consider what "winning" means. If just using test pass rate, consider that tests might not encode what good means in your repo. I have been having success using "equivalence with merged PR" as judged by an LLM as a signal.
Yup, trying to be really strategic about testing. I didn't end up sticking with it, but I tried requiring test cases to cite a matching clause in the task assignment. But also: these tests are only indicative. Only a human can score a run.
Would open sourcing it make you feel like the results might be compromised? I'd rather keep it private so it's specific to my use cases and not included in any training date (no matter now small the signal in the overall data).
I am increasingly wondering if we are in a post-language world in terms of development. Why would I ask an agent to write a server in anything other than the most efficient language, although efficiency can take several forms: runtime, token usage during development, and wall clock dev time (affected by slow compile times for example).
My intuition is that type-safe languages with fast compilers are the best option. Maybe Go? I personally prefer Java just due to my experience running it in production, but am not sure there's many arguments for it over Go in a greenfield application. The other candidate would be Rust, but I worry about token efficiency and tool performance, I suspect it's not worth it for the runtime improvements.
All that being said, in this article switching to Python seems like a wild choice. Relatively poor performance, no compile time checking at all. Python's big selling point was developer ergonomics, which seems largely irrelevant now.
These are all just thoughts at the moment, I should try to find some evidence one way or another.
I felt similarly. I wonder why the author is so invested in Haskell specifically becoming an AI-pilled ecosystem and community when the choice of language rounds to not mattering?
Go is the perfect language for this new world. Its development loop is fast, it has types but not too much so, it is memory safe, it is easy to deploy and it runs efficiently enough for most line of business use.
Go was originally developed specifically for a world of interchangeable hard-working juniors, which is exactly what LLMs can scale up with only your budget as the constraint.
I'm wondering what advantage a hypothetical faster-compiling-Haskell would have in that world.
Language choice had less impact than people first assume even before LLMs in most software. A good engineering team produces good code in whatever language they happen to be using. In my own career I've worked in serious Java, Scala, Haskell, Javascript, PHP, and Python application stacks and I've seen plenty of good and bad examples.
I reckon language choice matters more at the edges of economic activity where a specific language feature really does make the difference in the end product, but most activity that is leveraging LLMs now is more generic enterprise SaaS software.
My thought reading this article was: Why write the system in any one language at all? And I don't just mean having some parts in one language and others in another language, I mean redundant implementations of the same parts. You can use an AI to rewrite parts of the system, and then throw away the old part... or you could just keep the old part.
That is: Have a Haskell base system. Have a Python "development" version on which you iterate at lightning speed. But also, in the background, moving at whatever pace it takes, have an agent running that imports all the Python development changes into the Haskell version. Have nightly builds of the Haskell version to reap its benefits (issues caught by the type system, more efficient native code). They must have continuous or nightly processes to fix bugs in the Python code anyway, there is no way that all the things they ship "while still on the call with the customer" are always tested on the full test suite and always 100% correct.
And it doesn't have to be Python/Haskell of course. The "development" version could be a (hypothetical?) interpreted Haskell. I have no idea if ghci would be useful for this. Neither do I know if the 15-minute Haskell build time is spent in the frontend (so an interpreter would have to pay that cost too) or in code generation or linking (which the interpreter wouldn't need to care about). Anyway, these are things I would think about before I did what the OP did.
You are ignoring LLM-ergonomics, some time ago I saw benchmarks showing that popularity the language (and so more data available in training date) was strongly linked with LLM's performances, with top results with javascript and python. I don't know if a year later this is still true, but is absolutely possible
There is one simple thing you have to realize why Python is the optimal choice. You have so much training data. Python is the second most popular language on GitHub and is easy to read.
I would argue that it helps me when performing maintenance to see and correct where types may have changed. Not always, and sometimes it is busy work I agree, but overall I prefer it.
Yeah I generally avoid using 'var' to elide method return types for that reason. In my early phase of var enthusiasm I even had it result in a bug that would have been caught if the type had been made explicit.
I do still really like it in the `var list = new ArrayList<String>()` case though.
I agree, and I'm not sure how something like 'var accounts = calculateAccounts(something)' can be thought of as better in a code review setting. I suspect using "var" or equivalent will be considered a problem by most companies within the next few years.
Those writing in python and javascript don't honestly know any better. They grew in a world without unit testing or without products that need to grow into gigantic systems maintained over the next decades.
This reminds me of a developer writing in jRuby because "it was better". While he was in the company he'd still give support to his own works, after leaving nobody else wanted to pickup those "better" things and would prefer to write workarounds to the tool, it effectively became a black box that few could improve and even worse to test. As result, those portions had to be rewritten in proper Java so that we'd be able to deeply measure/test and improve.
There’s nuance in programming. Both of these statements can be true. Var reduces boiler plate when you’re duplicating information in both the lvalue and rvalue.
I’m in agreement that when the information isn’t in the rvalue that you shouldn’t use var.
Side note, the Connection Machine is pretty much the coolest looking computer ever: https://www.computerhistory.org/revolution/story/73 It looks exactly to me what a powerful and slightly scary computer from an 80's movie looks like.
I mentioned this last time around [1], Tamiko Thiel worked with Feynman and Hillis at thinking machines and is responsible, amongst many other things, for how cool the CM-1 and CM-2 looked.
And it was designed after the T-shirt that feynman wears on one of his most known pictures [0]. BTW: there we still have a nonfunctional CM which we equipped with LEDs to put fun games on it at the CS faculty in Karlsruhe [1]
The blinken lights panel on the original machine was functional, each small cluster of processors controlled one LED and there was a microcode instruction for latching the LEDs.
We found this museum completely by accident on a visit to the US, and it is a delight. When I saw the Connection Machine all lit up, I squealed :)
It seems like the reason it's on like that is because it used to belong to the NSA so they took all the insides out and destroyed them when the machine was donated. The blinkenlights are driven by Raspberry Pis or something similar.
I can understand the frustration from a climate change perspective, but you still get the benefits of cleaner air locally, increasing energy independence and freedom from commodity cost fluctuations.
It's an interesting trade off between swapping station cost and time to charge. I suspect charge times will decrease enough that it's not worth all the physical investment in swapping, it may already not be.
Swapping for regular cars is a dead end I think. But for trucks and such I imagine it makes a lot more sense. Much longer charging times vs a quick swap. Much easier to put battery in an easy-to-swap location. Fewer swapping stations needed before it makes sense to utilize it for transportation companies, ie team up with a company that has a few fixed large-volume routes to get going.
My understanding is that this the vulnerability only allows memory access to related Safari/Webkit processes (specifically those sites that were opened with a window.open call). So passwords stored in a separate password manager app are inaccessible unless that app autofills the password into the compromised Safari window/process.
1. You are using the current point-in-time cost for batteries as compared to a subsidy spent over years while battery costs were dropping.
2. You do not include transmission upgrades and costs to distribute energy from centralized batteries.
3. You ignore the benefits and efficiencies of having generation and batteries distributed and close to the source of demand. It also increases resilience of the grid.
4. You assume the political will was there for government and taxpayers to bear the full cost burden of development of the batteries.