This kind of problems is really not where LLMs shine.
Where you save loads of time is when you need to write lots of code using unfamiliar APIs. Especially when it's APIs you won't work with a lot and spending loads of time learning then would just be a waste of time. In these cases LLMs call tell you the correct API cells and it's easy to verify. The LLM isn't really solving some difficult technical problem, but saves lots of work.
This exactly. LLMs can't reason, so we shouldn't expect them to try. They can do translation extremely well, so things like converting descriptions to 90-95% correct code in 10-100x less time, or converting from one language to another, are the killer use cases IMO.
But expecting them to solve difficult unsolved problems is a fundamental misunderstanding of what they are under the hood.
I picked this problem specifically because it's about "converting from one language to another". The problem is already solved in the literature. I understand that doing cutting edge research is a different problem, and that is explicitly not what I'm doing here, nor what I am expecting of the tool. I have coauthored an actual published computer science paper, and this excercise is VERY far from the complexity of that.
Could you share some concrete experience of a problem where aider, or a tool like it, helped you? What was your workflow, and how was the experience?
I'm a senior engineer (as in, really senior, not only years of experience). I can get familiar with unfamiliar APIs in a few hours and then I can be sure I'm doing the right thing, instead of silently failing to meet edge cases and introducing bugs because I couldn't identify what was wrong in the LLM output (because, well, I'm unfamiliar with the API in the first place).
In other words: LLMs don't solve any noteworthy problems, at least yet.
I feel sort of the same way but I'm desperate to understand what I'm missing. So many people sing such high praises. Billions are being invested. People are proclaiming the end of software developers. What I'm looking at can't be the product they are talking about.
I'm perfectly happy reading man pages personally. Half the fun of programming to me is mastering the API to get something out of it nobody expected was in there. To study the documentation (or implementation) to identify every little side effect. The details are most of the fun to me.
I don't really intend to use the AI for myself, but I do really wish to see what they see.
Maybe for happy path cases. I've tried to ask ChatGPT how you can do a certain non-obvious thing with Kafka, and it just started inventing things. Turns out, that thing isn't actually possible to do with Kafka (by design).
Where you save loads of time is when you need to write lots of code using unfamiliar APIs. Especially when it's APIs you won't work with a lot and spending loads of time learning then would just be a waste of time. In these cases LLMs call tell you the correct API cells and it's easy to verify. The LLM isn't really solving some difficult technical problem, but saves lots of work.