I've seen a lot of LLM uses that are really just zero/few-shot classifiers with a lot of extra steps, so it is interesting to see more models that are taking advantage of all the intelligence encoded in the latent spaces of these models with really efficient output. It feels like this is an under-explored area of LLMs right now and I'm excited to see what comes out of it.
I've found sorta the opposite - in any area, it can just do everything for you, or it can be an incredible teacher. I've been re-learning a lot of higher-level math and it has been an knowledgeable, infinitely patient, always-available tutor. Of course, I could just have it do just about any math I want for me, but that's not the point.
Kinda the same with language/technology stuff - it can be a great tutor and it can scaffold other parts of a project for you. It can give you feedback and let you focus on the interesting parts.
I guess the motivation itself may be hard because of the fear of it taking over much of our jobs, but having this kind of help/feedback is pretty cool for the sake of learning things just because they are interesting!
> I've found sorta the opposite - in any area, it can just do everything for you, or it can be an incredible teacher.
Please don't. I've had all of Codex, Claude and Gemini convincingly tell me absolutely wrong stuff, pointing it out with easily verifiable example they come up with more and more weird reasons.
Things don't become correct simply because most sources are again - easily and logically verifiable - wrong. This already was a plague when people "just googled" stuff and effectively returned with the most SEO optimized answer. Now we have very convincingly written instances all over the place.
If these were singular instances I wouldn't be so worried, but if you are learning it already is very easy to learn something wrong. This is why back in the days when people still used physical books to learn new things it was a good idea to check first which books are actually recommended. There have been a lot of "experts" that wrote things they clearly misunderstood but worked for all the examples in their books.
To give a common example for both the backend and frontend devs, that isn't about a specific projects. LLMs and Google searches frequently turn out wrong results regarding CORS caching and how it works in relation to domains/hostnames. The circumstances under which Content-Disposition work are another example. I think a lot of wrong statements that LLMs are "convinced" about are due to wrong statements (sometimes in otherwise correct response) of popular Stack Overflow answers.
It's saddening how much wrong "common knowledge" exists in the industry. I have been bitten by a lot of these, but it feels when people don't even actually code and think anymore this will just rise forever.
Can these be wrong? Certainly. So can humans. Many of your examples are of humans being wrong. That doesn't make LLMs - or humans - useless. The fact that they are not infallible is not a reason to avoid using them and I'm not going to throw out a tool that has been incredibly valuable to me because someone on the internet got some bad CORS advice.
Learning is about so much more than accessing information, though. It is about building mental models, resolving ambiguity, exploring things that the source doesn't explain very well, and so much more.
Questions like "What do these lines of code do?" or "How does this fit into the big picture?" or "Wait, this doesn't make sense?" are rarely answered by the source.
This is a bit fresher in my mind in the math domain, but I don't think it is any different in any number of other domains, including coding. I've been working through a math textbook, gotten confused about how the author gets from step 2 to step 3, taken a picture of the text, and had AI explain it to me - it almost always gives me a much better understanding of what is going on and helps make things so much clearer. There is a level of interactivity that can't exist in a book or other "source" of information.
I think there is a bit of tension when it comes to learning and sometimes the struggle itself is informative, but there is a reason people hire tutors and go to classes taught by teachers vs just reading a textbook, and cutting yourself off from a tool because you've seen it be wrong about something seems like a silly mistake.
The opposite problem can happen- the CEO uses the product all the time and becomes blind to problems. “It has always worked that way”, or “who would want to do that!?”” are much more common than pure apathy.
They also get massive subsidies and tax breaks for building these data centers. They require the negotiations be done in secret and often fight to keep the agreements secret to make it so people don’t flip out when they see how bad they are.
Kind of a fun toy problem to play around with. I noticed you had thread coarsening as an option to play around with - there is often some gain to be had here. I think this is also a fun thing to play around with Nsight on - things that are impacting your performance aren't always obvious and it is a pretty good profiler - might be worth playing around with. (I wrote about a fun thing I found with thread coarsening and automatic loop unrolling with Nsight here: https://www.spenceruresk.com/loop-unrolling-gone-bad-e81f66f...)
You may also want to look at other sorting algorithms - common CPU sorting algorithms are hard to maximize GPU hardware with - a network sort like bitonic sorting involves more work (and you have to pad to a power of 2) but often runs much faster on parallel hardware.
I had a fairly naive implementation that would sort 10M in around 10ms on an H100. I'm sure with more work they can get quite a bit faster, but they need to be fairly big to make up for the kernel launch overhead.
> I'm surprised you're not touting the "save on your power bill" benefits.
At ~$600/kWh for capacity, the ROI isn't great. I have a pretty big differential on my rates because I have an EV, and even then I'd need over a decade to make the $1,000 back assuming I fully discharged it every day.
Are you sure they ditched CUDA? I keep hearing this, but it seems odd because that would be a ton of extra work to entirely ditch it vs selectively employing some ptx in CUDA kernels which is fairly straightforward.
Their paper [1] only mentions using PTX in a few areas to optimize data transfer operations so they don't blow up the L2 cache. This makes intuitive sense to me, since the main limitation of the H800 vs H100 is reduced nvlink bandwidth, which would necessitate doing stuff like this that may not be a common thing for others who have access to H100s.
I've only done the CUDA side (and not professionally), so I've always wondered how much those skills transfer either way myself. I imagine some of the specific techniques employed are fairly different, but a lot of it is just your mental model for programming, which can be a bit of a shift if you're not used to it.
I'd think things like optimizing for occupancy/memory throughput, ensuring coalesced memory accesses, tuning block sizes, using fast math alternatives, writing parallel algorithms, working with profiling tools like nsight, and things like that are fairly transferable?
So many fond memories of this game - it was a really fun blend of railroad sim and economic sim that I haven't really found since. I'll never forget the "ding ding ding" sound that goes off when a train pulls into a station and earns you a bit of cash!
My non-expert brain immediately jumped to double-pumping + maybe working with their thread director to have tasks using a lot of AVX512 instructions prefer P cores more. It feels like such an obvious solution to a really dumb problem that I assumed there was something simple I was missing.
The register file size makes sense, I didn't think they were that much of the die on those processors but I guess they had to be pretty aggressive to meet power goals?
reply