Hacker Newsnew | past | comments | ask | show | jobs | submit | dev_hugepages's commentslogin

If i'm not mistaken, this already exists, and the assembly language here is called PTX

https://llvm.org/docs/NVPTXUsage.html


PTX is a bytecode format, the CUDA driver JIT compiles it when uploading into the cards.

Can't the same also be said of much of the x86 vocabulary at this point?

I appreciate that we can upload SPIR-V directly. The API still feels overly obtuse but it's not so bad.

SYCL gets close but is language specific.


This is called an oomkiller. The kernel has one but it kicks in very late and the kernel prefers to do page trashing instead of killing processes.

systemd-oomd should be integrated in systemd, you can configure it to your liking and see if it improves your problem.


I wish there was an easy way to configure it to say "target user processes first, specifically java (or these days python)" as in my experience they are always the culprits. Processes owned by system accounts or root should be the last ones killed.

Similarly, in the past I have wished for the ability to exempt a process from the oomkiller. I've run servers where the top memory user was also the server's entire reason for existence, and if that process gets killed the server may as well be down. It would literally have been better for any other process to get killed, but it was always the application process because of the memory usage.

systemd-oomd works reasonably well and there is source code. Perhaps claude can help add more detailed policy support to it.

Just found this comment:

https://news.ycombinator.com/item?id=49663299


I believe that you mean: https://en.wikipedia.org/wiki/Thrashing_(computer_science)

Chris Siebenmann discusses when the OOM killer triggers: https://utcc.utoronto.ca/~cks/space/blog/linux/OOMKillerWhen

Chris disables systemd-oomd after it obliterates his X session with no explanation: https://utcc.utoronto.ca/~cks/space/blog/linux/SystemdOomdNo...


> First off, this is exactly how systemd-oomd is supposed to behave under memory pressure. The documentation is specific on this; systemd-oomd itself says:

> > [...] If the configured limits are exceeded, systemd-oomd will select a cgroup to terminate, and send SIGKILL to all processes in it. [...]

> By having the user@.service template be enrolled in systemd-oomd, Fedora made the cgroup that systemd-oomd would select to be killed be all of your processes (across all of your sessions, if you have more than one). ...

Maybe *Fedora* has fixed or improved in the last 4 years. Or maybe they don't run Fedora.


One thing Fedora does now, is use zram.

In my experience it works really well. I wonder why my computer is a bit sluggish, and find out I have several gigs in zram.

If that was in swap on a disk, it would be really painful.


If it were swap on disk fronted by zswap, it'd be even better ;)

I'm a Fedora developer and I can assure you that Fedora's behaviour when it runs out of memory is still terrible.

Adding to this, glad systemd-oomd finally added solid rulesets in 261

Made it far easier to target any containers that got too hot rather than ever risk anything higher priority.


To be fair the internet also had huge amounts of slop before the advent of chatGPT


I always wonder why this comment chain exists in every thread about LLMs uniquely harming our sphere of information.

Yes, <bad code, fake pictures, security vulnerabilities, malicious actors> existed before LLMs. No one is saying otherwise.


This is a quirk of the last gpt image model (gpt-image-2). It put this sort of high frequency noise on all of the image especially if it's in a "drawn" style. There is often lots of other tells that this model in particular generated it.

Image models somewhat watermarking the image in a way that's very easily identifiable by a human seems present in all the image models of the big labs, since DALL-E 3 on OpenAI's side and the first nano banana on Google's side. I have no idea what they did to reach this and why they don't try to fix it.


Just use Ideogram. Locally.


I'm unsure why you're using an LLM to generate images. Don't we already have models (some made by the same company) that do this?


I do AI assisted images for fun and browsing their art I can't see any tell of AI or even tracing on AI; people became paranoid and jump to conclusion without any substance and I find it sad that people who put a lot of time and effort into their craft are getting knee jerks comments like this.


> people became paranoid and jump to conclusion without any substance

Before AI, the only question was who made a particular artwork. Human-created practically a certainty.

When AI-generated art was new, it was all exiting & the process described to boast about AI capabilities or the skills of people deploying AI. But human-created still the default assumption.

Now, AI-generated <anything> is everywhere, and that default assumption of "human-created" is often questioned.

The natural next step is "AI-generated" as default assumption, unless claimed / shown (or even proven) otherwise.

Imho: assumption is the root of all evil. Investigate! Or take an author's word for it unless they're shown to be a liar. But as another commenter noted, it's a kind of tragedy of the commons. Methods to prove you as a human created <artwork> will be a thing.


> I was put off by the overly dramatic presentation. It gets tiring that the author apparently finds this more exciting than I do, and writes like it's enthralling.

That's one of the main tells that AI wrote this. All the stylistic tics that people usually point out combine to make the writing seem more important than it is.


I heard there's a way to arrange code such that the compiler can autobectorize easier. I wonder if there's a way to do that here?

Would probably have to pass `-C target-cpu=native` to cargo so that llvm is allowed to use AVX512.


Good question. I personally doubt that the compress instruction is easy to coax compilers into generating, as there are many edge cases to consider.

For example, you'll notice here that we perform a full vector store of 8 elements unconditionally, even if only a few of the elements are active. This is safe, though, because the output buffer is as large as the input buffer, and we're chunking by 8, so we'll never trash memory past the end; but this is a tricky analysis. Performance-wise, we rely on the CPU's store buffer to make these overlapping stores cheap.

Instead, you might think that you could just store the elements which are actually active, using a masked store. In fact there is also an intrinsic for this purpose (_mm512_mask_compressstoreu_pd), but it is extremely slow on some CPUs, namely Zen 4, so it's dangerous to use unless you know exactly what CPU you're using. (In my testing, there also seems to be some weird hazard on Zen 5 where multiple memory-destination compress instructions to nearby, even non-overlapping, addresses are serialized. But I haven't looked closer at this.)


I think WUFFS "iterate loops" might help. WUFFS requires that processing a chunk of N items has code to process one at a time, which means it'll work for any N. However you can optionally provide specialisations for doing K at a time and the compiler is responsible for carving the input up as appropriate so e.g. N = K + K + 1 + 1 + 1 your K-at-a-time code runs twice, the extras are handled 1-at-a-time.

So this divides up the problem, the compiler can vectorize your 8-at-a-time code without needing to handle edge cases where N isn't a multiple of 8, and if a later pass notices we actually never end up using those edge cases they're dead code, if it doesn't they're just a rarely-taken branch once.


I believe the bounds check, in particular, is devastating for autovectorization. There are ways around it, but it requires additional code in safe rust.

Edit: actually looks like autovectorization is in play here [1]. Doesn't look like the bounds check gets in the way at all.

[1] https://godbolt.org/z/af4qGba5o


There's no autovectorization there; scalar f64-s just are always stored in xmm registers. And the bounds check is still there.


Compress patterns aren't recognized by any open-source compiler autovectorizer as far as I'm aware of. (I think intel's proprietary C/C++ compiler can?)


Hey, could you share the prompt you're using for "teacher mode"?


Someone posted their AGENTS.md some time ago, that is used in practice to teach their students[1]. That might be useful to you in this context as well.

[1]: https://news.ycombinator.com/item?id=48359858


What parameter would you advise for min_p?



as temperature approaches infinity, min_p must approach 1 to stay coherent.

Assuming your temp is below 2, min_p of 0.1 is fine (and disable top_p and top_k). You can try 0.05 for more diversity.

Remember that subsequent methods are better, min_p is a mid-tier sampler that just happens to be the best implemented in most inference providers right now.

Also I'm the author of the "conspiracy against high temperature sampling" thing that selfhoster posted in the comments, so you can ask any questions about that piece you want.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: