In this demonstration they use a .docx with prompt injection hidden in an unreadable font size, but in the real world that would probably be unnecessary. You could upload a plain Markdown file somewhere and tell people it has a skill that will teach Claude how to negotiate their mortgage rate and plenty of people would download and use it without ever opening and reading the file. If anything you might be more successful this way, because a .md file feel less suspicious than a .docx.
> because a .md file feel less suspicious than a .docx
For a programmer?
I bet 99.9% people won't consider opening a .docx or .pdf 'unsafe.' Actually, an average white-collar workers will find .md much more suspicious because they don't know what it is while they work with .docx files every day.
Curl|bash isn't any less safe than installing from random a ppa, or a random npm or pip package. Or a random browser extension or anything. The problem is the random, not the shell script. If you don't trust it, don't install it. Also thinking that sudo is the big danger nowadays is also a red herring. Your personal files getting stolen or encrypted by ransomware is often worse than having to reinstall the OS.
It's not really different than downloading a .msi or .exe installer on Windows and running it. Or downloading a .pkg installer on macOS and running it (or running a program supplied in a .dmg). Or downloading a .deb or .rpm on Linux and running it.
It's all whether or not you trust the entity supplying the installer, be it your package manager or a third party.
At least with shell scripts, you have the opportunity to read it first if you want to.
Because everyone uses airgapped disposable micro VM's for everything, right? No one would be stupid or lazy enough to run them on their development laptop or production server, right? Right!?!
Maybe the good side-effect of LLM's will be to standardize better hygiene and put a nail in the coffin of using full-fat kitchen sink OS images for everything.
No, of course every reasonable developer works with a bag full of disposable e-vapes, each one used to run a single command on and then thrown into a portable furnace.
Adobe added embedded javascript to pdfs. Its an option to turn it off but its enabled by default. I turned mine off a long time back and never notice any problems but I don't use a lot of pdfs with interactive forms.
I have yet to see an exploit that can be performed with a .txt file. PDF files can have all sorts of interactive junk and nested files embedded in them - you can get really crazy in that format.
Mind you, that opinion isn't universal. For programmer and programmer-adjacent technically minded individuals, sure, but there are still places where a pdf for a resume over docx is considered "weird". For those in that bubble, which ostensibly this product targets, md files are what hackers who are going to steal my data use.
All PDF security can be stripped by freely available software in ways that allow subsequent modifications without restriction, except the kind of PDF security that requires an unavailable password to decrypt to view, but in that case viewing isn’t possible either.
Subsequent modifications would of course invalidate any digital signature you’ve applied, but that only matters if the recipient cares about your digital signature remaining valid.
Put another way, there’s no such thing as a true read-only PDF if the software necessary to circumvent the other PDF security restrictions is available on the recipient’s computer and if preserving the validity of your digital signature is not considered important.
But sure, it’s very possible to distribute a PDF that’s a lot more annoying to modify than your private source format. No disagreement there.
You think a recruiter will be a forensic security researcher? Having document level digital signature is enough for 99% of use cases. Most software that a consumer would have respects the signature and prevents any modifications. Sure, you could manually edit the PDF to remove the document signature security and hope that the embedded JavaScript check doesn’t execute…
Nothing that hard. When I had a technically similar need (for non-shady purposes unrelated to recruiting) I found easy installable free GUI software for Windows that worked just fine with a simple Google search. No specialist expertise needed.
Yes, most consumer software does respect what you say. But it’s easy for a minimally motivated consumer to obtain and use software which doesn’t.
However, the context we were discussing was neither a consumer nor a forensic security researcher, but a recruiter trying to do shady things with a resume. I don't expect them to be a specialist, but I do expect them to be able either to get the kind of software I just described with a security stripping feature, or else to have access to third-party software specifically targeting the recruiter market that will do the shady things - including to digitally signed PDFs like yours - without them having to know how it works.
GP attack vector was probably recruiter editing the CV to put their company name in some place and forward it to some client. They are lazy enough to not even copy-paste the CV.
What is this measure defending against (other than getting a job)? The recruiter can still extract the information in your signed PDF, and send their own marked-up version to the client in whatever format they like. Their request for a Word document is just to make that process easier. Many large companies even mandate that recruitment agencies strip all personally-identifiable information out of candidates' resumes[1], to eliminate the possibility of bias.
1: I wish they didn't, because my Github is way more interesting than my professional experience.
Once again demonstrating that everything comes at a cost. And yet people still believe in a free lunch. With the shit you get people to do because the label says AI I'm clearly in the wrong business.
People trust their browser nowadays, I'd expect the attack to be even easier if you just render the markdown in html, hiding the injection using plain old css text styling like in the docx but with many more possibilities.
You can even add a nice "copy to clipboard button" that copies something entirely different than what is shown, but it's unnecessary, and people who are more careful won't click that.
I will never stop being disappointed that we have an API to control the clipboard. There is no use of this that I have ever found beneficial as a user.
Possibly apocryphal quote from a Yosemite park ranger talking about the difficulty of designing a trash can that a bear can't open but a human can: "There is considerable overlap between the intelligence of the smartest bears and the dumbest tourists." - https://yro.slashdot.org/comments.pl?sid=191810&cid=15757347 (earliest instance of it I can find)
I don't really follow the analogy here to be honest.
The analogy is that AI is suppose to be able to do _What humans do_ but better.
But you also want AI to be more secure. To make it more secure, you'll have to prevent the user from doing things _they already do_.
Which is impossible. The current LLM AI/Agent race is a non-deterministic GIGO and will never be secure because it's fundamentally about mimicing humans who are absolutely not secure.
Probably referring to the rat's race between making trash cans hard for bears to tamper but usable for tourists.
The analogy is probably implying there is considerable overlap between the smartest average AI user and the dumbest computer-science-related professional. In this case, when it comes to, "what is this suspicious file?".
