Yes, it’s unusual to get an email with more than a couple dozen attachments, but that’s not for the parser to decide. Hard-coding a limit like is fundamentally the wrong design. A library should be able to parse anything you throw at it, and if that’s an email with 10,000 attachments, it should parse it correctly. Sooner or later, it’ll run out of memory, of course, but that’s my problem.
A better design would’ve been something passing the limit as a configuration object, something like:
Eh, there's a limit where it is not reasonable to continue parsing. I worked on the mailer for a huge webmail service, and it will at a certain point stop considering your inputs and respond with SMTP 5xx. Its just about knowing the limits of your software. If you knowingly write a recursive descent MIME parser then you have to also reason about the limits of how many stack frames you are permitted, or whatever limitations your language and runtime have. It's not OK to write such a parser and then just obliviously accept all inputs.
We had a similar problem at my last company, where out boss sent a mail with his trip photos to the access limited everyone@ list - so our mailserver had a good 2 hours downtime while filling 2k email boxes with 20MB of attachments...what a mess
There are things that the parser must necessarily decide because nobody higher up the stack will understand them or think to limit them and override the default, even if a configuration object were to be exposed.
For example, one variant of this kind of MIME bomb would be defeated by a limit on the number of RFC 2231 parameter continuations acceptable per header. Even if you were to allow the user to override the default limit as you propose, your library would still need a sane default, because you would probably find that not many people would see RFC 2231 parameter continuations as a threat vector or even think to specify a limit, let alone know what a sane limit would be.
In the end, to be secure, you would have to expose so many limits that just using the library would be a mess.
As soon as you say "that's not for the parser to decide" you run the risk of things falling through the gap and no policy decisions being made at all.
You also have to accept that sometimes the parser designer alone knows best the complexity analysis of the resources the library will need.
Depends on what your goals are as an API designer. If you want a fast and scalable approach, your suggestion would indeed be most appropriate. On the other hand, most users are likely best served with a parser that just returns a single object, which they then can interact with as they desire.
Could that not be easily done as an add on that utilizes the callback approach? The parser library could offer both APIs and the caller could decide which to use and what to do when running out of memory etc.
I wonder if the lazy nature of Haskell's Attoparsec parsing library would have prevented the issue by default (without needing to set-up a limit in the parser). I think it depends on how the resulting lazy structure is used, but it might be true that the person writing the parser may not need to worry in the least about it, since evaluation and resulting memory allocation would be up to the person that uses the result from the parser because of the language's lazy-by-default nature.
IT support friend of mine told me a story how the place where he worked (major US bank) basically took its whole email infrastructure offline when the company essentially DDOSed itself with a single email.
Someone in IT was testing an email distribution list for the whole company (CEO announcements, that sort of thing) and sent out a test email to the effect of “Hello, did this work?”.
Two problems.
1. They had emailed the whole company for real, not the test list they thought they emailed
2. The list was setup such that anyone in the company could email it and also send a message to all other employees in the bank.
Within about 5 minutes everyone in the bank had hundreds of emails turning up. Initially this was people saying “Yes I got it”. Before long people were emailing to say “Please stop sending all these emails” which in turn got sent to the whole company.
Of course some people then started replying all to tell people to stop replying which only added fuel to the fire. The exponential traffic growth effectively brought the email system to halt for many hours until the mess could be cleaned up.
The main problem I always see to that is the fact that reply all doesn't have an confirmation or isn't the default.
I had similar issues back at an large company, some users replying 1k people sharing an unique link to an internal questionnaire because someone before them replied the original email asking for the link.
That and permissions, never let something like this be accessed by end users.
IMO, the problem starts with sending an email to thousands of recipients (or groups containing thousands) in the “To:” or “CC:” field.
I cannot think of cases where that is the right thing to do. If you want to host a discussion with thousands of people in real life, you let people send in questions beforehand, manually filter and group them, then hold a panel discussion between a selected representative group (or something similar)
Discussions over email are harder to do than face-to-face ones, so, at the least, that’s what you should do with email to.
If, on the other hand, you want to inform thousands of people, using BCC is fine.
If you want to make sure everybody knows that a message is sent to thousands, start the email with “this message was sent to everybody in the foo, bar, and baz departments” or something similar.
We had a similar problem at my last company, where out boss sent a mail with his trip photos to the access limited everyone@ list - so our mailserver had a good 2 hours downtime while filling 2k email boxes with 20MB of attachments...what a mess
This isn't really a garbage collection problem. Any runtime using the same basic code would allocate system-crushing amounts of RAM, which is a problem on its own regardless. This will cause other problems for other memory allocation techniques, such as this being a great recipe for fragmenting all of your RAM, and still having a fairly slow deallocation process due to sheer number of objects being churned.
The real problem here is that where a streaming parse is called for, a parser that fully manifests all the objects in RAM was used. Streaming parsers remain relatively rare.
(It's actually a legit advantage of the XML library infrastructure; they're readily available there, though typically you'll be using them directly because the libraries that build further layers of abstraction on parsing often concretely manifest everything in RAM anyhow....)
Ever since my first IT job back in 2004 setting up spam filtering I've been told that any e-mail server can be crashed. Executing a DoS on an e-mail service is among the simplest kind of remote DoS you can do.
And this is about new nodejs libraries that aren't even mature yet. I'm talking about actual mail services based on Postfix and Exim.
Last time I migrated a Sendmail setup to Postfix back in 2013 they actually shutdown port 25 when receiving too many connections. That's funny. An MTA shutting down its main port because it can't handle the connections, while postfix replacing it can handle it just fine.
The article is from 2018, so many things in the article that say "ALL VERSIONS" are inaccurate.
For example, I believe this link is the patch for v2.3.0 of mailparser <https://github.com/nodemailer/mailparser/compare/v2.2.0..v2...., which resolves this issue.
I've often wondered if zip bombs are sometimes used in email attacks. I imagine they could take down some virus/attachment scanners if the devs have not anticipated them.
This would have been in early 2000's, when all the major AV products - particularly those doing mail gateway scanning, were vulnerable to them. At first it was a zip attachment, expanding to several GBs. That got patched. Then it was bzip2. That got patched. By the fourth iteration, the companies finally caught up with what was being done and took active countermeasures in their software to protect themselves.
One of the first ones I remember was to limit the unpack memory and if the attachment was getting allocated too much, give up. If I recall correctly, it took a couple of weeks before malware authors came up with the idea of padding their executable payloads with a few tens of megs of leading NOPs...
Yes, they are still a thing, especially with David Fifield's variant, but even with simpler zip bombs. You would be amazed at what can crash an antivirus engine.
A better design would’ve been something passing the limit as a configuration object, something like: