LLVM was originally designed for C like languages, until recently used the system linker (lld is a recent change), so any language that uses it for its backend tends to suffer from the same lengthy compilations as C and C++.
Unless they create their own intermediate language, so that they can do as much work as possible, before passing the bucket to LLVM.
Hence why Swift has SIL, Rust MIR, and so forth for other languages.
I don't think the use of the system linker is why Swift has SIL, nor Rust has MIR. It's true that one of the possible benefits of Rust's MIR is faster compilations, but it is fundamentally motivated by representing (and checking) semantic information that is unavailable at the LLVM level. Whatever linker LLVM happens to use is orthogonal to this.