Aww, this is really disappointing to hear but an excellent read nonetheless. I always had Xi in the corner of my eye as a very interesting way to make a text editor that had a lot of good things going for it: modularity, native UI, speed…I think if it worked out it would have been really great. It's really sad to hear that it didn't. Thank you, Raph and the rest of the Xi contributors, for working on it for all these years; even if it didn't upend the text editor market I am appreciative of the effort and the lessons you've learned along the way.
Some other side conversations I've had about this that might be interesting and relevant:
* Native UI is not always the right choice. That might seem surprising coming from someone who pushes native UI extremely strongly in general, and I don't want to be misunderstood as recommending you choose something else for your next project (if you have to ask: don't). But in some very specific contexts, such as performant text editing/rendering, it can be problematic to use a high-level control. In fact, many of the text views I use in are not using the standard platform controls but a custom view doing its own drawing: Terminal.app does its own thing (it doesn't even use NSScrollView!), Xcode has the Pegasus editor written in Swift, iTerm draws text using Metal, and Sublime Text is Skia. Platform controls are meant for 99% of apps that don't need absolute control over performance or text rendering, and sometimes you can push them quite far. But in some cases it's appropriate to abandon them. (I will note that all of the examples handle actual text editing very, very well. If you don't know what the standard shortcuts for editing text are for my platform, you're not going to do it right.)
* I love that LSP exists and that it makes my "dumb" editor "smart". But having worked with it a bit, I think it might suffer from some of the same problems: it uses a client-server architecture and JSON to communicate, which means it is not all that performant and needs to keep track of asynchronous state in a coherent way. And not just the state of one document, but the states of many documents, all in different contexts and workspaces, all with their own compiler flags and completion contexts and cursor locations–it is really hard to do this correctly and quickly, although many language servers do an admirably good job. Perhaps it's just "good enough" that it's here to stay, or maybe it has enough support that it can have some of its issues wrinkled out with time.
I agree with you that getting shortcuts (and other similar aspects) right is more important than which code is used to implement text editing. We're very much going to try to take that to heart as we implement more text capabilities in Druid.
Regarding LSP. Yes, while the functionality is there, it is very difficult to implement a really polished user experience with it. I think a lot of that is in the design of the protocol itself; it's hard to avoid annotating the wrong region when there are concurrent edits. It's interesting to think about how to do better, but I fear you'd be bringing in a lot of complexity, especially if you started doing some form of Operational Transformation. I would highly recommend that if someone were to take this on, they study xi in detail, both for what it got right and what was difficult.
I want to echo this, it's disappointing, but I'm thankful to Raph and the team for all the work they put into it. I had dreams of being able to write an editor where I could concentrate on making a cool UI in a different language/framework and have Xi do all the hard stuff.
I think LSP will end up being the "good enough" solution for niche languages, while popular languages get dedicated IDEs/mega-plugins, a la the various language flavours of IDEA, XCode, etc.
One important thing to note is that all your examples of custom text are non-rich editors. Text editing becomes astonishingly more difficult if proper rich text and i18n is required. Then, Core Text / TextKit is a pretty good solution because anything else quickly requires years of work. The best alternative I know of is using a Javascript-based editor in a WebView - or jumping to something like Qt.
Concerning LSP I can assure you that my NeoVim with native lsp support is great. Maybe because the situation before was just downright very bad so what we have now is just so much better.
Some other side conversations I've had about this that might be interesting and relevant:
* Native UI is not always the right choice. That might seem surprising coming from someone who pushes native UI extremely strongly in general, and I don't want to be misunderstood as recommending you choose something else for your next project (if you have to ask: don't). But in some very specific contexts, such as performant text editing/rendering, it can be problematic to use a high-level control. In fact, many of the text views I use in are not using the standard platform controls but a custom view doing its own drawing: Terminal.app does its own thing (it doesn't even use NSScrollView!), Xcode has the Pegasus editor written in Swift, iTerm draws text using Metal, and Sublime Text is Skia. Platform controls are meant for 99% of apps that don't need absolute control over performance or text rendering, and sometimes you can push them quite far. But in some cases it's appropriate to abandon them. (I will note that all of the examples handle actual text editing very, very well. If you don't know what the standard shortcuts for editing text are for my platform, you're not going to do it right.)
* I love that LSP exists and that it makes my "dumb" editor "smart". But having worked with it a bit, I think it might suffer from some of the same problems: it uses a client-server architecture and JSON to communicate, which means it is not all that performant and needs to keep track of asynchronous state in a coherent way. And not just the state of one document, but the states of many documents, all in different contexts and workspaces, all with their own compiler flags and completion contexts and cursor locations–it is really hard to do this correctly and quickly, although many language servers do an admirably good job. Perhaps it's just "good enough" that it's here to stay, or maybe it has enough support that it can have some of its issues wrinkled out with time.