anyone thinking about hanging their hat on writing decentralized, local first, crdt-driven software should also consider the complexity of changing your schema when there are potentially thousands of clients with different opinions of what that schema should be. some interesting research has been done[1] in this domain but i haven't seen any library of this ilk that supports real-world schema evolution in a way doesn't make me really, really wish i had a proper backend to lay down the law. the fact that migrations are in a "coming soon" state makes me wary of using jazz, and i wonder how they would approach this problem with the library as it is now.
It’s a tough problem! I think 90% of it is solved with GraphQL/ protobuf like rules (“only ever introduce new fields”)
There are some edge cases where you might loose easy access to some data if two clients migrate concurrently, but we’re hoping to provide patterns to mitigate these
Edit: Right now it all depends on you to implement migrations “the right way” but we hope to provide guardrails soon
I've been thinking about this as well and wondering if one possible approach to avoid the eventual messy database state—where fields are "append-only" and never deleted—might be to include the up/down migration logic as part of the payload.
This approach may require a central authority (with no access to user data) responsible solely for providing the schema and migration patterns as code transformations.
Since running arbitrary code from the payload introduces potential security risks, the migration code could be cryptographically signed, ensuring that only valid, trusted transformation code is executed.
A possible additional security layer would be to have the transformation code execute in a sandbox which can only output JSON data. (keeping a possible full before-migration version as backup in case something went wrong would always be a good idea)
Another option would be to use a transformation library for migrations, but in this case, the approach would only describe (as JSON) the functions and parameters needed to transition the schema from one version to another.
> I think 90% of it is solved with GraphQL/ protobuf like rules (“only ever introduce new fields”)
Agreed, that’s the only
sensible thing to do. Not sure it’s 90% though.
> but we’re hoping to provide patterns to mitigate these
Hope is not confidence inspiring for the most difficult problem at the heart of the system. That doesn’t mean it has to be impossible, but it needs to be taken seriously and not an afterthought.
Another thing you have to think about is what happens when data of a new schema is sent to a client on an older schema. Does the “merging” work with unknown fields? Does it ignore and drop them? Or do you enforce clients are up to date in some way so that you don’t have new-data-old-software?
As a former long-time Ableton user, just want to recommend Bitwig Studio[1] as a viable (improved?) alternative that runs natively on Linux. I've used it for live performances and label-released music for over four years now and other than the typical setup issues related to Jack (one time thing resolved in an hour or so) I couldn't be happier.
>>other than the typical setup issues related to Jack (one time thing resolved in an hour or so)
I was never able to get Jack up and running on my MBP. I tried both Jack and Jack2, compiling from source, using their prepackaged binaries, using the cli vs the GUI, etc, .. nothing doing. Do you have any resources you can share detailing how you resolved the issue or how you configure it? Feel free to PM, emails in profile.
I'm really tempted here. I've got a paid Ableton license, and I absolutely love Ableton, but it's the main thing keeping me from dumping Windows 10 for Linux.
Transitioning between Ableton and Bitwig is seamless from a UI standpoint. They have essentially the same controls.
The only thing that isn't great moving from Windows to Linux for production is your VST ecosystem becomes much more limited. Running plugins under WINE is alright, but if they're heavy you'll run into latency issues. There's a sizable collection of plugins that are native under Linux though[0]. I've used many of these with great success.
You can get pretty far with the built in instruments and effects in Bitwig, especially with the new modular grid. I haven't felt much need for third party stuff.
I've used Elm professionally for about a year, having done full rewrites of both a bloated React/Redux SPA and a map data visualizer with complex JS interop using ports to Leaflet, AWS Cognito, etc. Luke Plant's blog post doesn't at all reflect my experiences with Elm or my feelings about the core team.
For those not privy to the Elm tea, a brief primer: this blog post is primarily concerning a months-old issue regarding the removal of the ability to use native modules, which are effectively patches to the Elm runtime[1] which circumvent the core features of the language that provide its greatest strengths: a genuinely helpful compiler and ironclad runtime guarantees (still have yet to encounter a production runtime error that wasn't on the JavaScript side of things!) This "feature" (used loosely) was largely undocumented, always verboten from distribution in user packages, and never intended as anything more than a stopgap measure in extremely rare cases.[2] It was /officially/ not a core language feature intended for widespread usage, and consistently advised against by the Elm team. The removal of native modules was spoken about publicly months before the breaking upgrade. It came as no surprise to me. As an engineer who tries really hard to be responsible, I do not build software which relies on features which are advertised as 'do not use unless absolutely necessary' and are soon-to-be deprecated unless I'm willing to accept the inherent risk of doing so.
The core Elm team was, in my opinion, extremely communicative, well-reasoned, and thoughtful in this change and in others, despite the inconvenience to a small subset of people. For each feature discussed in Luke's post, there is an accompanying Discourse thread[3], discussion on Google groups, Github gists[4], etc., that carefully lays out the reasoning, and carefully consider the scope of a change (in the instance of the removal of custom operators, the Elm team analyzed the package ecosystem and determined less than 5% of packages were affected)[4].
I won't cover the moral arguments around the obligations of open source maintainers as this has been covered ad nauseam with the Clojure and Rust fiascos of a similar sort - though I personally believe maintainers don't owe anyone anything, and I try to treat all free software graciously, as a precious gift, lucky to receive it at all. I will say that Elm's somewhat slow, closed BDFL-ish governance is well-documented as well[5, 6], available to all who seek to understand the Elm development process and decide for themselves if this is an ecosystem to hitch their wagon to. Of course the Elm maintainers get to patch their own runtime with native modules, because they are implementing core parts of the language, it's practically tautological. That users living in userland cannot do it is not unfair; it's a language design feature. What Luke seems to see as a stifling of an open community by the closure of issues and deletion of posts on the Discourse is often basic organizational maintenance to handle redundancies. Conversations around these issues have happened for months and in some cases years, and decisions have been made. They may not be to everyone's tastes. That should be just fine! If someone went to Famous Amos Cookies on Discourse, Slack, their mailing list, and opened a Github issue and PR suggesting this genius idea they just had, it's so good, wait for it - an oatmeal cookie without raisins! - I hope they would clean it up, close the issue, delete my posts, etc., for their own sanity. Not to mention that the opening to Luke's opus here is an admission of his own rudeness to the Elm team on most of these fora. Of course the relationship with maintainers will be strained with this kind of behavior.
There are critiques of Elm, to be sure. The pace of releases is somewhat slow, custom operators might be nice for 3rd party parsing libraries, it might be cool to have a PR merged in to get that fuzzy feeling only OSS contributors get. But you cannot reasonably argue that Elm's team has been unfair, discriminatory, uncommunicative, or arrogant (pathos, so much pathos here!). I don't generally post comments anywhere, but articles like this are irresponsible and damaging not just to the well-being of maintainers who are being generous with their time trying to make something with great care, but to people (like many top level commenters here) who might have tried Elm but won't due to an inside baseball post from a spurned developer. It makes me actually sad, like want-to-cry sad.
I have been using elm at work (in prod) for about three years, my team and I have rewritten a hard to maintain 50kloc+ angular application to elm. The rewrite started with elm 0.18 and was finished with 0.19.
We started using elm extensively about a year before 0.19 was released and it was very clear to us from the start that even though it's possible to use native modules, it's a feature that should not be used and will not be supported in the future.
The experience we have had with elm and it's community have been nothing but great.
If we had stumbled upon a blog post/ thread like this when figuring out how to proceed, I'm not sure we would have given elm enough thought and tested it out properly.
This makes me really sad as well.
I am only a hobbyist user of Elm, but the original post also made me sad.
Here's an interview from 2017 with Evan where he discusses Elm's approach to interop.
https://elmtown.simplecast.fm/b06499a6
Skip to about 8.5 minutes in.
While Luke's approach certainly could have been better, I don't think it had any bearing on the result. The reason for not allowing Javascript into Elm libraries seems fundamental to what Elm is trying to achieve.
If this was a serious project, you wouldn't irreversibly break userland. If you're going to call it kernel modules, compare yourself to the Linux kernel, where breaking ANY userland is forbidden.
You can call out to JS using ports and custom elements and accomplish nearly everything you could possibly need. This blog post is focusing on a largely undocumented, always discouraged "feature" (used loosely, as even from its inception it was verboten, couldn't be distributed via the official package manager, etc.) which allowed users to hack library support directly into the Elm language while risking (and losing!) many of the guarantees Elm provides. Whether it's a good fit for your startup is another question, but I just wanted to assuage that concern.
This is untrue--and I'm not entirely sure you followed the article. You can't do synchronous tasks like Intl's format for strings from floats as many locales use "," over ".". It is a pure function `Float -> String`. This should not have to go through the entire update cycle to handle receiving these kinds of value. Using ports adds async to this equation which means you need to write the ports, and handle and store all temporary values which tightly couples and makes a mess everything.
Conceptually it's `Float -> String` once the Intl object is instantiated, but you can't use it in Elm anyhow. You'd have to reimplement and ship the entire in-browser Intl spec.
[1] https://www.inkandswitch.com/cambria/