As a newcomer to Rust and perhaps having been spoiled by Go's very readable specification, I was surprised that Rust doesn't have a definitive language spec. How do the people working on Rust ensure everyone has a common understanding of the language without one?
Some stuff is quite nailed down, some stuff is not. In the end, not breaking existing code is the most important thing.
These things are always on a spectrum. Just because a spec exists doesn’t mean that it has holes; Go’s spec isn’t formally proven, for example, so you could make a similar claim: how can people know that it all works without a proof?
The answer is that it’s all a spectrum. Many languages don’t have anything resembling a spec at all!
(We are interested and actively working on such a thing for Rust, but we’re shooting high. It’s gonna take a while.)
I have many times run into issues where a written down language spec would have been a big help. The are very useful features in the language that aren't mentioned anywhere in The Book, discovering them requires hoping someone on twitter mentions it, which is crazy.
I would love to see a feature freeze and focus on writing down a spec and speeding up the compiler.
Some things that are covered are a bit hard to find. For example, I didn't know if operator overloading was covered; it turns out, it's mentioned in https://doc.rust-lang.org/book/ch19-03-advanced-traits.html, but it's not in the outline and there's no table of contents on that page, and it's only actually mentioned as an example of how default generic type parameters work. The topic it is covered well by the `std::ops` docs (https://doc.rust-lang.org/std/ops/index.html), but since it's a language feature as well as a library feature, I'd expect it to be covered by the book as well.
#[path = "path/to/file.rs"] doesn't seem to be mentioned; there are probably a bunch of other obscure attributes like this that aren't mentioned.
repr? I didn't find any mention of #[repr(C)], #[repr(packed)], #[repr(align)] etc. I also didn't find union.
simd? Seems to only really be covered by the 2018 edition guide. Googling "rust simd" turns up references to the old, unstable std::simd.
The first two examples are a couple of things I've recently looked for and couldn't find; the others are ones that I noticed browsing around just now looking for other things that might have been missed.
Now, some of these are documented in the reference, in the nomicon, in the 2018 edition guide, or in the standard library docs. But that's one of the problems; that there's no one, or even two or three, places you can go to find everything you need.
The reference has large warnings at the top about being incomplete, although it's actually not as incomplete as I remember, it might be a good idea to remove the scary warning from it, or to do an audit of it and only put the warning on the sections that are found to be incomplete (like the memory model, which is clearly incomplete as it hasn't actually been formalized yet), but take it off of the sections that actually are up to date.
And Google doesn't always help. It still sometimes gives me first edition book links at the top. When I was trying to look up all of the `#[cfg(target_...)]` forms I could use, I tried to search for "rust cfg target", https://www.google.com/search?q=rust+cfg+target, but that gives me Rust By Example, a first edition link, and a link to a placeholder that refers to both the first edition and the reference. It took a lot of clicking through to actually find the reference page on it.
Thanks! So I think that some of this difference is that you're mostly describing library features, not language features. It's also true that some of these examples blur the lines. I'm going to make some notes.
> it might be a good idea to remove the scary warning from it, or to do an audit of it and only put the warning on the sections that are found to be incomplete
Completeness is hard, as you've seen with the book! This is what we'd like to do, but it's non-trivial.
I'd say that most of these are language features. The only that are mostly library features are operator overloading and simd, but they both have associated language features supporting them.
I agree that completeness is hard, but I feel like it could be a bit easier if there were some consolidation of the material.
At the very least, make sure that everything covered in the 2018 edition guide is also covered in the Book, in at least a similar level of detail. I'd also make sure that everything covered in the Reference is covered in the Book, or the Book provides at least an introduction to the topic and then a reference to the more detailed information in the Reference.
> I'd also make sure that everything covered in the Reference is covered in the Book,
They have very different goals, so this will never happen. The Book's job is to teach you how to program in Rust. The referece's goal is to (eventually) be a full specification. The former must be a sub-set of the latter.
I do agree that everything should be documented, but the book cannot cover every last possible thing.
Yes, that's why I included the second part, "or the Book provides at least an introduction to the topic and then a reference to the more detailed information in the Reference."
For example, I don't think that every last detail of #[repr(...)] needs to be covered in the Book; but as the entry point for most people, it's how they discover features, so it should at least provide an introduction, and link to the reference for more information.
Likewise, while I think that the std::ops documentation does a good job of covering all of the details of operator overloading, I think the Book could provide a slightly more discoverable introduction to operator overloading which isn't just being used as an example of another feature, default generic type parameters, and provide some links to std::ops which might help with the Googleability problem.
One thing that I think really needs to be focused on is some SEO for the docs. For some reason, the way the Rust docs have been organized over the years, with old versions of docs and the old book being replaced by the new one, means that a lot of times, when I search for something I find old, out of date docs.
For instance, with operator overloading, if I search for "rust operator overloading", I find Rust by Example, which is OK but has a lot of text in comments which is harder to read, followed by a first edition link which warns about being out of date and then provides a link to the new book but not where in it to look, as well as a link to the 1.30.0 version of the first edition book, then a link to std::ops (which if I were a newcomer, I might not realize was relevant), then a 1.5.0 edition of the book.
The first edition operator overloading chapter seems pretty reasonable, actually, and it's present in the table of contents so its more discoverable, and it contains a link to `std::ops`. But because of the new structure of the second edition book, the first page I see if I try to check the book tells me that it's outdated, and directs me to the new book in which this material is harder to find.
And in the new book, even once I find it, there's not much to direct me to where to learn more about the topic. There's no link to `std::ops` documentation; I have to separately look that up.
Part of the problem was that the first edition book was a bit more like a more accessible reference, while the second edition has a more narrative or tutorial structure. So there's no longer a place to go for an accessible reference to all of the language features. There's the Reference, but that's written for detail and precision and not accessibility, and as discussed, is incomplete. There's Rust by Example, but that's mostly just code examples with commentary in comments. The second edition book covers a lot of the same material as the first edition, but because of the different structure, some of the "reference information" on more advanced features is just buried in a couple of "Advanced X" chapters.
I'm not sure of the best way to address this; I admit that getting this right isn't easy.
One option might be to have the new book be in two halves; one of which was the narrative/tutorial portion; chapters 1 through 18, and 20, perhaps, though they might be able to be slimmed down a bit and some material moved to the second portion. The other half which is more of an "accessible reference" for advanced features; the material in chapter 19, but expanded a bit and with more discoverable per-topic chapters like the "Syntax and Semantics" section of the first book, plus the Nomicon material, and any of the topics which are missing.
Or, alternatively, just expanding on and integrating the "advanced X" material, Nomicon, and missing topics into the overall structure of the book.
Or another option would be to have the Book just be the Book focusing on the main narrative/tutorial introduction to the most common features, and have a Reference in two portions; one longer one which covers every major feature but in an accessible style with examples, followed by a more formal appendix covering each last little detail.
I guess ultimately what I'm saying (sorry for taking so long and so many diversions to say it, but as you point out, it is hard to get this right, and I've been thinking about what I'd be looking for as I write), is that there should be some place that has an "accessible reference" to all major features which is organized somewhat like the "syntax and semantics" section of the first edition book. The second edition book, chapters 1 through 18, is definitely a better organization as a book than the first edition, but that structure doesn't lend itself well to covering every feature, and as you point out, you wouldn't want it to.
But the Reference also isn't a friendly place to point people, both because of the "incomplete" warnings but also because it's written in a more formal style.
And the fragmentation into the Book, Nomicon, Rust by Example, stdlib docs, 2018 Edition Guide, Cookbook, Reference, Cargo Guide, Unstable Book, along with some of the reorgs along the way, can sometimes make it a bit hard to figure out the right place to look. I feel like there should be one document somewhere that has a reasonable accessible introduction, with examples, to every feature, in addition to the more formal and succinct Reference.
Maybe I should put my money where my mouth is and try to compile something like what I mean. It would help to have a little bit of buy in from docs team leadership, though, so that there would be a hope of it getting merged and maintained. Perhaps a docs RFC would be appropriate?
Compare with Go's language spec, which is written in a formal (but readable) style with little math. I wouldn't want it to have examples because it's good that it's concise.
One way to start might be to have an official extended cheat sheet. A cheat sheet needn't explain everything, but it should list every language feature available. The idea is to give you a definitive "inventory" of everything available so you can search for answers elsewhere if you see something you don't understand. It should also be brief enough for the language maintainers to commit to updating it when they enable new features.
It is actually fairly comparable to the Rust Reference (https://doc.rust-lang.org/reference/introduction.html), though the way the Rust Reference is divided into chapters makes it a bit hard to see everything at once or search.
They both do contain some examples, as well as things like the syntax in a BNF like format.
Perhaps what I'm looking for is just to have the Rust Reference made a bit more complete and a little more accessible in format (the division into chapters doesn't really seem to help much), as well as improving the SEO since it doesn't show up in Google searches often.
> Go’s spec isn’t formally proven, for example, so you could make a similar claim: how can people know that it all works without a proof?
that's moving the goalposts. let's see Rust's language spec finalized first, then we can talk about it being proven. without the two you shouldn't throw stones.
There are no stones being thrown and no goalposts moving. Go's specification is clearly in a much more advanced and useful state than Rust's. The point is to respond to "how can people know." That is, it is a measure of degree, not binary.
I actually think a mathematical-flavored spec where a proof would be meaningful would be a bad idea to the extent that it makes the spec less readable by ordinary users. For example, Dart has a more mathematically flavored spec and it's not readable by most people.
Of course, formalisms can still be useful, but as a matter of writing style, maybe it's better to put that sort of thing in an appendix? (As sometimes done for grammars.)
I would rather have a formal spec as the definitive one. Deriving a formal spec from a readable one seems to hit ambiguous cases all the time. Just look at publications that do that for C or Java.
With a formal spec you could also derive an implementation automatically which is a great useable reference. Look up the K framework for one possibility.
One can generate an informal spec from a formal spec.
One can NOT generate a formal spec from an informal spec (or else that “informal” spec would actually be formal, after all).
So, a formal spec is strictly better than an informal one — it enables all the benefits of an informal spec, via the ability to generate any number of informal specs from it in many human languages, cultures, levels of detail, etc., and of course enables things like compiler reproducibility (which you cannot do at all without a formal spec).
That being said, any spec is probably better than no spec.
Since many programmers are not mathematicians, a formal spec will always have a smaller audience than a well-written spec written in English. You cannot automatically derive good writing from pure mathematics.
As a result, neither is strictly better than the other. They have different audiences and serve different purposes. The audience for pure mathematics is quite small.
I don't use Go, but I thought of it while reading this post too. I respect their well-documented restraint [1] and also their documentation. I can't think of another language that has both users and restraint, to that degree :)
In order to do this, there also needs to be an unambiguous formal semantics of Rust, so one might even hope that this will turn into some form of official language specification.
I think a full, accepted specification that is adjusted in line with design decisions is still a bit out. Because as soon as you have a blessed specification, people and tools and such are going to depend on it, which limits the ability to change and break the spec down the line.
I don't think Rust is stable enough yet for a real specification.
