I'm one of those people. I'm not sure I'd keep loving Ruby as much as I do now if the majority of Ruby developers end up using the type checker and I'll also have to. I like Ruby in part because it's strongly typed but I don't have to lose time writing types. That was very welcome coming from C and Java in 2006. I prefer to lose time when I pass a type that I shouldn't have passed. It doesn't happen often (not every year) vs at every single method definition.
To understand if this project could have a future outside Stripe, how popular is Python 3.6's optional static type declaration?
> I like Ruby in part because it's strongly typed but I don't have to lose time writing types.
That’s why type inference exists. Static typing doesn’t imply annotating everything manually with types. Granted, most static type systems require (or at least encourage) this but it’s not necessary everywhere (or indeed for most usage). Annotating public interfaces with types is, I’d argue strongly, already best practice everywhere (in languages that don’t support it in the language, it’s done in documentation).
> To understand if this project could have a future outside Stripe, how popular is Python 3.6's optional static type declaration?
Or rather: How popular are TypeScript or Flow? Very.
I'm not a typing person, similar to pmontra. Any programming language where print("Hello world") is longer than print("Hello world") bothers me.
And most of my JavaScript is untyped, because (and apparently this is common, someone did some stats on it a while ago) typing errors aren't a great source of errors.
However I love optional typing. Being able to build a thing quickly, and then add those guarantees if/when you feel they're needed is great. My current codebase is about 80 modules. Nearly all are untyped JS. The database code is TypeScript because that's where I feel type handling would add the greatest benefit - not that I have had typing errors, just that I like the additional peace of mind.
That’s not true. On the contrary, typing accounts for a substantial fraction of all errors (and refactoring could transform even more logic errors into catchable type errors). The best research (best methodology, sample size, by far) on the subject [1] puts the lower bound for the fraction of type errors in JavaScript at 15%.
Not doing a URL hunt as I'm at work, but as mentioned in the comment you're replying to (literally after you chopped it off), there was a large study showing the opposite on HN a while back.
> there was a large study showing the opposite on HN a while back.
Yes, sorry: I didn’t mean to ignore that part of your answer but the research I’ve linked to essentially supersedes everything else in that area. Prior studies were riddled with inadequacies in controlling for confounders and outright methodological errors. I’m assuming you’re referring to [1] and while that was an impressive study in its own right, their classification of bug sources is indirect and, ultimately, simply didn’t allow any assertions regarding the prevalence of type errors.
You'll have to justify this: "Prior studies were riddled with inadequacies in controlling for confounders and outright methodological errors."
Your recent study is pay walled, so I can only go by the abstract, but it doesn't sound any more rigorous to me.
I also find its comparing very different things. It says bugs in release, but it sounds like they consider commits to be releases, which would be very unfortunate.
Its also specifically targeting JavaScript, which prior study already showed to be one of the poorest performers in terms of defects (i.e., it tends to have more). For example, it would be interesting to know how many bugs were due to weak typing.
I'm also curious, adding the typed annotations should be a blind activity to be rigorous. Was it? Or did they knew there was a bug and what the bug was then they added the type info?
I also feel they're doing a one way comparison, having to add types can introduce other forms of bugs, but since this study is targeted, I find it to suffer a little from confirmation bias. Like type checker detects type errors, is a bit of a no brainer conclusion. I understand they mean this in the sense that 15% of bugs were type errors. But that doesn't mean TypeScript code would have 15% less bugs, because they did it one way only. It could introduce other bugs in the process.
Would have been interesting too to analyse how long it took for the bug fix commit to be made over the introduction of the bug. So you would know the real cost of them. Which is important, since annotating adds dev time, but so does big fixing. Would be nice to have time data.
Finally, the prior study demonstrate the same outcome. That static tends to have lower defect rate then dynamic and functional less then imperative. The difference is in the outliers. Clojure and Ruby were both outliers that outdid almost all static languages.
So actually, it would be great to reapply your study to them. Clojure has an optional type checker already so it could be a good one. It was also the strongest outlier.
> You'll have to justify this: "Prior studies were riddled with inadequacies in controlling for confounders and outright methodological errors."
That topic has filled whole blog posts. But in a nutshell, no previous study was able to compare directly the effect of typing disciplines due to confounders that they couldn’t control for in their experimental setup. For instance, virtually all previous studies compared different programming languages, which obviously differ by more than just their typing discipline. Many studies had very small sample sizes due to using human test subjects to write sample programs. These test subjects were almost exclusively students without real-world experience — or, in some case, any prior experience in the respective programming languages. Thus, many studies — from the outset — tested beginner-friendliness of languages rather than everyday use. Fair enough, but not the same as type checking benefits. Furthermore, most of these studies, for the same reason, were restricted to testing on artificial, academic, small toy programs rather than real-world applications.
The few studies that looked at big real-world data sets (the biggest and most rigorous being linked above) didn’t even look at typing discipline as an individual factor — again because they couldn’t regress it out as a single factor.
> Your recent study is pay walled
The link I posted has the full text PDF (that’s the whole reason for me to share Paperpile links rather than the original or DOI). See also an academic discussion [1] of the manuscript.
> Its also specifically targeting JavaScript
Which is explained in the paper: no other constellation allows to compare the effect of added static type checks as directly as JavaScript/TypeScript/Flow. That’s because (a) these languages only differ in their type checker, and are gradually typed, i.e. types can be added to just a subset of the program, which enabled the study methodology. And (b) there’s an extensive database of real-world code to analyse.
> For example, it would be interesting to know how many bugs were due to weak typing.
Fair enough. I’d expect the effect to be (much) less pronounced in a language with stronger type guarantees. Then again, the 15% number is absolutely an underestimate to begin with (see the paper and the blog post [1]).
> adding the typed annotations should be a blind activity to be rigorous
That’s not at all obvious, and from experience I disagree. Thinking about types is automatic neither in static nor in dynamic languages. Once you’ve figured out the correct type, yes, it’s a blind activity … but that’s a pretty empty statement.
> having to add types can introduce other forms of bugs
Again, this claim is far from obvious, beyond the trivial “if I make this type an `int` even though it should be a `string` then that’s a bug”. Fair enough, but adding the type annotation and performing a type check merely reveals this bug. The bug itself was present in the programmer’s flawed assumptions about the invariants in the code. Your assertion is equivalent to saying “compiling or interpreting the code [rather than writing it on a piece of paper and never touching it] can introduce bugs”. — The bugs are already in the code, we just didn’t detect them.
> Would have been interesting too to analyse how long it took for the bug fix commit
They only considered bugs they could fix within a highly constrained time frame, and only looking at local code around the bug. Hence, again, why the 15% is an extreme underestimate. Furthermore,
> So you would know the real cost of them.
The study calculated the token cost of adding types (and provide a justification for this metric). In a nutshell, the cost is negligible, especially in languages with strong type inference capabilities (Flow outperforms TypeScript here).
> Clojure and Ruby were both outliers that outdid almost all static languages.
Right, and I’d expect the same to still hold. But it shows one of the crucial shortcomings of previous studies: they primarily did not examine the effect of static vs dynamic typing. Rather, they examined the effect of different programming languages, to which typing is just one contributor amongst many. It should come as no surprise to experienced programmers that some languages (regardless of static vs dynamic discipline) vastly outperform others. For me, this is all the more reason to be excited about the headline topic: If done well, statically type checked Ruby could be an amazing language, with the added benefit of static typing at (as shown) virtually no cost.
> If done well, statically type checked Ruby could be an amazing language, with the added benefit of static typing at (as shown) virtually no cost.
I think this is where I feel the jump in causation to be too high. No one knows why Ruby and Clojure match up to Scala and Haskell. The prior study couldn't isolate the impact of a particular feature, but it did a pretty good job, at least similar to your study in finding overall defect rates.
Ruby with types, if we take your conclusion, should end up even outperforming Haskell by almost 10% in overall defect rates. That just doesn't sound believable to me.
I fear types are not independent variables. They affect other variables. And that's where I argue that they might in fact come at some cost which we still do not understand.
I still strongly feel they should have added types without knowing the bug. There's many ways to type data, those choices do matter. There's code that can't be typed precisely, or requires considerable efforts to type, there's even code that can not be typed at all, and must be rewritten differently to be typed.
Now I'm not going to argue JavaScript doesn't benefit immensely from static type checking. What I argue is that doesn't mean every language will see benefits from static types.
That said, I'm a big fan of gradient typing, especially when it can add runtime contracts over boundaries between typed and untyped. But, I'm not sure how successful they would be in Ruby. They've failed in Clojure, while no real analyses was performed, the community rolled back most adoption of it due to their impression of it not providing any value while adding extra effort. Obviously, the Clojure gradient typing system isn't as well maintained as Flow and TypeScript, so maybe that played a role.
Ruby has the advantage of 'duck typing', due to its strong OOP foundations. Not quite sure what Python has that reduces the demand for static type declaration.
I think part of the popularity of TypeScript could be down to a large proportion of javascript coders that have a background/preference for static typing. It could also be that certain classes of bugs are more prone to happen in JavaScript or it's harder to debug when bugs do happen?
Ruby has been my first professional language and I've sticked with it for a decade, but I never really understood duck typing before switching to Go, with its interfaces (a type implements an interface if it implements its methods, so we can use interface as function parameter type).
I don't get why we talk of duck typing about ruby : if I pass as parameter an object that does not quack like a duck, nothing will prevent me to do it and it will generate an exception when quacking. Did I miss something obvious?
The obvious part is that Ruby does not have a distinction between "compile time" and "runtime". As such, if you want to catch those cases, you need tests.
Duck-typing can be boiled down to: The type of a parameter to a function is defined by the methods that will be called on it, not by its class.
E.g. if you have:
def foo source
dosomething(source.shift)
end
Then absent additional restrictions inferred from requirements of dosomething(), the class of "source" is irrelevant. The relevant type information is whether or not "source" responds to "shift".
A proper Ruby-ish type checker will need to be able to handle that, or it'll push people to write non-idiomatic Ruby.
I wonder if this would be a good opportunity to use a typing system that's more like Go's, where you don't need to explicitly declare that a class implements an interface, but instead any class that has the right methods will implement the right interfaces automatically.
(In your example the method could be:
def foo(source: Shiftable<?>)
dosomething(source.shift)
end
where `Shiftable` is an interface that's something like
I've digged a bit on the subject (better late than never), it seems that duck typing is seen in ruby as a guideline, rather than a language feature. Basically, it's about recommending to use `#respond_to?` rather than `#is_a?`, `#kind_of?` or `Class#===` in end user code.
Oh, I see, given the interpreter is written in strongly typed language, allowing to go without static typing is a feature by itself, and allowing to test for methods presence allows to do duck typing, thus it allows to say that ruby implements duck typing.
It makes more sense than talking of duck typing explicit usage by end users, given it's quite rare we assert proper received parameters extensively.
EDIT : which makes me realize we have a machine language without typing (binary), on top of which we use a language implementing strong typing (C), used to build a language without strong typing (ruby), on top of which Stripe is building strong typing. Making a definitive decision is hard :)
Strong typing is about objects having a definite type. In Ruby, you cannot cast a String to a Fixnum, an exception will be raised. But in C you may cast anything to anything really, and your program will believe you, whether it could possibly make sense or not...
> A proper Ruby-ish type checker will need to be able to handle that, or it'll push people to write non-idiomatic Ruby.
It probably won't be idiomatic (maybe just add interfaces?) but I think duck-typing is overrated anyway. Just because an object responds to a method doesn't mean it's going to do anything like what you expect. To take a contrived example, Array.Shift and Keyboard.Shift are probably unrelated methods.
You don't have that guarantee with interfaces either - you're trusting that the developer isn't lying to you. My experience is that I've more often run into developers that have artificially barred me from doing what I wanted by checking for a given class, than that I've accidentally passed in something that satisfies a given type but does something different.
> You don't have that guarantee with interfaces either
True, but you can give the interfaces/contracts proper names (or namespaces). To use the example in the parent comment, `Array` and `Keyboard` wouldn’t implement the same contract even if both have a `Shift` method.
In my experience I lose less time specifying that an argument is a string in Go than I do adding a test to handle the possibility that nil gets passed in Ruby.
i appreciate type checking so much more now than i used to, and i find that i disliked it most when i needed it most (i.e., i blanched at it more when i was a worse programmer, even more likely to make type errors than i am now).
there are categories of checks in application code (as well as whole categories of simple unit tests) that i just wouldn't have to write with a static type checker. additionally, as other comments have pointed out, it'd make it easier to automatically refactor things.
i started coming around on it with groovy a number of years ago (after sorta hating the type checker in java/c/c++ for years before that). i'd find myself writing a bunch of untyped code, but then i'd also find myself going back and specifying types as i wrote tests, because it was often easier to just specify the type than to write a test for the possibility of the wrong type coming in.
as someone who mostly writes ruby code these days, i'd love to see an optional type checker get added.
You switched from Java to Apache Groovy to lose the static type checking, then later stuck types in your Groovy code to make it act like Java. Sounds like Groovy's value-add to the ecosystem was providing the syntax for doing that without writing `Object` everywhere.
In C#, I write type names when I declare a type, when I create instance of a type, when I declare method argument or return type (which you would probably do even in a dynamic language as a comment - you don't want people to have to go through your code to figure it out themselves, right?) and when I declare a collection of that type. Out of all of these, only the last one seems as "wasting time", because if I declare a collection and instantly use it on the next line, the reader can probably infer what type I meant to use - but in all other cases, not writing types would make the reader's job harder, not easier.
Don't the same principles apply when you're working in a dynamic language?
To understand if this project could have a future outside Stripe, how popular is Python 3.6's optional static type declaration?