Is Haskell more academic in nature or used heavily in Production environments?
Is there an application sweet spot/domain (say Artificial Intelligence/Machine Learning, etc) where it shines over using other languages (I am not talking about software/language architectural issues like type systems or such)?
I have no experience with Haskell but do use functional languages such as Erlang/Elixir on and off.
The topics are roughly sorted from greatest strengths to greatest weaknesses.
Each programming area will also be summarized by a single rating of either:
Best in class: the best experience in any language
Mature: suitable for most programmers
Immature: only acceptable for early-adopters
Bad: pretty unusable
The state of the Haskell ecosystem is that it is amazing at encouraging people to write libraries, especially libraries to assist in writing haskell code but amazingly bad at encouraging people to write actual programs, especially actual programs useful for something that isn't writing code.
Take out the programs that are for writing code, ghc, shellcheck, darcs etc. And what are you actually left with? git-annexe, pandoc, xmonad is hledger worth knowing about if you don't care what language it was written in? (Maybe it's amazing, first I've heard of it).
Whenever I bring this up, people find it upsetting. The Haskell community might need to get past both of those things. 1.0 was 27 years ago. So many fabulous programmers have gone down the road for entertainment or enlightenment so where are your damn apps if you're a language that doesn't suck for writing them, no really, where are they?
This is an excellent Haskell resource. Grades are given by both application domain (e.g., "web development") and development task (e.g., "testing" or "IDE Support"). Each of the categories has not only a grade but also list of relevant libraries and example projects.
This is a fantastic link & I love the discretely sorted topics with ratings (could be more granular but its a start). Is there a similar link for other PLs notably more pedestrian langs like Java/Python/C ?
We're using Haskell to produce an entire compute platform for building, composing, and running type-safe containerised micro-services for data science. This certainly touches on a lot of the areas that Haskell is traditionally known for being good at, e.g. building DSLs, type systems, interpreters, etc.
However, the work also includes a runtime platform that is more low-level, including building our own container system, talking to the Linux kernel, using cgroups/resource management, and distributed message passing - areas where languages such as Go have found a niche, and may be classified as Real World.
However for us, Haskell's type-safety, runtime performance, and extensive ecosystem has been a boon even in this domain. We effectively use it as an (incredibly powerful) general-purpose language here, and it's worked more than fine.
We're currently at around 15,000 lines of code with a team of 5 Haskellers, and it hasn't really been a problem regarding performance, understanding the codebase, or with newcomers to the team.
Any particular reason you're building your own container system instead of leveraging LXC or Docker?
For the massively parallel workloads you find in data science, it seems like you'd benefit a lot from the wealth of container orchestration tools around Docker (swarm, Rancher/Cattle, Kubernetes) in order to easily scale out your functions. Especially when many companies already have these set up for their more vanilla applications.
This is an example I've seen that can leverage a docker swarm for invoking functions, loosely modeled after AWS Lambda: https://github.com/alexellis/faas
Hi there - we've actually built a lot of our container ecosystem around existing Linux tools, including `systemd-nspawn`, `btrfs` and more rather than creating the whole stack from scratch - and again this is all controlled from Haskell. We experimented with Docker, Kubernetes and more, but found they they made lots of assumptions about what was running inside a container that didn't mesh with our compute model, so using lower-level primitives worked better for us.
We're really lucky also to have one of the main `rkt` developers joining us soon to work on the container side.
Application sweet spot is a funny thing, because it's mostly determined by the presence of specific libraries. What Haskell has more than anything is ridiculously general libraries. It's got perfectly good libraries for web development, Postgres access &access, but the only place I'd say it's got a real application-specific strength in libraries is parsing.
Weirdly, what Haskell is good at is generality. Which is much the same as for LISP.
I'd say the Haskell community remains academic/hobbyist, but that's not to say there aren't plenty of people doing Real Work in it.
I agree that parsing libraries are probably the strongest part of the Haskell ecosystem for applications. The only time I've chosen Haskell for a serious application was because of its parsing libraries (specifically parsec): https://blogdown.io/.
"academic/hobbyist" probably doesn't give justice to the benefit those works provide for languages that implement experiments pioneered in i.e. haskell. I say "probably" because I can't give any exact examples.
Ok, so the generality like LISP/Scheme with a few basic productions/rules and some basic operations could be used to create/represent any levels of complexity. Something like Northhead & Russell's Principia Mathematica.
Yes, and that's why you're often scratching your head at the function definitions: they're extremely general solutions to general problems. This, in conjunction with the documentation issue means that often you end up getting referred to papers, which does nothing to help the "only for academics" impression.
I think painting Haskell as a LISP but in a blank-slate world without packages is not quite accurate. There is a large library of existing packages that take you beyond the language's primitive constructions.
Haskell is good for problems where you want to spend most of your time thinking in terms of the problem domain and not so much about all the little implementation details.
For instance, if you were to write a CAD program, you probable want to be spending your time thinking about algebra and geometry, and not about little details like how an array is laid out in memory or when memory needs to be freed. The latter might be important for optimizing the last bit of performance out of a tight loop, but a lot of problems are hard enough on their own that just coming up with any solution that works is enough trouble on its own.
It's also good for problems where you want pretty good performance but don't have to be as fast as C, or where you want to take advantage of multi-core parallelism but don't want to deal with the usual complexities of threaded code.
Persistent data structures are really nice for certain kinds of applications, such as web services, where you have some complex internal state that needs to be updated periodically. Error handling is really easy when you can just bail out when something fails, since none of the state was ever modified-in-place and your old pointer to the old state is still valid.
The powerful static type checker makes Haskell a good fit for applications where it's really important that you calculate the right answer, or where you don't want to spend much time tracking down weird bugs.
Haskell isn't so good at applications where you need hard real-time guarantees or you want to have very fine control over the memory layout of your data structures or you want to be able to easily predict CPU or memory usage.
Haskell has a preference for looking academic, and is one of the favorites on academic circles. But that does not mean it's not used within industry.
I'd say that people mostly do not know what is running in industry. It could be any share of anything. We get a hint looking at job offers; Haskell is small there, but not unheard of.
On the sweet spot, Haskell is great for modeling complex domains; it's great for long-lived mostly maintenance stuff, since refactoring is so easy; it tends to generate very reliable and reasonably fast (faster than Java/.Net) executables, but hinders complete correctness proofs and has some grave speed issues that can ruin your day if you hit them.
I would like to know how is it to manage a team of Haskell developers. I do expect it to be better than Java/.Net in forcing safe interfaces between developers (thus making it reasonably easy to survive a bad developer in a team), but I never saw it in practice.
>> I'd say that people mostly do not know what is running in industry
well if anyone does know what's out there then HN crowd would be a pretty good suspect in that regard.
I personally have done my fair share of consulting and prof services engagements - dozens of them across all kinds of industries - I've never seen a Haskell shop.
Java, Python, JS, Golang, Scala, Clojure, .Net, C, PHP - definitely out there in the field. Haskell - not so much.
Your experience may be biased. If I had a shop that depended on external consultants for development, I'd avoid any language that I'm not certain to be mainstream. It's just too risky.
Then he'll have to look up some that he can't name off the top of his head? I can't name any Java consultancies, but I'm nonetheless confident that there are plenty...
With regard to Haskell, there are a lot of people interested in Haskell work compared to the number of positions presently available.
It's not as bad as you say, exactly. Yes, some trendy languages are overrepresented (coughRustcough), and there are people with blinders on about what programming outside the Valley is like. But there's way too much .NET, Java and such posted for HN to be wholly unaware of trends in the broader industry. Java may see some use in NorCal, but I really doubt C# does anywhere near as much as it pops up here.
I think I can safely say it isn't used heavily in many production environments but it is used heavily in some. For example, one of Facebook's abuse systems, dealing with 1M requests per second is written in Haskell: https://code.facebook.com/posts/745068642270222/fighting-spa...
As for particular places where it shines, I don't know of any in particular. I remember when taking my Declarative Programming course in university my lecturer loved to talk about an experiment the military did where they challenged a few teams to implement some kind of radar system in their chosen language and evaluated them based on time taken, correctness and code complexity (LOC) at the end. The Haskell team performed very very well. I can't find any details on this experiment though...
But take it with some sacks of salt -- it's a single research paper. It should be replicated, peer reviewed, etc to have any merit. It's methodology could be totally crap for example...
It is really interesting. I do get the feeling that the researchers really wanted Haskell to win, though. It would be nice if researchers with other favorite languages also arrived at the same conclusion.
I've been playing with it for about 5 years, started the local Haskell meetup, etc. I used it on a side project that reached beta in 2015, and on a startup at the beginning of 2016. Starting at the end of 2016 I started using it on a big contract. I had to convince the client it was a good idea, but they've been convinced by my team's productivity. All of the above were back end web services. The current project is a bunch of systems to underwrite and issue loans and involves an ML calculation (hosted on data robot).
I love it for general back end web development. I can actually solve problems so they stay solved.
I'm VERY curious about this! Backend web services not being the sexiest / most theory-requiring subject.
How did you convince the client, and why did you choose Haskell for this over, say, .NET, which throws a web service up quicker than it took me to type this? What sort of productivity are you achieving?
Haskell works really well for things revolving around languages.
At SQream (www.sqream.com), Haskell is used for CLI, SQL parser, SQL language compiler, SQL optimizations and a variety of tools.
We also use Haskell to generate Python code for testing (think: describing a testing scenario and sets of queries, translating into a runnable, predictable Python script that's easy to debug)
To me, this is Haskell's most appealing strength. The idea is, you can express your domain and logic in Haskell in pure functions which are highly testable, and then you can use that to generate code in any language you want. And the code you generate can have whatever zero-cost abstractions you choose to invent, all implemented in the nice, functional Haskell layer.
Haskell's strength at expressing and transforming AST's, in pure functional, understandable, testable code seems to be the key here.
Production use cases pushed me towards Haskell. What elegance there is serves towards reasoning about the correctness of complex code, however the tradeoff is difficulty in reasoning about performance. You can still get great performance! And I use it for prototyping in the Type-Driven Development style. Strongly disagree with "if it compiles it is correct" but we have great tools like Doctest and QuickCheck for that. I usually write types with DocTest examples first, then implement. undefined with lazy evaluation is really cool for figuring out function signatures first.
> reasoning about the correctness of complex code, however the tradeoff is difficulty in reasoning about performance
That's what drove me to languages like OCaml and Rust, which try to solve this tradeoff in a different way: Same thing about strictness and correctnes (despite slight differences in the type system), but lazy evaluation is only provided when explicitly asked for.
The cool thing about OCaml is that you can actually reason about performance, as long as you ignore memory issues (memory usage, garbage collector runs, etc.). Rust is heavily inspired by OCaml and allows you to also reason about correctness and performance of memory usage.
> The cool thing about OCaml is that you can actually reason about performance, as long as you ignore memory issues (memory usage, garbage collector runs, etc.).
If you ignore mem issues you can also reason about the performance of Haskell :)
If we want to reason about perf, I think Rust is currently leading in terms of a modern language that allows perf reasoning.
> If you ignore mem issues you can also reason about the performance of Haskell :)
Really? My guess (based on close to complete ignorance) would be that laziness would make reasoning about performance hard. Can you ELI5 why my guess is wrong?
All performance issues caused by laziness are memory issues. Something isn't evaluated promptly, so large amounts of memory are consumed storing unevaluated expressions. This causes drag on the garbage collector and excess cache thrashing when it finally is evaluated.
But all of that is just because data stays in memory longer than expected.
If you ignore that, laziness has a small performance impact, but it's less than the performance impact of interpreting code, which is something many people consider completely acceptable.
Both you and cies said that "All performance issues caused by laziness are memory issues." My intuitive sense was more like: You've got a lazy calculation that has several stages. At some point in the pipeline, you're doing something slow, but it's not obvious. When you get to the end and start using the data, the whole calculation is slow. But it's not clear where it's slow, because of the laziness.
That may not be "a performance issue caused by laziness", in your terms, because the laziness isn't causing performance issues. But it's laziness making a performance issue hard to find.
Does my scenario happen? Is it common? Is it easy to analyze/find/fix when it does happen?
Most generally, does laziness make it harder to reason about performance?
That happens sometimes, but the solution is the same as in a strict language. Break out the profiler and see what's taking all the time. There's a learning curve while you figure out what the usual suspects are, but usually you can find things pretty quickly afterwards.
Profiling is a bit of a black art in any case. I've seen horrible performance problems in python when we added enough code to cause cache thrashing to a loop. No new code was slow, but the size of it crossed a threshold and things got suddenly bad.
Some performance problems in Haskell are hard to track down, but most are pretty easy to find and fix. It's basically the same as every other language in that way.
Laziness can actually make things faster in some scenarios. Take Python2's range() function for instance, which eagerly builds the entire list and returns it. range() is commonly used for for loops, so Python builds this list (an N operation) and returns it, and then the for loop iterates over it (another N operation). But if you use xrange(), you get a generator instead, which can lazily construct values when asked for one. It turns the for loop case into a single N operation. Similarly if you have nested function call chains, if everything is returning an eagerly evaluated list, you have to construct the whole list in memory at each call and pass it to the next stage which will eagerly iterate over the whole thing to generate its list in memory and so forth, eventually the old list can be freed / GC'd when the new list has been made.. but with lazy sequences, you only need memory for the whole list at the very end, if you want it and not just individual values. Python3's range() replaced Python2's with xrange(). (The other option is a numpy-like approach where you eagerly allocate the list and then every stage just mutates that single thing.)
The 'memory stays around indefinitely' problem is caused by not cleaning up references to generators. You can make your own xrange() clone in Python easily, and start consuming values, but if you stop before consuming all of them (assuming it's not an infinite generator), that xrange() generator object still lives and has to keep around whatever references it needs to generate the next value if you ask it for one. When your whole language is based around laziness, you might have a lot of these and it may not be clear what references are still around that keep the GC from cleaning them up.
I wouldn't say reasoning about performance is necessarily more difficult, and profiling will light up a slow part of a lazy pipeline just as well as an eager one. My own struggles with laziness in Clojure were all around my own confusions about "I thought this sequence was going to be evaluated, but it wasn't!" which may have led to unnecessary calls to (doall), which forcibly walks a sequence and puts the whole thing in memory.
No projects under the Lumi name yet, but our team contributes to many FOSS projects and also have their own interesting projects such as Purescript by Phil Freeman who recently joined the team: https://github.com/purescript/purescript
I just saw a haskell meetup where they're using haskell for devops. Basically static typing the underlying machine configurations. As usual, being an Haskell code, the abstraction is generalized and you can apply the logic on other domains.
Thats really interesting. So config files themselves could be considered as some complex type and the inherent language type checking may be used to validate them?
Somehow reminds me of Structs/Records in Erlang/Elixir and using pattern matching, conditions and guard clauses to enforce validity or logic.
I've been using propellor for a few months now and it'fantastic. It takes a little while to get your round it but compared to puppet, chef and ansible, of which I've used all, it's a breath of fresh air.
I've been writing Haskell for nearly 10 years and am coming up on 4 years writing it full time. (You could say I know the language pretty well...)
I think it's just a pretty good general purpose language, for the most part? I know that's not very exciting of an answer, but I do consulting for people and I've seen Haskell for everything our clients do, from web development (a lot of web development), compiler development, distributed OLAP style analytic processing on large data volumes, custom languages/DSLs, machine learning, hardware/FPGA design work, desktop software (flight tracking/schedule software, used by on-ground airport personnel, with some incredibly beautiful code), cryptocurrency and cryptography etc etc. There are the occasional research-y things, and we've also just done things like that for random institutions or companies (e.g. prototyping RINA-style non-TCP network designs.)
I wouldn't say any of these were a particular "sweet spot". Though some things like compiler development or web development are pretty pleasurable. The language and libraries make those pretty good. Web development is one of the more popular things now especially. Everyone has to have their thing in a browser now, and there's a fair amount of library code for this stuff now. Definitely one of the stronger things.
You do get the occasional problem that just has this perfectly elegant solution at the core and that's always an awesome bonus. One of my favorites was the aforementioned flight software. It had this very strict set of bureaucratic guidelines but all the core logic ended up having a very nice, pure formulation that ended up being about 8 functions used in a single function-composition pipeline. (We use this sometimes as an example in our training of real solutions for real problems.) A lot of software is much more boring and traditional.
Mostly, it works well and stays out of my way, and lets me solve problems for good. And I really like the language in general compared to most other languages; it hits a very unique sweet spot that most others don't try to touch.
I do admit the sort of high level, abstract shenanigans are fun and interesting too, but I think of it more as a fun side thing. Just this week I literally wrote the documentation for a data type, describing it as an "Arbitrary-rank Hypercuboid, parameterized over its dimension." Luckily I do not think you will ever have to use that code in reality, so you can breath easy. :)
---
EDIT: and to be fair, there are definitely some problems. Vis-a-vis my joke, we could all collectively use better "soft" documentation. I think some problems like "operator overload extravaganza" are overblown due to unfamiliarity. Most real codebases aren't operator soup; but bigger problems are things like "When to use the right abstraction" are real problems we have hard times explaining.
I don't think performance is much harder than most other languages, but we definitely don't have tools that are as nice in some ways, nor tools that are as diverse. The compiler is also slow compared to e.g. OCaml. Some parts of the ecosystem are a lot worse than others; e.g. easy-to-use native GUI bindings are still a bit open. Windows is a bit less nice to use than Unixen.
There's a big gap between training in-person vs online/self taught content too, IME.
I am not an expert in any way, but parsers seem suited to Haskell. Perl 6's original parser was build using Haskell I believe and went on to influence the language a lot (again this is all stuff i have read, I have no real practical experience with either Perl 6 or Haskell).
Is Haskell more academic in nature or used heavily in Production environments?
Is there an application sweet spot/domain (say Artificial Intelligence/Machine Learning, etc) where it shines over using other languages (I am not talking about software/language architectural issues like type systems or such)?
I have no experience with Haskell but do use functional languages such as Erlang/Elixir on and off.