> over several days, to run a work load an order of magnitude smaller
Here I sit, running a query on a fancy cloud-based tool we pay nontrivial amounts of money for, which takes ~15 minutes.
If I download the data set to a Linux box I can do the query in 3 seconds with grep and awk.
Oh but that is not The Way. So here I sit waiting ~15 minutes every time I need to fine tune and test the query.
Also, of course the query now is written in the vendor's homegrown weird query language which is lacking a lot of functionality, so whenever I need to do some different transformation or pull apart data a bit differently, I get to file a feature request and wait a few month for it to be implemented. On the linux box I could just change my awk parameters a little bit (or throw perl in the pipeline for heavier lifting) and be done in a minute. But hey at least I can put the ticket in blocked state for a few months while waiting for the vendor.
> It is [...] entirely optional to call free. If you don’t call free, memory usage will increase over time, but technically, it’s not a leak. As an optimization, you may choose to call free to reduce memory, but again, strictly optional.
This is beautiful! Unless your program is long-running, there's no point of ever calling free in your C programs. The system will free the memory for you when the program ends. Free-less C programming is an exhilarating experience that I recommend to anybody.
In the rare cases where your program needs to be long-running, leaks may become a real problem. In that case, it's better to write a long-running shell script that calls a pipeline of elegant free-less C programs.
> Wanna build a video game that teaches developers how to code and use Twilio? Let's try it! Wanna build an AI application with Tony Hawk and have Tony Hawk debug the code live on stage? Sure!
These are the strange effects of ZIRP and infinite QE. Many companies never had to care at all about profit and could just do things "for fun", and still see valuations skyrocket as long as they hired more people. What a time.
35mb seems small to you? That's an absolutely massive binary.
But to answer your question, you're going to see some performance improvements if your binary can fit into lower levels of the CPU cache.
Kdb+/q for example is less than 700kb, which mean that the entire thing can fit into the L1 instruction cache on high end server CPUs. And that size is even more impressive considering its only dynamically linked to libc and libpthread.
This small size contributes to the out of this world performance the system has. And remember that's the language and the time series db.
> Java has a culture of over-engineering [which] Go successfully jettisoned
[looks at the code bases of several recent jobs]
[shakes head in violent disagreement]
If I'm having to open 6-8 different files just to follow what a HTTP handler does because it calls into an interface which calls into an interface which calls into an interface (and there's no possibility that any of these will ever have a different implementation) ... I think we're firmly into over-engineering territory.
Google builds non-PIE, non-PIC, static, profile-guided, link-time-optimized, and post-link-optimized binaries and probably DGAF about calling conventions.
What I really want to see is a music programming language that does not require elementary knowledge of trig. In fact, I don't want it to use any numbers at all.
They all do: Faust, Impromptu, ChuCK, csound, SuperCollider, etc., etc. I suppose that by itself should convince me that there's no other way (there are things like Orca, but I'm thinking of things that look more like conventional programming languages).
I seldom think about numbers when I'm programming a synthesizer -- I just turn this knob more this way or this slider down a bit. Why can't a music programming language be more . . . gestural? Not sure what the right word is. I want the benefits of a textual programming language, but I don't really want to have to start thinking about sound waves in terms of literal numeric frequencies, particular since I really don't do that while doing sound design.
I don't meant to pick on your project in particular; it looks really cool. But since you're writing one of these, perhaps you can answer my question. Why do these languages ask synth programmers to think in terms of precise numbers when programming an "actual" synth isn't really like that at all?
Every data category is given away to affiliates. Several categories are sold to ad networks.
And also the part on that page where they very explicitly say "To request to opt out of our future sale of your Personal Information and/or our “sharing” of your Personal Information for purposes of cross-context behavioral advertising, or any future processing for purposes of targeted advertising, click here to go to the web form or email us at permissionslip@cr.consumer.org."
And also the part where they say a giant middle fingered FUCK YOU to everyone not protected by force of law, "We may decline to honor your request where...there is no legal obligation with which CR must comply." Which is buried so far down that it may as well be in the back of a disused lavatory marked beware of the leopard.
> I know everybody is in love with Microsoft these days because of Open AI
The cognitive dissonance this provoked in me is astounding. The only people I've seen praising anything AI-related at the same kinds of people who were boosting cryptocurrency and NFTs. Are there really thoughtful, mature people who think that Open AI is in any way positive?
EDIT: The most pro-Microsoft opinions I've seen have mostly been related to VS Code. Personally I don't get it - it's basically functionality-equivalent to IntelliJ, with maybe _slightly_ nicer visuals - but, sure, it works well enough, I guess!
CLI data processing is always fun and cool. But it tends to also be limited in scope and practicality, not to mention performance if you're chaining operations between function calls and it needs to re-parse the data every time.
If you want to avoid SQL, it's really hard to beat a "data frame" data structure for tabular data processing including things like joins.
It takes a little effort to fully understand the configuration file format (hint: you've got to read the documentation, not just look at examples to fully grok it), but it's so worth it, IMO.
It's also a nice treat to have the founder and technical leader (Willy Tarreau) of the HAProxy company being so active in the community, so many years later (the initital release was in 2001). I regularly see him answering e.g. newbie questions.
Agreed. Haproxy is an absolute wonder compared to similar systems. It all just feels so much cleaner, thought out, and built from the ground up for many different use cases. It very much has a feel that reminds me a lot of the spirit of sqlite.
Roughly around when the oligoculture started, and browsers stopped being "user agents", what a coincidence!
We need more browsers that focus on user features, not website features. The web is basically beyond feature-complete already, but the tools we use to interact with it are so basic. Like riding a horse on an interstate!
I find that coloquially we use the term "science" in several distinct meanings, two of which are:
1. Science as the body of knowledge
2. Science as a method / approach
I also find that mixing the meanings/perspectives/intent up in a single conversation is common, sometimes accidentally sometimes intentionally.
In my ignorance (ComSci major, so not a real science :), I would describe myself as extremely positive / confident to "Science, the approach/method" and, through that approach, pragmatic about the "Science the current body of knowledge".
In other words:
Yes, there are very much limits to what we currently know, and some of what we think we know will turn out to be wrong, subtly or catastrophically. There are definitely huge limits and uncertainties to Science the body of knowledge!
But, acknowledging it is kinda the point, and the best way to figure it out that we are currently aware of is through scientific approach. (I've just realized I might even have become a zealot that you describe, because I can't even figure out what a plausible & feasible alternative method is, if your goals are to actually figure things out. To that point, I find humility and skepticism about your current science body of knowledge a crucial part of science the method, something which most other methods lack).
I find distinction is crucial especially in political and religious discussion frameworks. Otherwise, I never know if I agree or disagree with statements regarding "limits of science" etc.
(This is all further mixed up by zillion of daily popular articles where "Science says that [...]!!!" or "[...], scientists find", which... ugh, oversimplify at best and deceive more likely)
It's unsurprising when you consider that there are often several magnitudes of difference in code between what code grows to when you have the capacity and time and compounding user requests, and what a meaningful starting point that provides useful functionality above and beyond what you had without it looks like.
As an extreme example here[1] is an article by Brian Kernighan about a basic regexp matcher by Rob Pike. The code, with comments, is 35 lines of C.
Meanwhile, the regexp engine in Ruby 3.2.2, not even including the Regexp class visible to the language, is ~20597 lines excluding the headers.
They are not reasonably comparable, of course. Pike's code provides a very basic syntax which doesn't even support character classes, and does nothing to speed up the matching. The latter supports a very much more complex syntax, and is far faster for repeated or long matches.
But while Ruby's regexp engine is 588 times large than Pike's code, you of course get a vastly higher boost in overall system capability from those first 35 lines of code (from no regexp matching to some regexp matching) than the last 35 lines of code, or indeed the second 35 lines.
So if you have a small system and a small team, you work with that and start small and it won't be that surprising when you get a lot done in few lines of code, even though you have a long list of things you'll add when you can justify the extra resources (like those missing language classes, and a billion other features)
(Then you get a larger system, and you'll very soon find yourself wondering how you could manage with so little)
I think it's mostly surprising because most developers today aren't used to thinking about capacity constraints of small systems, and so starts designing for lots of features from the start (can't have regexps without character classes, and capture groups, and back-references, and ...).
Minimal desktops are productivity boosters. On top of that ones that can be controlled by keyboard input are super important.
99% of the time you just want information. Text. The chrome does not matter. Yet modern desktops and web pages make humans hunt and pick through poorly designed interfaces to get the info we need.
I use EXWM. It’s a time saver. I have no desktop because I am either using the full screen for a single application or flip flopping between a few apps that share the entire screen. This means every pixel of the screen is used and not wasted on useless images of empty fields.
In top of that I am able to designate windows and frames for specific jobs or functions, always able to recall the last terminal or most relevance.
We fucked up with movable windows. They are inefficient complicated and bring little value over a simple list of activities to switch between.
Same. I was an early Model S purchaser, and then got a second. I loved its zero emissions, responsiveness and safety rating, but Tesla's antics left me unlikely to ever be a customer again, and to be extremely skeptical for their long term success. Though I still hope for a major turnaround.
Both my cars did come with quality issued; the seat controls didn't work on my first car, and a tail light was out on my second. This is at delivery. That seems crazy on a 100k car, but it honestly didn't bother me too much.
My biggest motivation for buying an EV was to support and contribute to positive changes we need for the climate. It was galling to see Tesla buy a tranche of BitCoin, and felt like an absolute kick in the teeth. It made a complete mockery of the "zero emissions" aspirational tag. I realize that tag has never counted emissions created during production, but to pour money into an egregious emitter with negative social utility was bonkers.
Over the years, it was very concerning to see the "Full Self Driving" and "Robotaxis" lines so consistently promoted in a way that seemed to be to be an outright scam. I'm not a lawyer, but as a consumer I can't comprehend how massive fines or prison sentences have not followed.
But the real last straw was Tesla's woeful customer service. Everything is through a misanthropic app and when anything goes off-script, it becomes byzantine and kafkaesque. I had long long waits for replacement parts, but also a crazy situation where the Tesla team couldn't seem to process a payment from my insurer properly and then tried to hold my car hostage. I convinced the local service team to release my car so that I could drive it away when I showed them the paper trail, but resolving the situation so that Tesla did not think I owed them thousands of dollars took much longer. Twice I had to resort to physically going to a service center and politely refusing to leave until my issues had been sufficiently escalated and I had points of contacts who would respond to me. Every time I simply framed it as "This is a 100k car, and I think it's reasonable to expect to be able to talk to a person when things go wrong.". The Tesla staff were incredibly overworked, the lady dealing with insurance claims said she had 160 or so ongoing delayed customers. They were as helpful as they could be within the confines of the system, but it just seemed clear to me that Tesla's corporate policies were designed for a short-sighted kind of scale at the expense of customer trust.
As I said though, I really hope they have a major turnaround.
It wasn't obsolete 15 years ago. There were production mapreduces making double-digit improvements in key metrics (watch time, apps purchased) much more recently than that. The system I worked on, Sibyl, isn't well known outside of Google, but used MR as an engine to do extremely large-scale machine learning. we added a number of features and used MR in ways that would have been extremely challenging to reimplement while maintaining performance targets.
I'm not even sure the mapreduce code has been deleted from google3 yet.
To be fair, MR was definitely dated by the time I joined- 2007- and I'm surprised it lasted as long as it did. But it was really well-tuned and reliable.
Also the MR paper was never intended to stake Google's position as a data processing provider (that came far, far later). The MR, Bigtable, and GFS papers were written to attract software engineers to work on infra at google, to share some useful ideas with the world (specifically, the distributed shuffle in mapreduce, the bloom filter in bigtable, and the single-master-index-in-ram of GFS), and finally, to "show off".
The grumpiest and unhappiest developers I know are JavaScript developers (although granted this could just be because there are so many of them).
I have written a lot of JS in the last 4 years but have been writing code for around 16 years and I can say that this current period of React driven development has been the least fun I’ve had writing code.
I’m now scaling down the amount of frontend JavaScript I write and my enjoyment is going back up.
There's no legal requirements to call yourself a mechanical engineer or an electrical engineer in most of the world. I think the real difference is that the barrier to entry for programming is so low you get the full spectrum of people calling themselves "engineer" but the word has some connotation of "building things properly" which a lot of programmers definitely don't do.
I'm a non software engineer so maybe I do have a chip on my shoulder, personally I don't really have a beef with what people call themselves, however I will say the distinguishing factor in my opinion between software and the broader engineering world is liability and the culture that exists around that. I am professionally held liable for any work I perform while calling myself an engineer. There can be potentially very serious consequences if I am found to have been negligent or to have not followed best practices.
In most parts of the software world I don't think there exists similar circumstances. I'm not sure if there are agreed industry best practices or certification requirements for most types of software development.
I am both a software "engineer", and have separate non-CS engineering degrees.
What the vast majority of us in software do is not engineering. I think like you say liability is one part, but another big part is the immense breadth of unknowns we have to deal with and accept, but never really chase down. There's no budget or time given for properly figuring out and scoping a project (for most programming), because the penalty for failure isn't so steep. It's not written in blood, so we don't dedicate the resources necessary to really engineer it.
Some programming is arguably engineering. Many parts of aerospace for example. Some of the medical devices space. But it's overall a very small percentage of total programming happening in that world.
And you know, that's probably okay, for now at least.
I've spent 5 years at university and turning 30 this year, I'm going to quit my job in about a month and throw it all away. Can't say that my family is thrilled but I'm gaining weight like crazy, don't like my job, I don't fit in the culture, I don't like sitting down all day in front of a computer, I drink to much because of this.
I'm playing with the thought of opening a store in some small country town where I can cook food and sell basic necessities.
Is it a good plan, who knows but going on like I do now is also unsustainable.
So you're not alone and sometimes you just have to go for it.
Yes, but it supports the custom addon collection feature. This way you can use it in a browser that isn't based on the beta/nightly branches, and should be less crashy. I run Fennec to access a few privacy redirect extensions, largely to make reddit and twitter less awful to browse without their respective apps.
Here I sit, running a query on a fancy cloud-based tool we pay nontrivial amounts of money for, which takes ~15 minutes.
If I download the data set to a Linux box I can do the query in 3 seconds with grep and awk.
Oh but that is not The Way. So here I sit waiting ~15 minutes every time I need to fine tune and test the query.
Also, of course the query now is written in the vendor's homegrown weird query language which is lacking a lot of functionality, so whenever I need to do some different transformation or pull apart data a bit differently, I get to file a feature request and wait a few month for it to be implemented. On the linux box I could just change my awk parameters a little bit (or throw perl in the pipeline for heavier lifting) and be done in a minute. But hey at least I can put the ticket in blocked state for a few months while waiting for the vendor.
Why are we doing this?