Command-line Tools can be 235x Faster than your Hadoop Cluster (2014)
Conclusion: Hopefully this has illustrated some points about using and abusing tools like Hadoop for data processing tasks that can better be accomplished on a single machine with simple shell commands and tools.
This article is good for new programmers to understand why certain solutions are better at scale, there is no silver bullet. And also, this is from 2014, and the dataset is < 4GB. No reason to use hadoop.
The discussion we had here was involving TB of data, so I'm curious how this is faster with CLIs rather than parallel processing...
JQ is very convenient, even if your files are more than 100GB.
I often need to extract one field from huge JSON line files, I just pipe jq to it to get results. It's slower, but implementing proper data processing will take more time.
It breaks GDPR easily: GDPR enforces you to comply with opt-out by default, no workaround by prefilling before hitting submit.
While some think this applies only to personal data, then yes. But it takes only one line of code to use my phone number for testing while I test locally a register form in the application I'm developing.
Once it gets sent to Copilot I can threaten with legal action if they are not taking it down.
Has there ever been a GDPR fine that actually exhausted all applicable legal challenges within a sufficiently short delay from initial violation to actually matter?
Short delay: depends on your DPA, I doubt any country is fast enough. On the other hand, this is the legitimate interest of GitHub, so it would require investigation, maybe even litigation.
Not working from home is easier than from office. But in general why keep employees you can’t trust? At least on the scope of startups it shouldn’t be a problem. When you know each person face to face, everyone in the team knows if someone is slacking. Doesn’t matter if it’s wfh or from office.
Thing is, who decides how many tasks someone can do in a day? What if they get paid for 8 but only work for one, but the manager doesn't know they do, and they never communicate their workload is too low?
Why would you expect Homo Economicus to ask for more work? The companies they work for chase infinite profit at zero cost as a matter of principle, why shouldn’t employees?
This holds whether their butt is in a seat in some office or at home.
Same can be said for people using LLM agents to complete jobs faster than humans ever possibly can. It's not like they just fluked it. They've learned how to harness the capabilities of the tech. Now companies are introducing this stuff as a normal workflow but they are clueless as to how it actually works and expecting 10x output from people.
It will all crash when they will see that people can't do 10x, even with AI. It requires too much expertise and knowledge in the field to actually make it work as a hired professional. Look at the AWS outages... and they are professionals, right? RIGHT?
Nail guns are tools, just like hammers. However, you have to know how to use it and to know how to adjust the pressure for the depth you need. It also costs money, much more than a hammer. And you can't use normal nails, you have to have a specific cartridge of nails, and you must know how to adjust it, and ultimately to not die.
Nailguns are not as complicated as you think, anybody with IQ over 80 can be trained to the top proficiency in 30 mins. Same goes for other power tools, they are generally much easier to use and more productive than their human-powered equivalent. The effect of the construction industry adoption of those is in smaller crew sizes, which is also being observed in SW industry.
I think it's a fair comparison; experienced carpenters who've learned to work fast with a hammer, now asked to be 10x more productive while using a new tool they don't have experience with. It probably got more than a few a bit bothered.
get your carpenter a circular saw, a drill, a router, a hand planner and an orbital. on many jobs you ll get your 10x.
you don't need much experience. the tools make you insanely faster with much much less physical strain and maintenance time. they're simple, predictable, reliable, and obscenely powerful.
an experimented carpenter would take a few minutes to be decent at using then.
- stable and "slow" is best
- don't ever let the blade get pinched (by wood)
- be mindful of the cords
- keep the flesh out of the way
- goggles up and don't breathe the dust
There are very few contractors still swinging a hammer. They're going to be slower and more expensive than the competition, which is a major factor in getting the job.
This is a management issue. They should be keeping an eye on the progress and see if the pace is enough for the deadline. Estimates are a thing too, I know. But you can see if someone is slacking and falls behind. If the workload is low, why would it matter as long as the things are done?
Companies should stop penalizing people for being work efficient, and increase their salary if the workload increases.
Doing this will enforce Claude to raise prices or decrease limits. People do this yet they complain that services no longer offer the same amount of usage or increases prices... There is also corporate greed, but honestly, people abusing the system is what drives that.
As a strong believer in consumer rights and corporate accountability, it always irks me when businesses brandish their one-sided ToS that 100% works in their favor to the detriment of individuals and society at large. Bonus points if they claim the moral upper ground when they themselves break all social rules and norms with no accountability whatsoever.
I don't have any plans to subscribe or share access to coding agents myself, but it irks me nonetheless.
Out-of-the-box thinker with an amazing, efficient, hyper-focused mind. Tables got wiped in production? I am the one who stays calm while everyone is in panic mode, running around like it's some fire going on. I can bring unique and creative solutions that the competitors won't think about.
Some say I am a high-energy person that is super fun to be around, with an amazing judge of character, packed with a lot of resilience, intuition and top-notch pattern-recognition skills.
Had multiple clients across multiple industries, from gaming to advertising and real estate. I have switched from freelancing to full-time roles due to stability and better work-life balance.
Keeping the freelance business for B2B collaborations and as a side project for Open-Source, global initiatives or helping people.
I think it has to do with the building, perhaps there is a Faraday cage caused by the copper net layer in the walls. From a similar experience, the doorstep of my apartment is really close to my other 2 neighbours (ground floor), and they all have WiFi routers above the entrance, inside — it was very hard to connect and use Wifi in the house on 2.4GHz due to the bandwidth, even if I changed the channels manually, so I was forced to buy a 5GHz router to circumvent the shortage. Also, my LTE doesn't work at all at my doorstep, so it's pretty apparent.
Not really - more as an open-source alternative that has many benefits for running it yourself instead of paying or being limited by existing, highly-reliable SaaS products.
> Hadoop: bro
> Spark: bro
> hive: bro
> data team: bro