I’m also a duckdb convert. All my notebooks have moved from Pandas and polars to Duckdb. It is faster to write and faster to read (after you return to a notebook after time away) and often faster to run. Certainly not slower to run.
My current habit is to suck down big datasets to parquet shards and then just query them with a wildcard in duckdb. I move to bigquery when doing true “big data” but a few GB of extract from BQ to a notebook VM disk and duckdb is super ergonomic and performant most of the time.
It’s the sql that I like. Being a veteran of when the world went mad for nosql it is just so nice to experience the revenge of sql.
I personally find polars easier to read/write than sql. Especially when you start doing UDFs with numpy/et. al. I think for me, duckdb's clear edge is the cli experience.
> It is faster to write and faster to read
At least on clickbench, polars and duckdb are roughly comparable (with polars edging out duckdb).
I use them both depending on which feels more natural for the task, often within the same project. The interop is easy and very high performance thanks to Apache Arrow: `df = duckdb.sql(sql).pl()` and `result = duckdb.sql("SELECT * FROM df")`.
For the bigger tasks, Exasol might also be a very neat option for you. We have a free personal edition that can scale regarding data volumes, #servers (MPP architecture) and complex workloads.
Recently, we have also compared ourselves against DuckDB and were 4 times faster even on a single node. We are in-memory optimized, but data doesn't need to fit in the RAM.
There is one turbine near where I live in Scandinavia that is very noisy. It is a low thumping sound that penetrates houses and is horrid. Those living within a km perhaps more won a court case to remove it but the owner has appealed and appealed and during the years or appeals the thing keeps turning and keeps being noisy so people can’t sleep. My understanding is the simulation and calculations of the noise that were part of the planning process were flawed and did not accurately model the terrain.
Meanwhile, not 5 km away, there are a bunch of turbines with people living around them and no problem.
So the exact slopes etc of the terrain is very important.
That sounds very much like either tower thump or a broken bearing, I think the neighbors would have a better case if they pushed the safety angle because a turbine in a bad state of maintenance is dangerous.
Then they'll be forced to fix it and it will be quiet again. You can ask them if it always was that noisy, if it wasn't then that's an extra arrow in their quiver. I'm very much pro renewables but safety is a major concern and operators that do not work safely and/or ignore valid complaints are a net negative for renewables.
In this case it is the terrain. The sound is funnelled somehow. In the planning process the owner has to commission a study that reports if the terrain shape is ok and compute the noise on nearby houses etc. Apparently you can get what you pay for. In the appeal a more independent study pulled the original study apart.
The nukes are to deter the US. They have been steadily increasing their missile range to first reach regional bases like Guam and now the all the way to the continental USA, and are now even launching a nuclear powered and nuclear armed ballistic missile submarine https://www.hisutton.com/DPRK-SSN-Update.html
The nukes are a bargaining chip (disarmament). Basically, if your country has the human and tech capital to develop a nuke, you probably should because it's free money.
I don't believe that NK's nukes deter the US from doing anything. Would NK nuke Guam and risk getting carpet-bombed with nukes for endless days and nights until even the ants are dead? Artillery on Seoul doesn't matter. The US would just ask SK to evacuate it.
The US doesn't do anything about the DPRK because it's not economically relevant (i.e. it doesn't have the world's largest oil reserves etc). In an ironic way, their economy being closed-off and mostly unintegrated with the Western world maintains the peace.
The nukes have many roles perhaps but I think the fully developed weapons are for retaliatory strike.
They are the North Korean leadership saying that if the US (or China or anyone really) tries to surgically decapitate them (like the US just did in Venezuela) then the nukes are used to take the attackers with them
Yes that's the orthodox doctrine of nuclear deterrent. To be truly effective you need a triad of land-based ICBMs, nuclear-armed submarines, and aircraft-based delivery systems so that your second-strike capability remains intact through any decapitation attempts.
If you don't have the triad then you need to brandish your capability more ostentatiously, like France does with its deliberate refusal to commit to a no-first-strike policy. This is (one of the many reasons) why North Korea does so much sabre-rattling: they don't have a (publicly known) nuclear triad for deterrence.
Just a note that the importance of the triad is a very American perspective on deterrence and most other countries don't seem to approach this the same way the US does.
The Russians really have a quad (they also have mobile, truck mounted ICBM's that form a significant part of their deterrent, offering some of the guaranteed second-strike advantages that the US gets from SSBN's- and which their SSBN program does not provide nearly as well as the USN does). The Chinese only recently added a manned aircraft leg of their triad with the JL-1. The Indians technically have a triad- just no silo based systems, all of their land based missiles are from TELs, and they only have two SSBN's and do not do alternate crews so more than 1/3 of the time they don't have any deterrent at sea. The Israeli's are not believed to have any sea-based ballistic missiles, their sea-based deterrent would be Popeye cruise missiles and so vulnerable to interception. The Pakistanis are still building their first sea-based deterrent. The French and the UK have no land-based missiles, they are only sea-based and airplanes. The South Africans invested in the Jericho missile more for its space launched capabilities than its warhead delivery abilities, and never really looked at anything sea-based, so far as is publicly known.
I don't think they fill the same strategic purposes, though. The value of silo based missiles to the US is as a missile-sponge, taking most of the warheads from a Russian first strike and keeping them from American cities (forcing any Russian first-strike to be counter-force instead of counter-value). This is not particularly valuable, honestly, which is why only the USSR during the height of the Cold War (largely in reaction to Minuteman) and China very recently have also made the investment into large numbers of ICBM silos.(1)
I won't claim to be as much an expert on Russian doctrine, but they seem to consider their mobile missiles to be a survivable second strike weapon, while silo based missiles are obviously not. Because their boomer fleet does not offer the same assured second strike, they rely on those mobile missiles to play a greater deterrent role then the US does.
1: That is the official justification for the US silos. The real reason for silos is, if you want to build a truly insane number of strategic warheads, silos are the only way to afford it- ships and planes and even TELs are too expensive. So first the US (worried they were behind because of the Missile Gap) built a thousand Minuteman (then tripled the deployed warheads with MIRV on the Minuteman-III). Then the Soviets responded with 1000 SS-11s of their own. But if you are only building a few hundred warheads total, you don't bother with silos, they don't add as much value as other delivery mechanisms.
I'm saying: Whether or not the Russians consider their silos to be more or less survivable than their truck-based missiles is immaterial, and doesn't change the calculus at the strategic level, because one of two things has to happen in a first-strike situation:
- You blanket the entire country in nuclear detonations and pray that you catch all the trucks scurrying around like nuclear-armed mice
or
- You spam dozens of missiles at a small number of hardened targets and hope you dent them (missile sponge silos)
Either way, you're severely depleting your arsenal to an infeasible level to do this. These are both counter-force attacks where targeting is the only difference, which the Strategic function does not concern itself with. That's a tactical consideration. Survivability of a land-based asset achieved by different means is still survivability of a land-based asset. In other words, it's still functionally a triad.
In the case of France in particular, the argument I recall reading is that: a) France was entering a period of austerity in defense spending as the Cold War ended, b) its siloed missiles were obsolete and in need of upgrades which promised to be costly, and c) France isn't very large geographically, so the "missile sponges" were limited to that little plateau north of Marseille which is pretty darn close to several major population centers, where an Ivy Mike-sized airburst could endanger Avignon and Marseille, not to mention leave a plume of fallout all the way into Germany.
But I'm just an ex Air Force officer who's been to France a bunch, so idk how accurate that is.
>The real reason for silos is, if you want to build a truly insane number of strategic warheads, silos are the only way to afford it
Yeah, I guess I mentally slot road mobile missiles as more like "less effective SLBM on the cheap," at least for a country the size of Russia (not sure that is as true for someone like North Korea where I speculate there is a larger use-it-or-lose-it penalty). There is definitely more of a continuum here between "missile-soak" and "survivable deterrent"- e.g. at the limit you could, in theory, vaporize all of the oceans with nuclear weapons to kill all the boomers, which turns them into missile soaks, but at a truly insane level.
I've seen open-source estimates that the 33rd Guards Rocket Army can distribute their three divisions of mobile missiles across something like 5,000 square miles of Siberia, mostly steppe/taiga (which the 7917/79221 are supposed to be capable of launching from, again according to open source reporting). That's more than 10% of all of North Korea, to give an idea why it would be different for the two countries. Being open-source, I don't have a good estimate for the survivability of the TEL, but let's somewhat arbitrarily say 5PSI is the limit. A 300kt W87 can put 5PSI over 3 mi^2, so doing 5,000 mi^2 would be about 1700 of them, for a grid-square blanket search. That seems to be impracticable, just for one third of their missiles(1).
So I think it's more about guaranteed second-strike than soaking (e.g. at three warheads per silo you'd need ~600 missiles to soak up that many warheads, instead of the 70-odd from mobile). Which is why I have seen some people consider those missiles as more about assured second-strike than missile-soak, with hints that the Russians consider that their role. The Russian doctrine does not align exactly with the American one (2) for sure and there are hints that the Russians consider road-mobile to be different from silo deployments.
1: I'm not as clear on how much deployment space the 27th Guards Rocket Army, in the European parts of Russia has, and whether they will run into similar problems to the French wrt population centers. There is also a whole separate discussion about how much counter-force and counter-value are truly separate on the receiving end, given, e.g. if Barksdale gets nuked Shreveport is going to be very very sad. But the RAND people were sure they were distinct!
2: At least, as far as this monolingual American can tell. My main source for this is the Arms Control Wonk blog and podcast, which actually does read and report on what the Russians describe as their doctrine, they are my source for the "Russians seem to consider road-mobile as more survivable second-strike than silos."
> To be truly effective you need a triad of land-based ICBMs, nuclear-armed submarines, and aircraft-based delivery systems
The core parts for MAD land-based missile silos (to soak up the enemy's missiles) and submarines (to ensure a second strike). Planes are largely a diplomatic deterrent inasmuch as they're easy to send out and easy to recall.
But Pyongyang isn't playing MAD. It's playing credible threat. And for a credible threat, you just need missiles. (On land or on subs.) The point is that you raise the stakes of e.g. a Maduro operation to risking Los Angeles.
Strategic bombers are just as important because MAD itself is fundamentally a political and diplomatic tool. The reason you have strategic bombers is, as you correctly said, so that you can signal your posture and intent by stationing them, dispersing them, launching them, and (most critically) recalling them.
But again, because MAD first and foremost is a deterrent, you want to provide diplomatic offramps for both you and your adversary. This is crucial. Putting the B-52s on airborne alert sends a very strong message, but so does recalling them from airborne alert.
By their very natures, SSBNs and ICBMs are not capable of playing this role.
It’s oblique but this puts me in mind of an old adage I recently heard about war: Of 100 men, one should be a warrior, nine should be soldiers, and 90 shouldn't be there at all.
I think this is true of software developers too: only in companies, the 90% don’t really know they shouldn’t be there and they build a whole world of systems and projects that is parallel to what the company actually needs.
My memory is there were a spate of SO scraping sites that google would surface above SO and google just would not zap.
It would have been super trivial to fix but google didn’t.
My pet theory was that google were getting doubleclick revenue from the scrapers so had incentives to let them scrape and to promote them in search results.
I remember those too! There were seemingly thousands of them!
Reminds me of my most black-hat project — a Wikipedia proxy with 2 Adsense ads injected into the page. It made me like $20-25 a month for a year or so but sadly (nah, perfectly fairly) Google got wise to it.
Honestly, if I'd gotten in earlier I bet it would have made more. I also made zero attempts to SEO it -- there were no links to it anywhere else on the Internet, so it would have been in the very first tranche of useless duplicative spam sites of that type to be cleaned up. It was up from like 2010-2013 or so.
Another plausible explanation is that the new owners didn’t develop the community in a good way. Instead of fixing the myriad of issues that were obvious to almost all contributors they instead basically let it die?
The new owners (well, not really new any more) are focused on adding AI to SO because it's the current hotness, and making other changes to try to extract more money that they're completely ignoring the community's issues and objections to their changes, which tend to be half-assed and full of bugs.
