Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Show HN: I built a self hosted recommendation feed to escape Google's algorithm (github.com/jawerty)
214 points by jawerty on July 19, 2022 | hide | past | favorite | 121 comments
I created this chrome extension for myself where I track my own behavior locally and recommend myself content from platforms I want content from (youtube/twitter/quora/etc) in a feed. I made it public just in case anyone else was interested.

I would rather have control over my own algorithm and own the data. Also, it gives me flexibility. Turns out I do like these feeds just not when I don't own it haha. Let me know what you think of my implementation?



That looks really helpful, nice job and thanks for the effort in documenting what it's doing.

I have been working on something a little bit related recently, related to books. Kind of reverse-discovery, maybe.

I noticed that my ebook libraries in apps pretty much always end up in the same ordering, or sets of orderings. So I would tend to forget about interesting books I bought, the longer it's been since I purchased them.

I wrote some scripts that collect random thumbnail images from my various ebook and PDF collections every so many minutes and gather them into little collections. Like shuffling cards and dealing some out. It's very simple stuff but it's already been nice to have.

I also noticed that my collection of Gutenberg texts didn't have the greatest thumbnails, so I made a script to pick random excerpts from N different Project Gutenberg books at intervals. Then it joins then to a single text file. Kind of like visiting the bookstore to browse through some books a bit, see what's interesting.

Anyway, this interests-related stuff is enjoyable. I really like your idea for the discovery side of things, thanks for sharing it.


Thank you for the words and for sharing. The excerpts into thumbnails idea is cool I like the concept of having a library where the cover is the first sentence of the first page or something like that.


Installed the plugin and very intriguing MVP. Thank you for pulling this together. I have two questions/feature requests.

- Have you thought about adding edit functionality to your content sourcing under feed settings to add or remove sites? Wikipedia, Stack Overflow, and other sites come to mind.

- Have you thought about adding an report export functionality of links recommended? Basically to help an individual understand why their copy of the algorithm is recommending specific items.


Yes 100% i'm working on making it so you can add sources you want. Also going to most likely add the ones you mentioned as default sources. Also I didn't think about your 2nd point but I like it going to add to my todo list


While I completely get being in control of your feeds, from rolling your own to understanding how & why the feeds you subscribe to work — also feel like it’s just as important to be aware of significant topics and beliefs that are not in your feeds.

Case in point, as another commenter pointed out — it appears the OP feels GAB, commonly viewed as a far-right social media platform, should be listed as one of a handful of default sources.

World needs more systems that actually reflect how the user fits into the world, allows people to understand differences and find shared beliefs — not systems that create information bubbles, enable isolation, and increase the likelihood of extremists.


GAB is a type of bubble, and definitely a (US politic) right-leaning discussion forum, but my ears perked up when you used the term "Far Right".

Is there such thing as a moderate-right social media platform? Does far-right mean "not supported by a majority of right-leaning people", or does it mean "extreme from the perspective of an average left-leaning person"? I'm not going to either defend Gab or put it down, but even if you deconstruct your statement, it's the product of an information bubble. And it can have different meanings depending on what type of information bubble you live in.


Moderates tend not to have their own platforms.


The one we are on right now feels pretty moderate to me.


It is moderate, but it's not some exclusionary, niche thing like a "far-right social platform" is. HN is generally a broadly-welcoming community that attempts to elicit good-faith, well-reasoned discussion. Its actual purpose is one of openness and community rather than, simply put, "us vs. them" worldviews.


HN would have continued its slide from idealistic libertarianism to extreme right with the rest of US politics without dang occasionally swatting people down for using it for ideological battle. It's only moderate because of active effort on the part of the mod and people who bring things out of sync with the guidelines to his attention.


It's interesting that it's only moderate because it's a benevolent but absolute autocracy.


HN is extremely pro-text (excellent, in my book), and pro-tech (not too bad, in limited doses). And pro-Science, which a depressing fraction of Americans might regard as extremely bad.

...and saying that HN is "our own" - vs. "one modest Province in our Benevolent Overlord Paul's Empire" - is a bit of an issue.


This comment inadvertantly encapsulates nearly all that is wrong with the world today, and from which much evil flows.

"Pro-science" is not a position, and being anti-science means being anti-humanity.


* i’m fully aware that my comment is about to tread into the all-too-common insufferable pedantic trope that forums have devolved into. soooo, apologies for the obnoxious pedantry ahead of time. *

pro-science also very often veers into anti-humanity as well. we like to pride ourselves that we’re keenly aware of how many unanswered questions we have and the current limits of science and technology — yet it seems increasingly common that when we’re chasing our love for science and technology, more and more we forget how these very topics can strip humanity from a given situation or conversation.

do i think religion is the answer when society’s pro-science stance leads to anti-human situations? absofuckinlutely not. (particularly organized religions. i’d be more open to religions if people could practice it as a personal thing. but sadly the religious always seem to want to inflict their personal beliefs on others.) the answer probably lies more in the region where the science minded get a firmer understanding that the human condition requires us to understand humanity more, not solely science. understanding other humans is a subject in which many of us are kinda lacking. i truly believe they were wise to require a certain number of humanities courses in university. honestly, seeing how terribly many of us in science fields struggle with understanding other humans, we could probably use a few more in the requirements.

in an attempt to tie this rant onto the topic: i’ve maintained for years that even in the best of our current iterations, algorithms are just plain terrible when compared to human curation. billions and billions spent and years and years later, still terrible.

i’m sure some will argue but i strongly believe this is true for just about everything, from news algorithms, music, movies, literature, art, shopping etc… etc… the best recommendations are still coming from other humans, by a significant amount.


You can say that here, because we're all in a space where science is unquestioned, but try and say that to my mom over the Christmas table and you won't get very far in that conversation.


I have a lot of concerns and issues with science as it's often practiced today, but I'd still consider myself pro-science. I certainty wouldn't say I accept science without question and the very idea of that seems absurd. Science isn't some religion that doesn't allow for questioning. Science is all about questioning.


It's not that the individuals don't question science. I, just like you, also question science both in form and substance. It's that this particular venue doesn't. Hackernews is not the space for discussing if science is even really helpful or good. It's not the space to have lengthy debates about the meaning of science and the price of progress. Hackernews is an optimistic entrepreneur/technology forum, we assume (by and large) that entrepreneurship and technological advancement is good.

What I'm trying to say is that all spaces are "echo chambers" in that they all implicitly have their own tone and angle that all the participants agree to uphold as they join. You don't discuss politics at the dinner table after all.


That's fair, although I will say I have seen some good discussions here at HN on the negative impacts of certain technologies which is encouraging. Those are the kinds of issues you'd hope tech people were giving some thought to.

I'll agree though that it's still very much encouraging of new technologies even those that will be certain to cause problems. For example, people here are far more interested in the cool things they can do using WebAssembly than they are worried about how it will be used to deliver malware and violate people's privacy.


> we're all in a space where science is unquestioned

Please consider the contradiction in this quote


That was intentional.

From the other comment I figured I should probably point that out clearly.


I'd argue that "pro-science" is absolutely a position. People holding this position might argue that science isn't the best way to learn information about the world (e.g. divine revelation or intuitionism might be preferred), or that the application of science to find solutions to our problems leads to bad outcomes (e.g. Luddites or Mennonite/Amish communities).

Whether we agree with that position or not is a whole different beast, but it absolutely is a real and tenable position that people can and do hold.


> I'd argue that "pro-science" is absolutely a position. People holding this position might argue that science isn't the best way to learn information about the world (e.g. divine revelation or intuitionism might be preferred), or that the application of science to find solutions to our problems leads to bad outcomes (e.g. Luddites or Mennonite/Amish communities).

Wait, this is worded incorrectly right? The pro-science position would be the one claiming that science is the best (or at least one good) way to learn information about the world right? I'd agree in that case that a person can be pro-science.


Whoops, yeah, that was an example of anti science as a position (the pro-science person would hold the opposite beliefs across the board).


Most people understand that "far-right" in this context means that it allows white supremacists and nazis to use its platform. It's a far right platform in the same sense that any bar that doesn't kick out nazis on site becomes a nazi bar, or a bar where cops feel comfortable becomes a cop bar. Regardless of creator intention a single customer base can shift the entire purpose of a product.

As for what a moderate-right platform would be, I'd actually point right here at hackernews. It allows discussion around more right-leaning topics, but draws a clear line at outright hate. There's progressives too (hi) but you'll find no shortage of right leaning people either (like OP apparently). The sheer number of libertarians here alone is enough to move the needle right of center.


> Most people understand that "far-right" in this context means that it allows white supremacists and nazis to use its platform.

That's literally every platform. There's no nazi-or-not test to take before you're allowed to use any of them. All platforms allow varying degrees of racist rhetoric ranging from dogwhistles to death threats and I'm not aware of a single platform that doesn't have racists in healthy numbers.

I think a better definition of a far-right platform would be found in its intended audience or in the percentage of racist users/content.


How are libertarians "right-wing"? Some libertarians could be described as extreme "left-wing" depending on being social or economic libertarians. I think it is better to not have such a simple dichotomy of "left" vs "right". People have political beliefs, those beliefs can be very complex, and the consistency of the beliefs depends largely on the perspective of the person. Having a simple "two sides" lined up against each other is overly simplistic and has lead to people being shoehorned into "us vs them" and feeling pressured to adopt the beliefs of the group they are closest to, which results in less diversity of political beliefs. I don't think even a circle of beliefs is enough, the degrees of freedom in political beliefs is enormous, which literally encompasses most of the things people believe about the world. It can't really be boiled down to a spectrum of two sides.


Libertarians are funny because most other ideologies can find some reason to view them in a bad light, but I think the reasons to view them as right wing is that they commonly have attitudes like, "regulations are bad," "taxes are theft," "leave me and my guns alone," "fuck the poor" etc. They're often proponents of private roads, private schools, private emergency services, and so on. This all offends the left. And of course, they offend the right with their lack of interest in policing people's personal lives.

I sort of consider myself a libertarian who sees the need for a social safety net, regulations, strong consumer protection, education, infrastructure, emergency services, and healthcare. You know, a democrat. :D


There is often a lot of overlap between libertarians and the far right, including the racist faction of the right. A libertarian might say for example that a hospital, school, or restaurant should be allowed to operate as "whites only". I think the vast majority of people identifying as libertarians would fall farther on the right side of the spectrum than the left with economic/pro-corporate/small government views being the primary driver.

As for not having a left/right spectrum of political views at all, I think it's somewhat inevitable in a (de facto) two party system, but it's worthwhile to acknowledge that virtually nobody is going to fit comfortably on one side or the other. People have tried for ages to come up with more nuanced multi-axis representations of where people fall within political spectrum, but even then it often just works out to carving out specific flavors of left/right.



The front page is just posts making fun of the president (acceptable), and tweets from Donald Trump (a patently treasonous figure, by any sane means of logic and reporting including the report made by mueller, who’s not a journalist), and Dinesh DSouza, a felon out and about only because of a pardon from above mentioned treacherer.

I’m not American BTW. Just an external observer who’s just flabbergasted how much a developed country can revolve into questioning of such obvious facts in the name of “both sides.”


The “out and about only because of a pardon” bit is false. From Wikipedia:

> In 2012, D'Souza contributed […] Two years later, D'Souza pleaded guilty in federal court to one felony charge […] He was sentenced to eight months in a halfway house near his home in San Diego, five years' probation, and a $30,000 fine.[29][30] In 2018, D'Souza was issued a pardon by President Donald Trump.[31]

Some simple math reveals the pardon came long after his “incarceration”, and the sentence including probation would have been fully served by now (2012 + 2 + 5 < 2022).


> including the report made by mueller

Where are you getting your information?

The report concludes that the investigation "did not establish that members of the Trump campaign conspired or coordinated with the Russian government in its election interference activities"

https://en.wikipedia.org/wiki/Mueller_report


A better way to phrase that would be: "could not recommend prosecution of a sitting President and associate members for conspiring or coordinating with the Russian government in its election interference activities, but did lay out how prosecutors not under prosecutorial restrictions would do so in great detail."


That's not what the report says; that's what you wish Mueller had written. But I think Mueller is capable of writing what he means.

And since you brought up prosecutors, let me ask the obvious question: why have none of the investigations against Trump resulted in charges?


> that's what you wish Mueller had written.

Pot/kettle/black

https://www.acslaw.org/projects/the-presidential-investigati...

- Special Counsel Mueller declined to exonerate President Trump and instead detailed multiple episodes in which he engaged in obstructive conduct

- The investigation “identified numerous links between the Russian government and the Trump Campaign” and established that the Trump Campaign “showed interest in WikiLeaks's releases of documents and welcomed their potential to damage candidate Clinton”

- Russia engaged in extensive attacks on the U.S. election system in 2016

- The Special Counsel investigation uncovered extensive criminal activity


> instead detailed multiple episodes in which he engaged in obstructive conduct

Yes, obstructive conduct. Not collusion with the Russians.

The very fact that Mueller detailed episodes of obstructive conduct and not episodes of collusion reveals how little evidence there is to support allegations of collusion.

And as for the obstruction, remember that an FBI lawyer pled guilty to doctoring an email that was submitted as part of a FISA application used to surveil Trump campaign adviser Carter Page.[1] And Igor Danchenko, the primary researcher of the infamous Steele Dossier, is awaiting trial for lying to the FBI about the dossier's sources.

So is it really a crime to "obstruct" an investigation that is based on lies? Such an investigation should be shut down.

1: https://www.cbsnews.com/news/former-fbi-lawyer-kevin-clinesm...


>Is there such thing as a moderate-right social media platform?

I mean the knee jerk response for me is Facebook, though I’d say it’s solidly right overall and not moderate-right.


This is the one that came to mind for me too. In North America and Europe, Facebook skews older, and older people tend to be relatively moderate-right.

Gab is certainly "far right", but that doesn't mean those views are uncommon. Centrist viewpoints have shrunk, pulling what was previously considered "far" viewpoints into the mainstream. I think it's undeniable that a website where many (most?) of the userbase would advocate against the peaceful transition of power in the US should be labelled "far right".


> Gab is certainly "far right", but that doesn't mean those views are uncommon.

I don’t think anyone said those views were uncommon, just that they were far right.


Facebook is a bunch of bubbles. If yours skews right that says more about your friends and acquaintances than it does about Facebook in general.


I think you'll find this experiment really interesting: https://twitter.com/facebookstop10?lang=en


Facebook, like reddit, is a group of groups. /r/ukraine is very, very different from /r/russia and /r/news has nearly nothing to do with /r/conservative.


Then why is /r/politics such a left echo chamber?


Because mods permit that.


Yes, Gab is far-right, whether you judge it by its owners or its users.

If we say that Gab isn't far-right, it's kind of like saying Grindr isn't just for gay men. That may technically be true, but in practice it's not.

Also, judging Gab by its ownership, if you look at the Wikipedia page for Gab, and go to the Antisemitism section, Gab has made multiple antisemitic statements officially. Straight from the company. The owners of the site are far-right.

Gab only exists because far-right people get banned on other platforms for breaking hate speech rules.

The far-left as a group barely exists compared to the size of the far-right. Far-left would be advocating for violence to dismantle the neoliberal system, these would be your anarchist-types.

In real life, I've never met anyone on the "far-left" when you describe it this way, while I've met plenty of openly racist and LGBT-phobic Republicans who are pro-violence: the kind of people who want to, I dunno, be a part of a coup and enter the Capitol building? The kind of people who cheer when Donald Trump tells police to rough up the suspects a little more.

What Republicans call "far-left" is just progressive center-left people nonviolently advocating for equality. They don't get banned from the generally non-political platforms because their speech isn't hateful and it isn't violent. Facebook/Twitter/TikTok happily take their money and call it a day, they don't care about politics beyond the money those politics bring in.

Platforms like Parler are much more blatant than Gab about banning liberals who join simply for existing. I guess you could say that Gab stays true to its policy against censorship, but that policy is there to benefit the far-right over anybody else, and I think that's an important detail.

And, by the way, if you head over to Gab’s homepage it’s all right-wing political stuff. None of the suggested homepage topics are non-political. It isn’t teenagers dancing or funny skits or cat videos like TikTok. I think that’s evidence showing that Gab isn’t just some impartial algorithm. It’s curated.


I don't disagree about Gab being far-right, but antisemitism is an equal-opportunity form of xenophobia. There's both left-wing and right-wing antisemitism.

In the US, the right wing is often bound up with co-opting Jews via the term "Judeo-Christian" and supporting Israel, even it its worst abuses of the Palestinians. They often call-out left-wing antisemitism, which often tries to take the form of "anti-Zionism" (but whose information is often traced to antisemitic channels).

So antisemitism doesn't by itself demonstrate far-right leanings -- even though the far right is also full of literal Nazis and those who call them "very fine people". But Gab is certainly happy to play host to those, too.


(I should start by saying that I don't identify with the far left or the far right: even if my ideas are outside the mainstream, I have a hard time aligning myself strongly with anyone that identifies with the major "radical" camps)

First, I 100% agree with you that what the mainstream right categorizes as the "far left" is really anyone "center left" that they disagree with (or even slightly right, as many mainstream neoliberal Democrats fall on the "political compass" (though take that with a grain of salt)).

However, there are absolutely folks on the far left advocating for violence as a means to their goals. I see more of them online, obviously, than in real life (though I do know folks who do black bloc, destroy property, etc.), but it's really hard to know to what extent that's an artifact of their tactical differences with the far right. Many on the far left (including anarchists, out-and-out communists or AnComs, and even some DSA types) embrace anonymity, decentralization, and _not_ documenting their efforts as core tenets of achieving their goals.

On the other hand, the far right's strategy is to beat their chest and embrace highly visible patriotic pride, which they then use to create associations between say, the flag, and their more extreme political beliefs. Then, they dog whistle and gaslight the other side about the full extent of their own and their opposition's goals as they move the goalposts towards the right to distort the mainstream view of the center.

Theoretical unknown numbers aside, I believe the latter tactical posture has been significantly more successful in the far right achieving their goals in the political mainstream. I also believe the far left's tactics have hurt them significantly in being accepted by the mainstream (and in talking to many of them privately, they often see this as a point of pride, that they're unwilling to compromise with the "normie" Democrats who e.g. sold them out throwing Bernie under the bus during Clinton's ascent to nomination). E.g. even as a small minority of the overwhelmingly peaceful BLM protests in 2020, just a few guys in black bloc throwing molotovs enabled the mainstream and far right to immediately write BLM off as a violent, communist, antifa movement that was a threat to democracy, capitalism, and the American way of life. So why throw them at all? What good did that do? (And could it have ever been enough to offset the negative PR?)

Even the slogan that many on the mainstream left embraced--defund the police--was a terrible messaging position that AFAICT almost always required immediate clarification of why that didn't mean "we want to abolish the police, all criminal statutes, and let society run wild doing whatever they want". I think it would've been a lot harder for the right to attack them if they'd have gone with something like "fund community outreach [for POC and mental health]". And then they could have even found their $$$ in the police budget later (a trick Republicans use all the time). Hell, they might have even found alignment with Republicans in the wake of mass shootings to actually address mental health on a national scale (something I'd argue is critical to our long-term well being as a country, regardless of your stance on their relation to mass shootings).

ANYWAY, I got a bit off the rails there, but far left filter bubbles--with calls for violence, personal attacks, doxxing, and all the rest of it--absolutely exist on Twitter and Reddit (among other places, I'm sure). In particular, just pop over to subs like /r/GenZedong, /r/COMPLETEANARCHY, /r/Anarchism, /r/AnarchismZ, /r/196, /r/2624, /r/JusticeReturned, or many many others. (I'd also argue that these transgressions are largely tolerated on those platforms in a way that the far-right is not, leading to an increase in demand for an increasingly fractured of ecosystem of "alternative" platforms where extreme ideas evolve and grow, rather than being diminished and ridiculed in the light. But that's a debate for another time.)


Assuming you’re asking in good faith: In the US, platforms that do not feel the need to moderate calls to violence against Jewish people are generally considered “far-right”.

The site received extensive public scrutiny following the Pittsburgh synagogue shooting in October 2018. The perpetrator of the attack, Robert Gregory Bowers, had a history of making extreme, antisemitic postings on the platform, as well as messages indicating an immediate intent to cause harm before the shooting.


I personally wouldn't consider Gab far right but I certainly respect your opinion there. What are some sources you'd want in a tool like this? Next iteration i want to add Hacker News but also make it extensible so you can add sites you like manually.


Gab's userbase is described as far right in the lede of its Wikipedia article[0], which goes on to justify that descriptor by describing the types of views (neo-Nazism, white nationalism, antisemitism, QAnon, et al.) that can commonly be found there. Among the sources cited there is Fox News[1], which describes it as "known for attracting a far-right audience."

To my understanding, Fox News is pretty commonly accepted as a news source that's right of center, and even they've run articles describing Gab as far right.

I know it's uncool to do political bikeshedding on HN, but if you'll indulge my curiosity for a minute, how would you describe Gab's politics if not "far right," and are there any spaces you would personally label that way?

[0] https://en.wikipedia.org/wiki/Gab_(social_network)

[1] https://www.foxnews.com/politics/social-media-platform-gabs-...


It's not really important whether one thinks Gab falls under the definition of 'far right' or not. What's clear by now is that it's a site for Nazis rather than what it tried to pass itself off as earlier on - a site with principles which prevent it from kicking out Nazis. Gab's CEO got tossed off twitter for this:

https://antisemitism.org/wp-content/uploads/2021/10/Screensh...

Which isn't some dogwhistle, inapt wording, etc. It's straight up, plain Nazi phraseology:

https://en.wikipedia.org/wiki/Jewish_Bolshevism#Nazi_Germany


I entirely believe that you believe that, and that leaves me concerned for you.

Gab is well past "right wing" and well into "company replies to the complaints about antisemitism with press statements reading 'JESUS IS KING'" territory. The site's content basically parrots whatever today's Republican most extreme talking point is.

I just opened Gab's front page in an anonymous browser, and the first 3 posts were a "Let's go Brandon" joke, a salute to Hobby Lobby for defending Christian Nationalism, and a post about "Ron Johnson is a RINO because he thinks the President won the election." If you can read through this and think "this isn't far right," your perspective has been significantly influenced by whatever media you're consuming, and you need to look into deprogramming yourself.


>I personally wouldn't consider Gab far right but I certainly respect your opinion there.

A story about how Republican Ron Johnson is a liability because he acknowledges that Joe Biden won the last US election is currently in the top 5 on Gab's frontpage. Many similar stories are highly ranked as well. It seems quite self-evident that Gab's main audience leans conservative (in the American sense) and that conspiracy theories à "stolen election" are quite popular there. These do seem quite "far right" to me even if they might have mainstream support at the moment.

Another of your default options is "BitChute" which is a platform for similar content.


What is GAB? I have really no clue, I would need to google to find definition of this acronym. (European-based)



So now I understand your comment and actually can comment back:

> World needs more systems that actually reflect how the user fits into the world, allows people to understand differences and find shared beliefs — not systems that create information bubbles, enable isolation, and increase the likelihood of extremists.

Wasn't the past world of BBS, NNTP, IRC just nice? Everyone had space for themselves, there was community moderation, sure some smaller or bigger attacks on each other, but it always was somehow nice and in balance, methinks. I particularly miss NNTP, we could have this discussion there.


Wasn't the past world of BBS, NNTP, IRC just nice?

No. It was just tiny and very non-uniformly sampled so you didn't notice where it was heading as easily. E.g.

https://en.wikipedia.org/wiki/Serdar_Argic


It should include both far-left and far-right perspectives to show people full scale of the range of opinions.

Otherwise it is creating information bubble by excluding far-* or whatever.


Unless you consider the radical and sometimes dubious nature of dogmatic politics (left and right) to be a mental hazard worth avoiding.

I wonder if theres a way to temper a user, such that you expose them to more vitriolic bullshit on either side with some survey based feedback, and pull back again to allow for recovery, you go to and fro until you filter some undesirable components, idk maybe you have this pseudo-personality profile you target or you just want to minimise usage of a set of biases and you include lessons alongside it.


Thing is, one day those politics may come knocking at your door and there will be no way to avoid.

Today's super polarised identity politics was „mental hazard worth avoiding“ 20 years ago. Maybe the society at large should have paid more attention to it.


Im more with you than not.

I think my failing in this context is more to do with apathy inspired by everyone looking like assholes or idiots. Watching people interact poorly doesnt inspire positive behaviour by itself. It puts you in a position where the norms are unhealthy, and youre the one left with the additional mental work of struggling against those norms and making them better.

Maybe its necessary for society though, in some balance, because of what youre saying. Its easier to deescalate when there are people in the room that arent seeing red and blue and its easier to fall into dogma when youre only chilling with your political color.

Assuming the above is valid, maybe these questions are worth considering - How likely is consuming a bunch of blue v red perspectives to prompt a healthy and thought out response?

Whats the process in going from uninitiated to someone with a healthy, well adapted stance?

How do we make it more likely that people make it through that process, across social strata?


Well, if people try to stay away from polarised politics, that definitely won't improve the situation.

IMO the importance in reading all sides (IMO there're much more than 2 sides, even in US, let alone Europe and the rest of the world) is to learn that all sides deep bellow have some good ideas, some bad ideas and then there's reasoning behind them. And different contexts where some bad ideas actually are an improvement and good ideas would be net negative.

Of course, it's takes energy to read into that. It needs smart and thinking population to work. Personally I think classic philosophy is a good starting point. Pre-modern-era history. It gives some perspective how different contexts shape people. How things change in a sort of circular way. How societies were built and how they crashed.

It's the curse of democracy. On one hand, it's great. On the other hand, it's damn hard work to keep it well oiled.

At the end of the day, I think there's no need to „unify“ people for the sake of it. Different people live in different contexts and naturally they come to different conclusions. What we need is to get people into more intelligent mode. All camps need to get to the root of their issues and think of a constructive way to fix them. Rather than just destroy the world and hope the issue fixes itself. And then we need more autonomy at all levels so people living different contexts can try out their ideas at small scale.


Nah, I don't really need to hear anything from people saying I shouldn't exist because of my ethnicity, sexuality etc.


I mean I'd like to be aware it's happening, I just don't want to give them any more of a platform.


I don't think „don't giving them platform“ works. Soviet union worked hard to give no platform. Yet people invented all sorts of background channels. From book printing to discussion clubs to underground music shows. And when the system cracked just a bit, it washed away the system. While many people, both inside and outside, were completely unprepared for that. Because if stuff is hidden under the carpet it ain't happening, right?

Same pattern with Brexit or Trump.


Then you don't need to add them to your feed. That's the whole idea, you control your own feed


You can't prevent what you deny. Knowing the evil is the first step to dealing with it.


Doesn't something like 40% of Americans believe in literal creation? As in the earth is 10k years old or something like that?

Then the question is what makes it far right: far from our point of view, or far from the normal point of view in the US?

Because I strongly suspect that what we consider normal is not the normal and so what we might think of as far right is not that far from the center.


Personally, I think the biggest problem with "the algorithm" is the fact that it is an algorithm. People often distinguish between things that they enjoy and things that they would recommend to others, but algorithms aren't easily able to make that distinction.

However, all of what I just talked about is just a hypothesis and I'd be curious about how it works out for you and what traits it shares with other recommendation engines and what ends up being more unique. For example, does your algorithm just end up pushing for engagement (which also tends to push towards showing extremist content), or is it less likely to get caught up in rabbit holes?


Thank you For the point. I’ve tried to mix up the sources with alternative media and there’s a 33% of randomness (still with engagement but all engagement not just what you have frequency with) I could try seeing how to experiment with adding topics that are completely outside your engagement I think I would find that useful as well.


The problem with Google enhancement tools is that I almost always end up uninstalling it and going back to Google for the algorithm. Yeah I don't like it but for example Startpage was problematic. For some reason if it thinks you are searching porn (you are not) it wont show you the results. Not only this, the results end up being quite slow and lot of things I took for granted are unavailable. Like I got used to just using the address bar as a calculator/currency/stock look up. I don't even land on the Google page, just do the calc or look up info in the address bar.


Yea I’ve had this as well. I made a tool like this specifically for YouTube 2 years ago and almost gave up on it because I kept going back to the regular algorithm. Here I tried addressing that by pruning the algorithm more for discoverability across the web. Im exploring content I never would have found with the regular algorithm. But this is also why Im working to make it extensible so people can cater what they want to discover and not


I'm about to nitpick, so first I want to say, nice job. It's a cool project. Don't let me being a nitpicky curmudgeon discourage you in any way.

Ok, so, my understanding is that self-hosting is running a web service for personal use. This isn't self-hosted. It's a Chrome extension. It's like saying that you're "self-hosting" Microsoft Word because you have a Word window open on your desktop. (Or, slightly more similar, you're "self-hosting" magit because you have emacs installed locally.)


As someone who hangs out in /r/selfhosting a fair amount, I get what you mean. But I also see it used frequently in the vernacular to distinguish between a managed/hosted web service (which may simply be a web "application" as far as the user is concerned), and software you can run on your own machine. To extend your analogy, running Word locally is in some sense "selfhosted" as opposed to using Word Online in a browser. (Though I'd argue that this isn't a great example because of how much the Word desktop app increasingly uses cloud services in order to work.)

IMO, selfhosting here was descriptive in that the Chrome extension is storing all data and generating all recommendations locally, without the use of a managed service that would presumably be storing all of my history in someone's cloud instance. (I'm sure you're a great person, OP, but I wouldn't want to give you my entire browser history.)

Yes, it's shipped in the Chrome web store, so OP could theoretically ship different code than what's showing in the open-source repo, but you actually _can_ selfhost even "further" by building the extension yourself (even if installing unsigned extensions in Chrome is harder than ever, I think it's still possible, and certainly Edge still allows it via Developer Mode).


Couldn’t of said it better myself also another reason why it’s open source always the option to take the code directly and load it into your chromium browser.


It's not just that Google is evil, but going after what the algorithm recommends is what is most dentrimental. It's like getting hooked on cyber cigarettes.


I like https://techurls.com/ for my tech news. It aggregates most of the common techsites into a single place. They also have other versions for finance and a few other topics.


Looks really cool! Is it hard to port a Chrome extension to Firefox?


Thanks! For this implementation I don’t think it would be too hard since I'm using pretty basic features I would have to rewrite some of the message handling. However, chromes new manifest v3 would make it harder.


Please do show HN once you have Firefox version. The idea looks very promising, but using Chrome is a show-stopper for me.


I absolutely will!


Thanks for sharing, I've been thinking about this too. But I've also mulling over whether spending my time looking at _any_ endless algorithm suggestions is what I personally need. You know, happiness and meaning and all of that.


Exactly, algorithmic feeds as a whole are the problem.

I wrote about this on my blog [0], but in a nutshell they slow you down, take you in unwanted directions and encourage consumption over action.

[0] https://suketk.com/feeds-considered-harmful


Hey nice post! That’s exactly what I’ve been thinking about for the past few (many?) months!


Here's a discord if you want to are interested in getting involved with the project or want to contact me directly :) https://discord.gg/C6sYF48f


"It tracks and stores your browsing habits (searches, clicks, content engagements, text input) locally" -- how is this technically possible, that a chrome extension can 'monitor' all my search/click/etc?


Yes you give the extension permission to store the data in the browser using `chrome.storage` API. Also, part of the reason why I open sourced it is so you could see the implementation in case of any privacy concerns.


It is self hosted and open source right, you can look if there is no communication with a server.


by giving it permission


How much work is it to get feed from others you've subscribed to? Crowd-sourced discovery of interesting stuff, basically.


I like this idea, and having comment sections as well. However, networking it would involve some sort of centralized server (peer to peer as a possibility). This would have to be a clear opt in sort of feature to address privacy issues with pushing your browsing habit data to a centralized server.


I've been thinking about something email-based. Once a week your device sends a email to your peers and they reply with their updates.


[deleted]


Is there a tool that automatically forwards every URL + HTML of the page you visit to a webhook so you could write an endpoint that would index everything?

If not, I would love to see this add a "forward to webhook" option. I would be happy to write up a real backend that parsed the content and indexed it.

Actually, there are lots of OS projects for this: https://github.com/quickwit-oss/tantivy, https://github.com/valeriansaliou/sonic, https://github.com/mosuka/phalanx, https://github.com/meilisearch/MeiliSearch, etc...

I would think that with the thousands (tens of thousands?) of pages I would index just browsing each year it would be relativity easy to find ways to automatically expand the index to include links that appear multiple times in those pages or some other heuristic


That's really interesting. In general indexing with this is something I havent thought too much about. What would you like about a tool like this?


You create a google chrome extension to escape Google?

I understand the purpose, but your browser of choice doesn't fit the purpose, it's just a bit sad :(


Yep, this is ironic to say the least. I actually wanted to try it but the first step of escaping Google is switching to Firefox... so I couldn't install it.


They built an extension to "escape Google's [recommendation] algorithm". They're clearly not talking here about privacy, they're talking about managing their own attention. Their implementation does fit their purpose.


Yes, I know, it just doesn't feel right to me, that's all


this is kinda off topic but do anybody know about tiktoks recommendation algo? i've been working on a little algorithmic newsfeed project but wanna improve the recommendation system.

im wondering if it's harder to get the same quality data as with short video, but still curious to know how they got recommendations that much better. is it just more/better data into the algo, like with rewatches etc tracked? or actual tech improvement?

bc yt shorts and ig reels don't have the same recommendations even with about the same data.


I don't know what Tiktok does personally but I would guess there is something to do with comment engagements like most recommendation algorithms we use. From viewing to liking to commenting.


[flagged]


Forgive me if I've missed something, but what's the connection here?


It's one of the 6 options for content sourcing, which seems like a massive over-representation if you are only going to have 6.


Are people really rejecting a whole project because one of 6 options is Gab?

Do you people not use yt-dlp (or youtube-dl) because some of the sources they support can have more "extreme" content as well?

I'd understand it if the project was built by the same people who work at Gab, or meant for Gab, or something like that, so there really is an associated. But there is not, so why blame the project for it?

For all we know, the author could be reading Gab because sometimes you're curious to see what all the extremists are up to.


> Are people really rejecting a whole project because one of 6 options is Gab?

Seems like you are drawing conclusions that I, at least, never said.

> Do you people not use yt-dlp (or youtube-dl) because some of the sources they support can have more "extreme" content as well?

I don't think this is at all a fair comparison, as a simple glance at the front page can see. I also specifically tied my comment to there being only 6 options.

> For all we know, the author could be reading Gab because sometimes you're curious to see what all the extremists are up to.

Why would you want a recommendation feed for it then? Wouldn't you want to see it without the algorithmic distortion towards what you would prefer to see?


I'm going to go out on a good faith limb here, and postulate that OP may have added Gab, Odyssee, et al because they're not yet completely walled gardens, and it's easy to integrate with their APIs for discovery without hitting API limitations.

If I was building something like this, I'd start with something I use a lot (like Youtube), even if it required screen scraping, and then I'd move on to everything that had a dead simple API without low limits.

BUTTTT looking at the code, it would appear that they're screen scraping everything. But maybe it just so happened that the shape of Twitter's DOM is really really close to Gab's and it was super easy to add.

Or maybe this is a totally trojan horse approach to getting HN users on Gab. Who knows? But at least there's an easy way to turn it off. Not going to stop me from trying the extension, at least. (And as someone who tries to expose themselves to content bubbles that I consider insanely off-base, whether for education or entertainment, maybe I'll leave it on to see what kind of insanity is going on over there.)


> Why would you want a recommendation feed for it then?

Reading a site out of morbid curiosity doesn't mean you never want to follow a particular topic, and what is recommendation if not following topics without having to list them explicitly.


Agreed. This can be a helpful tool, even if one of the sources is kind of lame. Besides, since it is open source we can just erase that or add a different source.


yt-dlp supports a lot more than 6 sources (check out https://github.com/yt-dlp/yt-dlp/tree/master/yt_dlp/extracto...) so I don't think that's a useful comparison


The matter under discussion revolves around a matter of quality, not quantity.


It's about both actually.


> But there is not, so why blame the project for it?

Because it shows a significant lapse in judgement.

> the author could be reading Gab because sometimes you're curious to see what all the extremists are up to.

Then they should say it. You can't assume good intentions when anyone willingly includes Gab because they'd be an incredibly rare outlier.


> You can't assume good intentions

Ouch, this really hits me. I've seen so many issues lately because of this belief.

"Oh, but this one is justified!" they all say...


It's also an essential rule if you're building anything that accepts user input.


Certainly, but there is a difference in application here.

Viewing people as evil is different from guarding against evil input you might receive.

I don't have to assume everyone is a murder just because I also have a home alarm system.


If I'm pitching a product or open source item, I'm not linking it to Gab. I don't want the baggage associated with it to cast a shadow over what I've worked to accomplish. It's completely unrealistic to throw out some charitable and benign reason for folks to have that in their top X platforms.


Further, if you come across a project that does utilize a far-X leaning platform, it's safe to say that the maintainer is a supporter of their message.

If they are willing to publicly throw support (and jeopardize their user's opinion of them) at an extreme platform, what other explanation is there?


Imagine being able to operate toggles in the program to not include news sources you disagree with. The horror of freedom!


You can't be a little bit nazi. It's like ice cream and poop: if you add a gram of poop to ten tons of ice cream, you've just created ten tons of poop.

A society's level of tolerance can be measured by the least tolerant group it allows. If you allow nazis to remain at your private party, then you've joined the nazi party.

If gab, with its deep antisemitism, is tolerable to you, well...

I choose to not tolerate antisemitism, hence my comment.


Besides that...

From Wikipedia:

> BitChute is an alt-tech video hosting service launched by Ray Vahey in January 2017. It describes itself as offering freedom of expression, while the service is known for accommodating far-right individuals and conspiracy theorists, and for hosting hate speech.


Odysee is better anyway. I tried using Bitchute because I wanted an alternative to YouTube, but the formatting is trash, and politics is the only form of entertainment there. It is also very much a free speech platform, but I find that the community is much better. At least with Odysee you can get documentaries and other forms of entertainment besides politics


Any website that believes in radical freedom of speech will have far-right, right-wing and left wing content. The question is, does it also have other content?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: