Once upon a time it was possible to identify "good" pages by how often they were linked - effectively crowd-sourcing the problem.
Natural selection: sites have learned to manipulate the game and the crowd.
Under pressure to optimize metrics that lead to SEO and better valuations, the internet is getting less useful from a user perspective.
I don't want to watch a video/slideshow, download an app, register for a forum, or read through a 2000 word fluff piece with interspersed ads and links to more information in which site has buried the one-sentence answer to a 2-second question in order to maximize my time on the site.
This garbage is what makes it to the front page of google these days. I guess the poor sites that aren't user hostile just aren't SEO'd enough.
Metrics are destroying lots of departments in lots of companies. Everyone seems to fixate on the analytics for this week or maybe this quarter, there is little or no attempt to understand the why or think long term. So many companies have cut back severely on the sorts of risky investments they used to make regularly, because the data is being conflated with reality
What's strange is this is not at all a new problem, and Google has historically been quite good at fighting it.
The original search engines ranked pages by their content. Naturally this led to gaming by including keywords (remember huge invisible sections of pages that just repeated keywords thousands of times?).
Google's original PageRank algorithm was a complete breakthrough for this, almost completely disregarding content and instead ranking results based on the text other pages had used to link to the page. This was so good, in fact, that most of the other search engines from the time didn't survive.
This once again led to large-scale gaming with techniques like link farms, and a small arms race as Google came up with ways to squash these new techniques. The quasi-legitimate "SEO" industry spun out of this.
I think now we're in the same cycle again, and at this moment scam sites are winning. What's to be seen is if there's a looming breakthrough, or the arms race will continue.
I think a big problem now is that the sites are capable of tricking people. That is, if you have a "turing test" for reliable content website, blogspam.forums.net with 2000 backlinks and such obviously doesn't pass. On the other hand, Reuters.com does. But so does theonion, and Breitbart and random_russian_conservative_propoganda_site.us, and so it becomes a lot harder to differentiate the good from the bad without explicity curating by experts, because the sites can trick non-domain-expert humans, so what hope does a bot that hasn't tried to learn domain expertise of the topic have?
A good example of this is mailing list archives. Pages and pages of google links searching for mailing list posts are filled with maximum ad-cram (often dressed up like a fake forum) garbage, meanwhile the clean, no bullshit mailing list achive everyone links to (marc.info) is nowhere to be found at all.
This is likely because marc.info isn't playing the SEO game, and spamsite #43254 is.
Once upon a time it was possible to identify "good" pages by how often they were linked - effectively crowd-sourcing the problem.
Natural selection: sites have learned to manipulate the game and the crowd.
Under pressure to optimize metrics that lead to SEO and better valuations, the internet is getting less useful from a user perspective.
I don't want to watch a video/slideshow, download an app, register for a forum, or read through a 2000 word fluff piece with interspersed ads and links to more information in which site has buried the one-sentence answer to a 2-second question in order to maximize my time on the site.
This garbage is what makes it to the front page of google these days. I guess the poor sites that aren't user hostile just aren't SEO'd enough.