I think it's much more impressive that stuff that was added in 2018 and 2019 has a much higher probability of being deleted today than if it was added 2017...
Wonder if that has anything to do with covid and maybe new businesses models opened 2 years before failing and therefore websites of this companies disappeared.
Also I think it would be nice to see a graph of new websites being opened other the same time span.
I mean, that kind of makes sense. A lot of small websites are probably for temporary projects, or may even be experiments. When the project ends, it usually makes financial sense to quit paying for hosting and domains.
I’ve often wondered what the implications of the internet will be for future historians. On the one hand, there is now an enormous body of writings from not just the educated elite as in the past but from all sorts of ordinary people, which is something that has never really existed before.
On the other hand, how and for how long will these writings be retained? If we stop writing things on paper, will these digital writings become completely inaccessible at some point? Could we have a situation where there are almost no writings from a certain period down the road? That would be unfortunate.
There are so many way to adequately protect digital information from solar flares. That would be the least of our problems, the actually dangerous part of geomagnetic storms is the severe power outages and the severance of the electrical grid.
Freely licensed works will be preserved a lot better because there will be more copies of them.
Likewise the fediverse is a step in that direction: this message will be federated to hundreds of servers so is more likely to survive longer than if I posted it to reddit.
Already a lot of stuff is becoming one harddrive failure away from being lost forever. Companies don't care about preserving content, so it's largely up to random people happening to have saved a copy of something for it to still exist at all.
I believe it's often because nobody does their own website anymore but instead uses managed services, e.g. Medium. Or bits of information, that would've been worth a blog post some while ago, end up on sites like StackOverflow, Reddit, etc.. And once these services want to monetise these contents, they usually start with limiting public access.
And OTOH TikTok, Instagram Reels and YouTube Shorts are doing everything they can to further limit people's attention spans and get them addicted to those services. So the people capable of and/or interested in producing proper "content" are dwindling, too.
If the information was important wouldn't it already be passed around and expanded upon? The Internet is probably 99% junk, at least the posts I've made. Only the good stuff like goatse survives.
Problem is, people rarely realize the importance until they're lost. Plenty of posts from 90s and 2000s containing valuable insights are probably lost forever. Remember that not everything online is in English, either.
Finding sources about Bush and Cheney fuckery from 2000-2008 is getting increasingly difficult. Their crimes are getting memory-holed.
EDIT: Specifically, does anyone else remember the specific act that Bush wanted to hit Quakers with terrorism charges over? I remember it being a bunch of Quakers in kayaks doing a blockade of a naval ship, preventing it from leaving port to go to Iraq. I can't find a fucking word on it anymore, and I can barely even find sources on Bush wanting to hit Quakers with terrorism charges other than some broken links at the ACLU. Quakers, as a reminder, are the only religious group in the USA that are default conscientious objectors because violence is 100% antithetical to their religion. These are the kind of people they wanted to use "terrorism" charges against.
From a historical or intellectual archaeological perspective, no one in 2000 BC Babylon thought their pottery would be of historical significance, but 4000 years later, it is. These websites, particularly ones independently created and maintained by hobbyists, are snapshots of the ideas of the time and people that created them. These websites may not have been intensely popular, but they were in many ways a foundational part of the inchoate tapestry of the internet that would eventually become the "modern web."
On the flip side, nobody can be expected to keep their website up for 4000 years. Hosting costs money and time, and at some point, the thing you're hosting will fall out of relevance enough to no longer be worth the cost.
This is why archiving is important. Hopefully most of the content that was lost was archived at some point. Getting a good chunk of that content onto long term storage would do future generations a favor (even if it's just a bunch of tape storage locked away in a warehouse or something).
This is true. Right now the OG internet is sort of kept alive by oral history, but we have the technology to save these websites in perpetuity as historical artifacts. That might be a good coding project - a robust archiving system that lets you point a URL at a webpage and scrape everything under its domain and keep a static collection of its contents. The issue, though, is that this doesn't actually truly "capture" many web pages. A lot of the backend data that might have been served dynamically from a database isn't retrievable, so the experience of using the page itself is potentially non-archivable.
That's pretty interesting. It looks like they define inaccessible links as urls that get a 404 or the server doesn't resolve.
I wonder if there are any real implications of this. We seem to know it and work around it in some cases, e.g. StackOverflow saying answers need to contain quotes from pages they reference.
For some real-world examples of this issue, you can look at how the only reason we have any of the early BBC news reels and TV shows is because of copies recorded by people on their TVs. The BBC reused the tapes that they recorded on for new programming to save money on buying tapes. When they started to think about the preservation of news and shows like Dr. Who, they had to turn to the general public and ask them to donate any recordings that they might have made.
It's estimated that more than 50% of all video games are lost forever because companies didn't care to save a master copy, and this has already come back to bite some of these companies in the ass with the recent trend of remakes and remasters. There was a recent remake of one of the GTA games from the early 2000s that was very poorly received, and it turned out that the company who worked on it only had the mobile phone port of the game to work with because Rockstar hadn't bothered to keep a master copy of the game. There was another recent remake of a game that was very obviously done using a pirated copy of the game as the source, because they hadn't even bothered to remove the cracker's logo from the game.
With examples like that and Sony recently removing thousands of people's access to music and movies that they bought on basically a whim, it's pretty clear that preservation efforts will be done in spite of companies rather than helped by them. And so that means copies of things will be one random harddrive failure of some single person on the internet away from disappearing forever.
I've nothing really to add but the gta game was San Andreas and Take Two replaced the already functional ports with the mobile version so all that's available now is the shitey mobile version. I own/owned it on PS4/5 and PC and now I don't play it at all because if I redownload it I'm getting the mobile version.
Well, websites can still be made, so not quite the same, but I get what you mean:-).
And similarly, a ton of written material was lost e.g. when a library was burned - often unique or rare material subsequently lost in other ways, so very much the same process.
The difference is that most of that content lasted for at least a few decades, if not centuries before being lost to time. As content on the internet is 'destroyed' if no one hosts it any more, a lot of valuable content is being lost in just a few years after being created. Archiving needs to be more widespread and better supported if the resources and culture of the internet as it has evolved over time are to be preserved for posterity.
Some government should finally grow the balls to reform copyright, it's insane that basically the whole world uses this broken system that, among other things, makes archiving illegal
The thing is, we can do better, it is not a technological problem as during the analogue/paper age with chemical degradation, it is a societal and legal issue.
It's a technological and a physical issue. We just can't store every bit of information plus a picture of everyone's cat. We can't guarantee that no information ever gets lost. We've also not really stored and archived every shopping list, advertising, pamphlet, silly poem, ugly drawing etc. since the time of the printing press and that's okay.
It might be a good idea to store and archive some written material as time passes but we want to be a bit picky about what we store. That said, I wouldn't mind to find more shopping lists and less posh documents in museums.
I wonder how much of this stuff may still be around on harddrives somewhere. Random blogs probably not because they were using shared hosting that would overwrite and reuse the space when the blog went down, and typically destroy the drives when the servers got decomissioned. But maybe large platforms like Geocities might still be archived somewhere.
Probably a good bit. I have a backup from my personal website from 2006 to about 2013. Along with a lot of media going back to my teen years. I'm really lucky I've been a good digital steward of my own data and haven't lost almost any of my personal digital history.
I wonder how much of this stuff may still be around on harddrives somewhere.
Probably quite a lot!
Just as an example, I'm a part of an art and writing focused community that's been around off-and-on since the late 90s. Typically each member has/had their own website. So a few years ago when we went from an "off" phase back to "on" again, a major project became reconstructing the stuff that used to be on Geocities, the various smaller platforms of the 90s and 00s, and ISP-provided webhosting. And obviuously it's hard to judge how much stuff we don't remember and therefore don't know we're missing, but well over half of what we have reconstructed has come from "I found my external hard drive from 2006 and it had X, Y and Z on it!" I personally had ~3000 files sitting on my NAS, which I had moved off my own hard drive at some point, but had been unwilling to delete, so I just dumped it into long-term storage. Four years into the reconstruction project, we still occasionally find files we thought were lost forever, usually when someone's found an old hard drive in a box in their attic/basement. The found content often was created by someone else, but downloaded and archived by the hard drive owner.
Although this is representative of just one community, given how apparently common it was for people to download offline copies of websites they liked, it could well be that large swathes of the old internet are sitting on people's hard drives, waiting to be rediscovered.
Some web pages are extraneous, fedundant, or only relevant for a limited period of time. A sign up page for a concert doesn't need to exist permanently. Consolidating a large website down to fewer pages that are accessible for everyone is a good thing.
Archiving services that retain web pages that deserve saving are how we should retain that history of the web, but the actual creators don't necessarily need to indefinitely maintain a web page that becomes obsolete.
Yes, a lot is lost that could have just continued to exist and archiving is good, but getting rid of clutter is not a bad thing.
Yeah but Amish Rakefight GFY (a dedicated page you sent someone a link to which then toñd you to Go Fuck Yourself -along with a counter to twll you how many people were told to fuck off) has been gone for a few years now....and damn i miss it.
But how do you know if it's the clutter that is being removed? One of the indicator listed in the article is broken links from wikipedia. These links are very likely to point to informational resources or news articles, but are also being lost at a high rate.
You are trying to reach neo. Please select all the pictures with crosswalks. In order to contact an AI agent instead, you must agree to our data policy.
That's an exaggeration. We had nice things back then with forums and ICQ\AIM\others, which we don't have now, but the tech allows us to have them. It's the society that has degraded.
The technology is working against it too. App search engines are just spam ads now and will never find that niche forum that has what you are looking for, like they once did 20 years ago.
I think a lot of stuff got adjusted when mobile became a thing.
Instant message apps just got replaced with Whatsapp, Signal, etc.
Monetisation is a huge problem. Nobody ever expected to make money off a wonky old webpage they made in HoTMetaL, or a MySpace page. Now everyone is on homogenised platforms, they're quite happy to accept bucketloads of money to project whatever you have to say to the masses, and none of it is good. All the hate you see out there isn't society. It's money.
Instant message apps just got replaced with Whatsapp, Signal, etc.
Alternately, Whatsapp, Signal, etc... are instant message apps. I'm a little surprised none of the messaging apps that had been popular on PCs managed to stay popular on mobile.
This is the main thing that happened, I think. I met some old friends recently I hadn't seen in a while and it's wild how differently we engage with the internet. My main source of interaction is on a laptop, and even then a non-trivial amount of my web interaction is purely via the terminal. Of all of my friends, one of them had a PC, and they don't use it. Their engagement with the internet is purely on mobile devices. I was dumbfounded. Like...how do you do stuff on a phone. I hate phones. They're so much worse than a good keyboard. But I also hate the current version of the internet and they seem to love it.
And that, I think, is the core difference. It's not that the phones took over, it's that the keyboard died for the average user. A keyboard allows a complex degree of engagement that is difficult, if not truly impossible, to match on a device meant for short bursts of canned responses and auto-complete suggestions. It forces individually brief, but ultimately continuous pre-programmed engagement.
And that's the entirety of the modern internet. It's why tiktok is so popular. It's why youtube shoves Shorts down your throat when you visit. It's why Twitter took off. It's also why a website like reddit, that was based initially around the kind of engagement I like, is so hard to monetize and why the attempts at dumbing it down and strangling it of anything that isn't that same kind of superficial engagement (and by God are they trying) is so difficult for the website's leadership: because all the other places that are more profitable than it are designed to do that from the jump, and they have to superimpose that strategy onto a content aggregator whose main attraction was a robust, nested comment system.
I keep thinking about what was, for me, the Golden Age of the internet. I know it's different for everyone, but from around, I guess, 2009 to 2017 I was online a lot. And a lot of what the internet was and how it operated and the ideas there, especially on reddit, were so formative to who I am. And I keep feeling like I never appreciated it or really thought about how vibrant and interesting it was while it was like that. It feels like when you're a kid and you see a wave for the first time, and it's building and building and it seems like it'll be building forever, getting bigger and bigger, but then suddenly it collapses under its own weight and is gone as if it were never there, and after the fact you just wish you'd appreciated it for the wonder it was in that moment. Part of it's just getting older and the general feeling of nostalgia that comes with age, but sometimes that nostalgia is justified.
All the hate you see out there isn’t society. It’s money.
I know, right? No matter how rude I am on one forum I still frequent, or in a groupchat of friends, or in family chat, it just dissolves because everybody wants to understand each other. In "global" social media it's some PvP, as if people didn't have boxing pears to vent their frustration there.
Forums are still around. People just got lazy and started using reddit instead. Search engines are also to blame since they don't bring up smaller forums in search results. People can go back to forums if they want.
That's the point. One can think what they want, decide and go there. Or one can just leave their home and walk with the crowd wherever it brings them. (A metaphor.)
Social media make it appear like you could live a life like the latter. Doing all things you need. Just walk with the crowd and never decide.
People are afraid of even the tiny bits of freedom, when they are looking at it from that human stream which never leaves them. They think that outside of it, if a decision they make is wrong, they are lost.
It's psychological, all of it, like a very subtle and less deadly slaughtering block.
pewresearch.org
Hot