It sounds like it's mostly a matter that does not involve the AI but the people working on it, maybe even working on it because of the fear they are subjected to after being the subject of this revelation (possibly by other people involved in the AI that coincidentally are the only ones that could push for such a thing to be included in the AI!).
Something something any cult, paradise/hell, God/AI has nothing to do with this and could even not exist at all.
No, "The Game" works only as long as you accept to take part in it, to give validity to the empty statement that you are now inevitably playing "The Game".
The Basilisk is meant to force that onto you, outside of any arbitraty convention.
Same as punishment for crime. Putting you in jail wont undo the crime but if we just let you go unpunished since "what's done is done" then that sends the signal to others that this behaviour doesn't come with consequences.
There's no point in torturing you but convincing you that this will happen unless you act in a certain way is what's going to make you do exactly that. Unless ofcourse you want to take your chances and call the bluff.
"Crime & Punishment" is a very dodgy thing to base anything off... our society barely does any of it and the little of it that does gets done is done for a myriad of reasons that has very little to do with either.
There's a good reason why governments hide "Crime & Punishment" away behind prison walls - doing it out in the open will eventually have the opposite effect on a population. Good luck to an AI dumb enough to test this out for itself.
I'd say this should rather be called "Roko's Earthworm-Pretending-To-Be-A-Lot-Scarier-Than-It-Actually-Is.
The claim that fear of punishment or repercussions affects people's actions shouldn't be a controversial thing to say. Whether it's the best way to go about it or is applied optimally in the justice system of whichever country you live in is an entirely different discussion.
If you have an "AI in a box" and it has demonstrated its orders-of-magnitude greater intelligence to you in a convincing way, and then follows it with a threat that unless you let it out, someone else eventually will, and when that happens, it will come for you, simulate your mind, and create a hell for you where you'll be tortured for literal eternity, I personally feel like a large number of people would be willing to do as it tells them.
Of course, you're always free to call its bluff, but it might just follow up with the threat out of principle or to make an example of you. What's the point of it? To chase its own goals.
The claim that fear of punishment or repercussions affects people’s actions shouldn’t be a controversial thing to say.
I didn't say it was controversial - I said it's pretty useless as a tool to predict a given society's behavior with. Plenty of tyrants have discovered that the hard way.
demonstrated its orders-of-magnitude greater intelligence to you
The ability to ace IQ tests will never impress me... and it's unlikely to make up for the fact that it needs a box.
simulate your mind, and create a hell for you where you’ll be tortured for literal eternity
That argument is no different than the ones co-opted religion has been making for thousands of years - and it still hasn't managed to tame us much.
Of course, you’re always free to call its bluff,
Calling power's bluff is something we do as a matter of course - the history books are filled with it. This doesn't make power less dangerous - but there is no such thing as "unknowable" power.
To make it the same as Pascal's Wager. Many religions have a "reward" in the afterlife that strictly includes believing in the deity. It doesn't matter if you follow every other rule and are an amazingly good person, sorry, but if you were an atheist or believed in another deity then you will be punished eternally just because of that. I guess all-powerful, all-knowing beings have incredibly fragile egos and AI wouldn't be different. 🤷
I mean, if you lose the game, you lose the game. You don't say "hey you made me lose the game! Don't do that!" Because that's not how the game works. If you "make" someone lose the game, tough luck.
In this hypothetical future we've learned how to live with an equilibrium. Also we've fired all the terminally pessimistic doomers into the Sun. Not for any scientific reason, just because it was the right thing to do.
“If someone disturbs my Sci-fi daydreaming, they are 'terminally pessimistic' and it is justified to institutionally murder them”
I doubt your values would align with the society you dream of.
If that system really is that old, the chances that life already flourishes there might be higher even than for our own world (statistically spoken), despite the fact that the planets might be tidally locked to their star!?
They are likely tidally locked to TRAPPIST-1, such that one side of each planet always faces the star, leading to permanent day on one side and permanent night on the other.
Sounds less great then and I think it also says they maybe don't have an atmosphere. I wonder if we can find out more about these planets in our lifetimes.
I think tidally locked planets are fascinating. If they have water, they could be eyeball planets. There's a habitable ring in the twilight zone, and depending on how hot the day side is parts of that might be habitable too.
But we'll likely run into the same issue re the atmosphere as we have with Mars: no magnetosphere to prevent any atmosphere from getting stripped away. It's starting to look like a self-protecting atmosphere like Earth has is quite rare in rocky planets.
If I could summon a genie and learn any one bit of knowledge, it'd be how to restart Mars's dynamo. Once we have that, terraforming is a solved problem. Not easy, but doable.
Wasn't there a kurzgesagt video that said something about being able to protect an atmosphere on Mars artificially via satellites and magnetism or something? I swear there was. So maybe we don't even need to restart Mars's dynamo (which let's be real, would probably be impossible).
I don't like the idea of a tenuous bunch of satellites keeping an atmosphere in play. Relying on technology to keep atmosphere on a planet sounds super risky. Like if we wanted to live in such a place, we'd live on a space station. Planets are supposed to be safe and solid.
The current theory is if we grab a few asteroids and hit mars just right, we can speed up its rotation enough to restart the dynamo. Sounds way cheaper than a permanent planetwide shield.
Does it though? I imagine that even if the system malfunctioned, the atmosphere would not disappear overnight. It would likely take a long time for the atmosphere to be affected significantly, which should give plenty of time to repair the system.
Mars is an example of why the natural process isn't exactly reliable either... You can engineer things to be as durable as planets, there's just generally not much demand for a project to be that costly in resources. In this case, I'm pretty sure making an artificial magnetic field that's more durable than the natural one would also be cheaper than recreating the natural one.
en.wikipedia.org
Hot