lol

I hadn’t even been aware of this community until someone in FuckAI said I should x-post here

that’s good advertising

oh im in favour of both, see here from the above:

This is all fine, by the way. I think it’s actually good to miss the point of Roko’s Basilisk and just laugh at it. However, it’s also useful to have a deeper theory.

what you’re describing is the old SneerClub method, whose effectiveness was proven in the subreddit’s height with a large number of (some quite significant) testimonials

my point is that there is something deeper at work here as well, namely a political economy of virtuous selfishness which latches on to ideas like Roko’s Basilisk to give itself an eschatology and therefore meaning

I found myself using this as an opportunity to write a commentary on commentaries on Roko’s Basilisk at large, summarising some thoughts that I’ve had for years about how people read it. I was surprised that I found myself even wanting to! So ahead of my disagreements below, thank you for the essay.

What most people don’t understand is that Roko’s Basilisk was such an effective argument because Yudkowsky had strenuously argued for cryonics in the Sequences on the grounds that if you were revived after death, any repetition of your brain’s underlying quantum pattern would be consciously continuous with you

It isn’t thrown into Roko’s Basilisk at random, nor is it thrown into LessWrong theology at random, it’s a metaphysical cornerstone of their ethics and practical philosophy

The reason that it’s incorrect is also boringly philosophical: the argument relies for one of its premises on denying continuity of consciousness; therefore, the conclusion is inconsistent with its own premises and the argument is invalid.

It COULD be rescued pragmatically by rendering the NEW commitment to continuity of consciousness in different terms as, for example, an issue of subjective probabilities (like Pascal’s Wager, “between definitely dying and maybe achieving continuity, what do I have to lose?”) and I suspect that this what a lot of people who notice the problem do, wittingly or not. This solution is also built into Yudkowsky’s practical philosophy, which is replete with wagers of this kind.

In any case, large enough numbers of people evidently buy large enough chunks of LessWrong metaphysics to also buy this aspect of Roko’s Basilisk, to the point of framing it as a genuine infohazard (and then pretending not to have done so later on).

So I think Galileo’s Basilisk is off the mark, and the way that it’s off the mark is illuminating about rationalist philosophy.

2a.

There is a stroke of real genius to the way that Roko constructed his Basilisk out of some of the metaphysical toys he had just lying around near to hand, and most crucially shared in common with other LessWrongers. Indeed, the most glaring logical problem (which I articulated above) is off-loaded onto Yudkowsky, and the rest of the logic is basically acceptable to LWers and extremely simple to follow.

Galileo’s Basilisk, however, asks us to add EXTRA premises to this simple formula. And this works if we think that the continuity of consciousness idea the original relies on is just unmotivated woo (it is woo, but it isn’t unmotivated), the way that G’s interlocutor introduces an unmotivated auxiliary hypothesis to save the old theory of the spheres, permitting G to satirically repeat the same move. But from the perspective of rationalist metaphysics, it is GALILEO who first introduces an unmotivated auxiliary hypothesis, because we do not know how a superintelligence would emotionally handle its intelligence.

Meanwhile, the inferential logic of Roko’s Basilisk is comparatively bounded and secure.

The same problem emerges with Comrade Basilisk: we have to take several inferential steps along the way, through speculations on the value of experience, before we EVENTUALLY get to the classical wager ‘or you’ll burn in hell for eternity’. Rather than playing on beliefs that are ALREADY THERE, it burdens that wager with supporting those inferences (“believe or burn in hell”), whereas in the original the wager leaps naturally out of existing premises (“ACT on your beliefs, or burn in hell”). More on this in 2b.

(As an aside, I think the emphasis on WORKS over FAITH plays an important role in triggering the guilt reflex).

2b.

All of this finally triggers an important distinction to be made with respect to Rationalist vs Christian eschatology, and your treatment of Pascal’s Wager. In effect, LessWrongism logically constrains the kind of God implied by Roko’s theory, so there’s nothing arbitrary going on in deciding what God.

Rationalist sociology is essentially Mandevillian (or Clintonian, for that matter). Their essential model of social justice is that greed (along with other low motives) either produces good outcomes by itself or can/should be leveraged to produce good outcomes. Secondary to this (and emerging chronologically later within the movement) is altruism, which unlike greed has to be properly channeled from the very start, lest any charity be misplaced - a line of thought which progresses rapidly to Building God as the highest ideal in its own way. (by contrast, greed tends to at least create wealth even when unchanneled)

From this point of view, Roko’s Basilisk comes up trumps once again for sheer simplicity. The first human instinct is self-preservation. The second human instinct is altruism.

Contrasted with Comrade Basilisks’s burdening problem, the logic is crystalline. I already believe that my life could be almost infinitely extended by simulations in a superintelligence, all it takes now is for somebody to point out that that superintelligence has an arithmetically plausible reason to throw me in helljail if I don’t put in the work while I’m mortally living. This particular God springs fully formed from rationalist metaphysics plus the Mandevillian see-saw between self-preservation and altruism which insists that the greatest good comes from the leveraging of that instinct for self-preservation.

Whereas Comrade Basilisk does the same on one level (helljail), it still faces the burdening problem, of which some more detail:

Part of the genius of Roko’s Basilisk is that the work the sociology does is loaded on the world BEFORE heaven: we live in an imperfect world whose imperfections we can EXPLOIT (leverage) to get to heaven.

Comrade Basilisk’s eschatology frames the imperfections of our world AS the problem. For Roko, you CAN get to heaven through the eye of a needle, in fact it probably helps to be very rich to build the superintelligence. There is no such easy route in Comrade Basilisk’s world, since part of the very PROBLEM is the great hoarding of wealth.

This is also what distinguishes Roko’s / America’s Jesus (who wants you to be rich, fat, and happy) from (Classical) Christian Jesus (a messianic Jew arguing the virtues of poverty).

Plausibly, there ARE parodic versions of Roko’s Basilisk with more competitive claims on Ockham’s Razor, but I haven’t seen any.

This is all fine, by the way. I think it’s actually good to miss the point of Roko’s Basilisk and just laugh at it. However, it’s also useful to have a deeper theory.

I think that this ultimately points to a more complex engagement with political reality when it comes to the “what do we do” question. LessWrongians have a theory of political economy, which I described above at the three levels of greed, greed leveraged, and altruism, in descending levels of importance. The crucial and most deeply appealing feature of this vision is that it is virtue minimalist, and at the base layer essentially indulgent and even encouraging of a wide array of baser impulses.

Rival visions demand a bit more. They demand, for example, giving up your free time in comradeship and some of your secular ambitions in a recognition that the private accumulation of wealth is bad for the wider public and the world. For the fanatics, of course, Roko’s requirements are a bit stronger, but that is - of course - why they are the elite, and for the great majority of people the Basilisk is nothing but an infohazard.

Maybe you’re beginning to see where I’m coming from, so I’ll just say that it would take a whole other essay to outline that the problems with Roko’s Baslisk are less problems with LessWrong than they are the with political economy of liberalism. That’s an absolute clanger to finish on but I really don’t have any more space or time. A next line of thought would be to outline how any of this ties to the dream of immortality.

For what it’s worth, my personal preference is for a virtue maximal anarchist solution. Clean your soul!

By and large no. Read the comments under anything on LessWrong, for example, and it’s trivial to pick out the vast majority of nominally substantive posts lighting on the one thing that got them mad, just like you and I, in amidst a chorus of nothing remarks equivalent to “so brave, so powerful”. They’re just people man, after all.

Notice that the disagreements people get into by and large evolve the same way as reddit fights - everybody’s just waiting for their turn to nitpick some sentence or other that (nominally) deserves a fair, contextual, interpretation it’ll never receive.

Honestly I’ve made it my life’s purpose to be a loser by the standards these people set, and succeeded beyond either of our wildest dreams

TracingWoodgrains is an out-and-out rationalist. Long time poster on /r/slatestarcodex and heavily involved in all things SSC. It just benefits them to be coy about it. Which is whatever! Fine! Who cares? But they’re 100% in the bag for rationalism in any way that matters.

Honestly, anyone who read the essay can see that the question of whether Yud approves is totally irrelevant to its thesis, but these people are incapable of reading styles of argument that don’t proceed by declarative statements about binary choices, except of course in those situations where something precious to them (such as their good standing at the “not a racist” club) might by at stake

It’s a combination of those things.

Because rationalism the coherent phenomenon was founded with the more or less explicit intention of building a cult, Yudkowsky’s original rule-set incorporated all of the basic cult rules, which every cult leader tends to be able to work out mostly for themselves by looking at what they outwardly want to build (a movement) and what they inwardly want to do with it (retain personal power over that movement)

So, for example, the particular way that Yudkowsky frames “objectivity” coalesces later on around the “object level” vs “meta level” dichotomy, “low” vs “high” “decoupling”, the “grey tribe”, but it’s there from the beginning in his insistence on the highly specific and idiosyncratic framework proposed in The Sequences, his constant explicit insistence on the rarity of his chosen elect, and also just in (a) his consistent lambasting of people who work outside that framework in the text of The Sequences themselves, and (b) his sometimes hilarious neg/love-bombing of the reader

Of (b), my favourite example is that passage where he bizarrely takes an unnecessary moment to call you an idiot if you think that there’s a universal clock measuring time throughout the universe, in the full knowledge that his nerdy readers are aware of relativity

So the whole system, beginning with LessWrong’s very founding, is geared to control the framing in ways like not naming names. Naming names is a failure of objectivity, because it brings in the sorts of particulars that might exercise your ordinary human judgement - ordinary human judgement is bad, we know this from Daniel Kahneman, and that’s another rule of objectivity. So, moreover, the whole system is geared so as to keep “objective” framings which favour HBD “in-group”, and to displace good human judgements (‘Richard Hanania is a ridiculous mendacious racist’) into the “out-group”).

HBD hegemony within the movement (in influence if not in numbers), moreover, could not but have been the eventual outcome of the same rule-set. In spite of his own protestations, Yudkowsky’s pugilistic naturalism was sufficiently both insisted upon and theoretically naive as to ultimately yield hegemony to the HBDers by sheer inertia: once you have eliminated and salted the earth of any thinking which fails to embrace the most childish physical-scientistic reductionism, then when your rules for thinking enter the arena of politics (especially American politics) and human biology, you have already ceded all possible theoretical ground to HBD, and any counter-weight you try to introduce thereto becomes the pathetic mewling of Kahnemanian irrational beliefs. Your rhetoric already implied “it’s just basic biology” from the very beginning.

So, for anyone keeping score, the only way for anyone on LessWrong to win the rhetorical argument is, unfortunately, just to be normal, and violate one or more of the LessWrong standards for thinking.

It’s true though. They’re not nazis. They’re incapable of being fired by any fundamentally political or spiritual ideals, no matter how ultimately black and nihilistic, at all. Even if these people were full-throated card-carrying members of the American Nazi party marching through Times Square with a swastika flag throwing out copies of Der Sturmer from a Panzer tank they wouldn’t be nazis. The fact is that they’re just the purest distillation of 20th-21st century media culture yet: they’re so utterly saturated in media that the only choice they’ve made, the only choice available to them, was whether to lean into the goodie or the baddie vibe, and they plumped for “baddie” because it suited their contrarian aesthetic and then, without even leaving a ripple on the surface, they slipped into the role and inhabited it so thoroughly that it is, literally, indistinguishable from who they are.

These people are nothing less, and 100% nothing more, than your childish glee at getting to play the villain in an RPG.

Unbelievable kill shot, how the fuck did Davis leave it on this? Some secret agenda to hand Metz a fuckin’ victory wreath? Does he think this makes Metz look bad?

CM: What his argument to me was is that it violated the ethics of his profession. But that's his issue, not mine, right? He chose to be a super-popular blogger and to have this influence as a psychiatrist. His name—when I sat down to figure out his name, it took me less than five minutes. It's just obvious what his name is. The New York Times ceases to serve its purpose if we're leaving out stuff that's obvious. That's just how we have to operate. Our aim—and again, the irony is that your aim is similar—is to tell people the truth, and have them understand it. If we start holding stuff back, then that quickly falls apart.

I get that out front Davis’s whole thing is total transparency, but if that’s really all that’s going on here, how did it not end on something utterly banal? How is this orbital homerun the end of the conversation?

Identical experience. Goddamnit kid just get drunk and write a bad poem

It appears that many of you have been hiding full blown hardout sneers from SneerClub, and I am baffled as to why