If you want to build a recursively self-improving AI, have it go through a billion sequential self-modifications, become vastly smarter than you, and not die, you’ve got to work to a pretty precise standard.
Rationality is the master lifehack which distinguishes which other lifehacks to use.
A burning itch to know is higher than a solemn vow to pursue truth. To feel the burning itch of curiosity requires both that you be ignorant, and that you desire to relinquish your ignorance.
I don’t want to rule the universe. I just think it could be more sensibly organised.
Through rationality we shall become awesome, and invent and test systematic methods for making people awesome, and plot to optimize everything in sight, and the more fun we have the more people will want to join us.
This is one of theprimary mechanisms whereby, if a fool says the sun is shining, we do notcorrectly discard this as irrelevant nonevidence, but rather find ourselvesimpelled to say that it must be dark outside.
I’m wondering if there’s a spell to make lightning flash in the background whenever I make an ominous resolution.
After all, if you had the complete decision process, you could run it as an AI, and I’d be coding it up right now.
If you want to maximize your expected utility, you try to save the world and the future of intergalactic civilization instead of donating your money to the society for curing rare diseases and cute puppies.
If you don’t sign up your kids for cryonics then you are a lousy parent.
Maybe you just can’t protect people from certain specialized types of folly with any sane amount of regulation, and the correct response is to give up on the high social costs of inadequately protecting people from themselves under certain circumstances.
There are no surprising facts, only models that are surprised by facts; and if a model is surprised by the facts, it is no credit to that model.
Existential depression has always annoyed me; it is one of the world’s most pointless forms of suffering.
Since the rise of Homo sapiens, human beings have been the smartest minds around. But very shortly – on a historical scale, that is – we can expect technology to break the upper bound on intelligence that has held for the last few tens of thousands of years.
We tend to see individual differences instead of human universals. Thus, when someone says the word ‘intelligence,’ we think of Einstein instead of humans.
There’s a popular concept of ‘intelligence’ as book smarts, like calculus or chess, as opposed to, say, social skills. So people say that ‘it takes more than intelligence to succeed in human society.’ But social skills reside in the brain, not the kidneys.
To confess your fallibility and then do nothing about it is not humble; it is boasting of your modesty.
What people really believe doesn’t feel like a BELIEF, it feels like the way the world IS.
When you think of intelligence, don’t think of a college professor; think of human beings as opposed to chimpanzees. If you don’t have human intelligence, you’re not even in the game.
Intelligence is the source of technology. If we can use technology to improve intelligence, that closes the loop and potentially creates a positive feedback cycle.
Singularitarians are the munchkins of the real world. We just ignore all the usual dungeons and head straight for the cycle of infinite wish spells.