Fast Losing Ground

I’m writing a series of posts summarizing my position on the Intelligence Explosion, and here I want to give a couple of examples of recent AI developments which should make even hardened skeptics consider the possibility that our creations might soon catch up with us.

But first, I want to point out that while the history of early AI research is marred by over-confident prognostications that wound up not panning out and causing several “AI winters”, it is also true that AI skeptics have a history of believing that ‘machines will never do X’, only to have machines do X not very long thereafter.

This is humorously captured in the following cartoon, attributed to Ray Kurzwiel:



Most of us are fast becoming acquainted with living in a world suffused with increasingly smart software. But many would be surprised to learn that there are computer programs in existence now which can write compelling classical music. Emily Howell is the product of several decades work by David Cope, who conceived of the idea of creating software to help with his music after experiencing a particularly bad case of composers block. The results speak for themselves:


Granted this is not exactly breathtaking; it might be what we’d expect from an advanced piano student who was still primarily leaning on technique because she hadn’t found her creative voice yet. But it’s a long way from the soundtracks of 8-bit video games I grew up playing, and it was written by a computer program.

But what about natural language? Computer-generated music is impressive, but can computers rise to the challenge of processing and responding to speech in real time? IBM’s Watson, a truly monumental achievement, managed to not only do this, but to utterly stomp two of the best jeopardy players of all time. Last I checked the technology was being turned to helping doctors perform better diagnoses.

In my mind the most impressive example is the lesser-well-known Adam (King et al., 2004), an almost fully autonomous science laboratory which, when fed data on yeast genetics, managed to form a hypothesis, design and carry out an experiment to test its hypothesis, and in the process discover something that was unknown to any scientist before. Though this may seem like light-years away from an AI doing, say, astrophysics research, the difference is one of degree, not kind.

Admittedly, we’re still not talking about general intelligences like human beings here. But the weight of the evidence points to a future where increasingly large chunks of civilization are being managed by intelligent machines. This may come to include the production of art, science, and even the design of new intelligent systems.


Your intelligence isn’t magical

I’m writing a series of posts summarizing my views on the Intelligence Explosion, and the first claim I want to defend is that we should take seriously the possibility of human-level artificial intelligence because fundamentally human intelligence is not magic.

Human intelligence is the product of the brain, an object of staggering complexity which, nevertheless, is built up from thoroughly non-magical components. When neurons are networked together into more and more sophisticated circuitry, there is no point at which magic enters the process and gives rise to intelligence.

Furthermore, human intelligence is the product of the blind, brute-force search algorithm which is evolution. Organisms are born with random mutations into environments which act as fitness functions.  Beneficial mutations preserve themselves by leading to greater reproductive success while deleterious ones eliminate themselves by lowering reproductive success. Evolution slowly explores possibilities by acting on and changing existing DNA patterns.

Even without engineering oversight, evolution managed to produce Homo Sapiens, primates with the ability to reason across a wide variety of domains and use their intelligence in ways radically different from the uses for which it evolved.

This is not to imply that our intelligence is well understood; my impression is that great strides have been made in modeling brain activity, but we are surely still a long way from having probed these mysteries fully.

Nor does it imply that building a human-level intelligence will be easy. For decades now AI researchers and computer scientists have been trying, making progress in various narrowly defined tasks like chess, but still nowhere near achieving the creation of a general reasoner on par with humans.

Additionally, it doesn’t imply that a human-level AI must actually resemble human intelligence in any way. AI research is a vast field, and within it there are approaches which draw on neuroscience and mathematical psychology, and de novo approaches which want to build an AI ‘from the ground up’, as it were.

But don’t lose sight of this key fact: the intelligence which produced these words is a non-magical product of a brain made of non-magical components which was produced by a non-magical process. It is hard for me to see where or why a skeptic could draw a special line in the sand at the level of a human and say ‘machines won’t ever get this far’.



Gnostic Creep

A while back I asked famed autodidact Eric Raymond about how he learns things, and he told me that he tends to study multiple subjects at a time with little to no structure involved. I tried this, and noticed that what usually happens is that each field I study suggests additional fields to study, and when I begin to look into those fields still further fields pique my interest, until I’m reading 10 books and 35 papers all at once and making only the most incremental of progress. Eventually the whole thing collapses on itself and I feel depressed for a couple of days.

I call this gnostic creep, a deliberate nod to the concept of “scope creep“.

You might be tempted to advocate for studying only one subject or one book at a time; this is pretty good advice, but easier said than done. For one thing, the front and back cover of a book are often fairly arbitrary beginning and ending points. You may get halfway through a book about the rise of the Nazi party in Germany, only to realize that you can’t make sense of these events until you stop and learn a little about the intellectual history of Fascism in pre-war Europe. But even when this isn’t the case, poor wording or explanations which assume too much background knowledge may force you to look elsewhere.

Example: I’m studying set theory right now in a bid to assemble the tools necessary to understand the research on Friendliness in Superintelligent AIs. One of the recommended books is “Naive Set Theory” by Paul Halmos. It’s very concise, but often utilizes archaic notation and proofs that are so informal that it’s difficult for a novice mathematician to find intellectual purchase on them. To compensate, I piece the ideas together by referencing other books, but I eventually find myself with too much on my plate and no clear strategy for proceeding.

So far, the only thing that has worked is taking a day or two off when the pressure of gnostic creep reaches a certain threshold. I also have a friend who is tutoring me in mathematics, so I’m going to try breaking my learning up into smaller chunks by meeting with him for 20-30 minutes several times a week rather than for 90 minutes once a week.

I’m inclined think that intelligence isn’t a significant factor here; a person who is smarter than me but who lacks a mechanism for temporarily erecting a boundary around a given gnostic enterprise would simply have gnostic creep set in a lot more quickly than in does for me. Presumably the gnostic creep for a genius like John Conway would result in their head exploding.

I seems most likely that those among my heroes who are world-class auhodidacts are doing something which makes them more effective, and which they probably aren’t even aware of.

A market for law?

Most people I’ve spoken to don’t believe that reliable courts could emerge from free markets, and I admit to being pretty unsure about the prospect myself despite being a libertarian.

But an Economist piece on international law provides at least weak evidence that a society with no State, or a severely weakened State, could still come up with legal services.

Apparently huge amounts of international law is handled by the legal systems of just two countries: America and Britain. This is partly because both of these countries have centuries of common law and strong legal precedents, meaning the rulings produced are relatively secure. Adding to the mix are centers of arbitration for international parties with cross-border disputes that have emerged in Singapore, Hong Kong, and elsewhere.

Bear in mind that there is no supranational governmental body determining which countries’ laws will be held as the international standard. Rather, over time certain legal systems have been judged superior, and when India has legal issues to settle with China, both countries opt for an American court or an arbitrator in Singapore.

The analogy isn’t perfect, but I don’t see why the same thing couldn’t happen on a smaller scale, in individual countries, as a result of market processes.

Reverse engineer your favorite writing.

I don’t in any way think of myself as a master of the written word, but I have gotten pretty good at penning nonfiction. There are a million ways to create better prose, but the one I am going to talk about is a species of very active, almost meditative reading.

Basically, I zoom in on any sentence or passage that captures my attention and try to figure out why, then I try to generate the sentence using contents from my own mind.

For example, one of my favorite authors is Sam Harris, whose writing is often sprinkled with gems like this:

” Topics like torture, recreational drug use, and wealth inequality can provoke outrage and misunderstanding in many audiences. But discussing them online sets your reputation wandering like a child across a battlefield—perpetually. “

Now he could’ve just said “talking about stuff online is bad for your reputation”. But instead his phrasing planted a vivid image in my mind of a little girl traversing a minefield, surrounded by smoke and ruin and silence. Even though he’s only talking about how people perceive him on the internet, it’s nuances like this that breathe life into writing that would otherwise be dry and of little interest to most people.

So I opened a word document and typed these two sentences from memory. Then I erased them, trying to imagine what mental state I would’ve had to have been in just prior to having composed them. What thoughts and images would I have needed to have penned those words myself?

Of course, Sam would’ve most likely been in a totally different state of mind, and I can’t ever know. But the point is not to mimic his mind, but to engage in a bit of mental ventriloquism, in effect casting my pen in his voice. How can I arrange the contents of my own mind such that the output is of as high a quality as his?

Put another way, it’s the opposite of paraphrasing; rather than taking thoughts and putting them in different words, I’m starting with different thoughts and trying to make the same words.

I did this with about half of his blog one night while drinking wine, and in one of my own essays I  wrote:

“The first step in navigating a path between the twin rocks of nihilism and absurdity is to remember that it’s always now”.

It’s safe to say that my writing has absorbed some of the style and strength of Harris’s.


In Memorium

I’ve had the poor fortune of being surrounded by a fair bit of death recently.

The first was that of my great-grandmother, a truly and thoroughly decent human being who had suffered the ravages of Alzheimers for nearly the past decade. As seems to often be true in these cases, she was a completely different person by the time the end came. I was very sad to hear of her passing, but we had all known the day was coming, and given the advanced state of her condition, I took some small relief knowing that her pain was over.

Another was that of Alexander Boutilier, or ‘Lex’ as he was usually known. Lex’s death came like a bolt out of the blue, and affected me deeply for reasons I couldn’t say. He was just so….alive, right up until I found out that he wasn’t. We weren’t particularly close, but he was a ferociously intelligent, exceptionally generous man who somehow seemed to have read every book ever written. Being a nerd was something he was proud of, and he had a penchant for spirited and far-ranging discussions.

Needless to say, we immediately liked each other.

In the wake of these tragedies, I did something I’ve been meaning to do for some time: I donated a non-trivial amount of money to a number of charities whose purpose is to put a stop to this ridiculous, needless suffering. The charities, in order of increasing abstractness, were the Alzheimer’s Disease Research Foundation, the Brain Preservation Foundation, the SENS Research Foundation, and the Machine Intelligence Research Institute.

Now, I’m not disclosing all of this for sympathy or to brag, and I know plenty of people would dispute my choice of donations. But what I want to encourage everyone to do is to bloody well take action when you see a part of the world you don’t like. I’m not in a position to stop Alzheimers or to expand the healthy human lifespan; but there are smart people out there who might be able to, and I’ll happily pay them to do it.

Additionally, I am publicly coming out as an anti-deathist. The fact that sentient beings are extinguished, forever and against their will, is a hideous blight on the world. Some day, I think, advanced civilizations will look back with inexpressible sadness that it took us so long to turn the gears and levers of our minds towards the problem of Death. So many little lights, gone, for no other reason then that we didn’t work quickly enough.

How inexcusably wasteful.

There are a few standard responses to this line of thinking:

1) Death is a natural part of life, and has been from the beginning.

Yes, well, being hunted by large carnivores and having your children die during birth were pretty commonplace for most of history, but at least in this part of the world that rarely happens anymore. A few generations ago everyone lived in fear of getting polio, and to be perfectly honest, I’m not even sure what polio is.

There’s no use pretending that there isn’t a lengthy precedent for doing away with sickness and discomfort, and I see no reason why it should be any different for Death.

2) Death gives meaning to life.

No, life gives meaning to life. Passion, novelty, sadness, the gritty texture of day-to-day living, the inevitable ebb and flow of a thousands shards of experience, these are the things of which meaning is made.

3) You’d get bored with immortality.

Maybe, but ending my life is a decision I should be able to make while I’m watching the sun burn down, or watching the Andromeda Galaxy collide with our own, when I’m as old as a planet, preferably with my family and a billion friends with me.

4) Death is far too inexplicable to ever be solved.

Everything was a mystery right up until the point that someone figured it out. Stars, the beating of the human heart, the origins of the continents… long is the list of things people once thought would never be understood which are now routinely taught to high school students.

Death is an engineering problem, and it should be approached as such.

5) Overpopulation!

Every material advance ever made, starting with fire and going right up to smart phones, has contributed to rising population levels. Unless you’re prepared to roll up your sleeves and start dismantling civilization, then I don’t see how your argument holds water.

I’ll miss my great-grandmother’s indefatigable spirits and simple, earthy charm; I’ll miss Lex’s sharp wit and boundless enthusiasm. The pain is made all the more acute by the knowledge that it didn’t have to end this way.

To paraphrase the oft-quoted and immortal Dylan Thomas poem, I do not intend to go gently into that good night.

May there some day be things besides words that live forever.




Luck I: Finding White Swans

Quoth the Master, great in Wisdom, to the Novice: “Ye, carry with thee all thy days a cheque folded up in your wallet.  For there may be many situations in which thou shalt have need of it.”

And the Novice, of high intelligence but lesser wisdom, replied, saying unto the Master: “Of what situations dost thou speak?”  

To which the Master replied: “imagine that thou dost come upon a nice piece of land, and wish to make a down payment on it. The real estate market moveth quickly in these troubled economic times, and you may soon find your opportunity dried up like dead leaves in summer.  What would you do?”  The Master, you see, did dabble in real estate development a little, and his knowledge was deep in these matters.  

The Novice thought for a moment, saying: “But always I carry with me a credit card.  Surely this is sufficient for my purposes.”

And the Master replied: “Thou knoweth not the ways of commerce.  Thinketh thee that all dealings are conducted within feet of a machine that can read credit cards?!”

The Novice knew the ways of Traditional Rationality and Skepticism, and felt it his duty to take the opposite stance to the Master, lest he unthinkingly obey an authority figure.  Undeterred, he replied, saying unto the Master: “But always I carry with me cash. Surely this is sufficient for my purposes.”

Upon hearing this, the Master did reply, incredulously: “Would thee carry with thee always an amount of cash equal to the reasonable asking price of a down payment for a piece of land?!”   

And lo, the Novice did understand, though he could not put it into these words, that the Master did speak of a certain stance with respect to the unknown.  The swirling chaos of reality may be impossible to predict, but there are things an aspiring empirimancer can do to make it more likely that ve will have good fortune.

Verily, know that that which people call ‘luck’ is not the smile of a beneficent god, but the outcome of how some people interact with chance.  


Consider for a moment two real people, whom we will call ”Martin” and “Brenda”, that considers themselves lucky and unlucky, respectively. Both are part of the group of exceptionally lucky/unlucky people which Dr. Richard Wiseman has assembled to try and scientifically study the phenomenon of luck.

As part of the study, both people were placed in identical, fortuitous circumstances, but both handled the situation very differently. The setting: a small coffee shop, arranged so that there were four tables with a confederate (someone who knows about the experiment) sitting at each table. One of these confederates was a wealthy businessman, the kind of person that, should you happen to meet him in real life and make a good impression, could set you up with a well-paying job. All the confederates were told to act the same way for both Brenda and Martin. On the street right outside the coffee shop, the researchers placed a £5 note.

Brenda and Martin were told to go to the coffee shop at different times, and their behavior was covertly filmed. Martin noticed the money sitting on the street and picked it up. When he went into the coffee shop he sat down next to the businessman and struck up a conversation, even offering to buy him a coffee.  Brenda walked past the money, never noticing it, and sat quietly in the shop without talking to anyone.

Fortune favors the…?

There are obvious differences in Brenda and Martin’s behavior, but are they indicative of more far-reaching differences in how lucky and unlucky people live their lives? First, let’s discuss what doesn’t differentiate lucky from unlucky people. Wiseman, having assembled his initial group of subjects, tested them on two traits which could have an impact on luck: intelligence and psychic ability. Determining that intelligence wasn’t a factor was as easy as administering an intelligence test. Psychic ability was ruled out by having both lucky and unlucky people pick lottery numbers, with the result being that neither group was more successful than the other.

Wiseman further tested for differences in personality using the Five Factor Model of Personality, which you will recall breaks personality up into Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism (the acronym OCEAN makes for easy recall) . Lucky and unlucky people showed no differences in Conscientiousness or Agreeableness, but did show differences in Openness, Extraversion, and Neuroticism.  It is here that an interesting picture began to emerge.

Ultimately, Wiseman was able to break luck down into four overarching principles and twelve subprinciples, summarized here:

Principle One: Maximize the number of chance opportunities you have in life.

  subprinciple one: lucky people maintain a network of contacts with other people.

  subprinciple two: lucky people are more relaxed and less neurotic than unlucky people

  subprinciple three: lucky people introduce variety into their routines.

Principle Two: Use your intuition to make important decisions.

  subprinciple one: pay attention to your hunches.

  subprinciple two: lucky people try to make their intuition more accurate.

Principle Three: Expect good fortune.  

  subprinciple one: lucky people believe their luck will continue.

  subprinciple two: lucky people attempt to achieve their goals and persist through difficulty.

  subprinciple three: lucky people think their interactions will be successful.

Principle Four: Turn bad luck into good.

  subprinciple one: lucky people see the silver lining in bad situations.

  subprinciple two:lucky people believe that things will work out for them 

  subprinciple three: lucky people spend less time brooding over bad luck.

  subprinciple four: lucky people try to prevent further bad luck.  

I suspect that LWers will have a unique set of reactions to and problems with each of these principles, so let’s take them one at a time.  In this essay, I will examine the first two.

Facing up to randomness

First, how would you go about increasing the likelihood of positive chance encounters? Well, you could start spending more time talking to strangers and making friends with people.  Indeed, one of the important differences between unlucky and lucky people is that lucky people are more outgoing, more friendly and open in their body language (lucky people smiled and made eye contact far, far more often), and keep in touch with people they meet longer. The age-old adage ‘it’s not what you know, but who you know’ has more than a grain of truth in it, and a great way to get to know the right people is by simply getting to know more people, period. The chances of any given person being the contact you need are pretty slim, but the odds improve with every person you get to know.

This actually works on several levels. Since the complexity of the world greatly exceeds the cognitive abilities of any one person, cultivating a strong social network positions you to take advantage of the knowledge and experience of others. Even if you are so much smarter than person X that they can’t compete with you along any dimension, they may still have information you don’t, or they may know somebody who knows somebody who can help you out.

Moreover, I’m sure everyone is familiar with the experience of struggling with a problem, only to have a random conversation (with a stranger or a friend) shake loose a key insight. This can happen locally inside your own head when you have the necessary raw material laying around but haven’t seen a certain connection. In this situation you would have eventually hit upon the insight but the process has been expedited.  More valuable still is when two or more people enter a conversation that produces an insight that nobody had the necessary components to produce for themselves; I think this is part of what Matt Ridley means when he talks about ideas having sex.

So you’re doing your best to meet more people and flex your extroversion muscles. Next, you might try and be more spontaneous and random in your life. Wiseman notes that many lucky people have a strong orientation towards variety and novel experiences.  Some of them, facing an important decision like which car to buy, will do something like list their options on a piece of paper and then roll a die.

You don’t need to go quite this far; it’s also acceptable to shop different places, take different routes to work, or pick a new part of the city to explore every month. The takeaway here is that it’s difficult to have positive chance encounters if you always do the same thing.

One of my favorite examples of someone positioning themselves to benefit from chance comes from Harry Potter and the Methods of Rationality, when Harry and Hermione first read all the titles of the books in the library and then read all the tables of contents.  From their point of view the books in the library are a vast store of unknown information, any bit of which they might need at a given time. Since reading every single book isn’t an option, familiarizing themselves with the information in a systematic way means creating many potential sources of insight while simultaneously reducing the cost of doing future research. Hacker Eric Raymond made related point in the context of winning table-top board games:

I made chance work for me. Pay attention, because I am about to reveal why there is a large class of games (notably pick-up-and-carry games like Empire Builder, network-building games like Power Grid, and more generally games with a large variety of paths to the win condition) at which I am extremely difficult to beat. The technique is replicable.

I have a rule: when in doubt, play to maximize the breadth of your option tree. Actually, you should often choose option-maximizing moves over moves with a slightly higher immediate payoff, especially early in the game and most especially if the effect of investing in options is cumulative.

What’s the common thread between extroversion, skimming the library shelves, and beating your friends at boardgames? Certain actions and certain states of mind make it more likely you’ll benefit from white swans.

(Clever readers may be saying to themselves: “okay, but doesn’t all this also make the chances of encountering black swans higher as well?” We will address these concerns when we talk about principles three and four.)

Attitude matters

We’ve covered extraversion and openness, but the lucky people Dr. Wiseman interviewed were also more relaxed and less neurotic than the unlucky ones. This has obvious consequences for when you are trying to meet new people, but research also hints that being less anxious may make you more likely to notice things you aren’t specifically looking for. This is probably why several of Dr. Wiseman’s lucky participants remarked on how often they found money on the street, found great opportunities while listening to the radio or reading the newspaper, and in general stumbled over opportunities in places where other people simply failed to notice them.

This attitude undergirds and complements much of what I discussed in the previous section; while you are trying to maximize your pathways to victory, don’t forget that constantly worrying and mentally spinning your tires will make you less likely to see a chance opportunity.

Pump your intuition

Lucky people tend to have strong intuitions, and they have a habit of paying careful attention to them.  I’m sure you’re skeptical of this advice, as I was when I first started reading this section. Given present company I don’t think I need to reiterate all the billion ways intuition can be derailed and misleading. That said, placing intuition and rationality as orthogonal to one another is a good example of the straw vulcan of rationality. Intuitions are of course not always wrong, and in some cases may be the only source of information a person has to go off of.

Two things put a little nuance on the proposition that you should listen to your intuitions. The first is that, as far as I can tell, lucky people don’t trust their intuitions immediately and absolutely. They don’t stand at a busy intersection, blindfolded, and trust their gut to tell them when it’s safe to cross. Rather, their hunches act more like yellow traffic lights, telling them that they should proceed with caution here or do a bit more research there. In other words, it sounds to me like lucky people treat their intuitions in a pretty rational manner, as data points, to be used but not relied upon in isolation unless there is just nothing else available.

The other thing is that many lucky people take steps to sharpen their intuitions, utilizing quiet solitude or meditation. Dr. Wiseman goes into precious little detail about this, including just a few anecdotal descriptions of people’s efforts to clear their mind. The rationalist community will be familiar with more quantitative methods like predictionbook, and googling for ‘improving your intuitions’ turned up about as much garbage as you’d probably expect.  If anyone has leads to legitimate research on improving intuition, I’d be happy to add an addendum.

Suggested exercises

Throughout the book Dr. Wiseman includes exercises which are meant to help people utilize the principles uncovered in his research to become luckier. Here are the suggested exercises for the topics discussed in this post:

-To enhance your extraversion, strike up a conversation with four people you either don’t know or don’t know well. Do this each week for a month. Additionally, every week make contact with a person you haven’t spoken to in a while.

-To relax, find a quiet place and picture yourself in a beautiful, calming scene. Make sure to visualize each and every detail of the location, including whatever sounds and smells are around you. When you’ve got the scene in place, visualize the tension leaving your body in the form of a liquid flowing out of you, starting with your head. once you feel sufficiently relaxed, slowly open your eyes.

-Inject some randomness in your life by making a list of 6 new experiences. These can be anything from trying a new type of food to taking a class on a subject you’ve always been interested in.  Number them 1 to 6, roll a die, and then do whatever corresponds to the number you rolled.

To be continued…