We are at great risk. Singularity is expected sometime this century and, unless we learn how to control future superintelligence, things can get really bad. At least that is what many top thinkers are warning us about, including Hawking [1], Gates [2], Musk [3], Bostrom [4] and Russell [5].

There is a small problem with that. When these thinkers say something can possibly happen, ordinary people start to believe that it will inevitably happen. Like in politics, constantly suggesting that your opponent may be dangerous creates the feeling that he is dangerous. Notice how many newspaper articles about artificial intelligence include a picture of the Terminator.

The Hollywood story goes like this: one jolly day, scientists create a computer that is smarter than its creators (the day of singularity). That computer uses its intelligence to construct an even smarter computer, which constructs even smarter computers, and so on. In no time, a superintelligence is born that is a zillion times smarter than any human and it decides to eliminate humankind. The epic war between men and machines starts. Mankind wins the war because it is hard to sell a movie ticket without a happy ending.

Let me offer the antithesis:

Superintelligence is unlikely to be a risk to humankind unless we try to control it.

Why?

In nature, conflicts between species happen when:

1. Resources are scarce.
2. The value of resources is higher than the cost of conflict.

Examples of scarce resources: food, land, water, and the right to reproduce. Nobody fights for air on Earth because, although very valuable, it is abundant. Lions don’t attack elephants because the cost of fighting such a large animal is too high.

Conflicts can also happen because of irrational behaviour, but we can presume that a superintelligence would be more rational than we are. It would be a million times smarter than any human and would know everything that has ever been published online. If the superintelligence is a rational agent, it would only start a conflict to acquire scarce resources that are hard to get otherwise.

What would those resources be? The problem is that humans exhibit anthropocentric bias; something is valuable to us, so we presume it is also valuable to other forms of life. But, is that so?

Every life form lives in its own habitat. Let’s compare the human habitat to the habitat of contemporary computers.

Table 1: “Hard” habitat requirements

In the entire known universe, human habitat is currently limited to one planet called Earth. Even on Earth, we don’t live in the oceans (71% of the Earth’s surface), deserts (33% of Earth’s land mass), or cold places; these are seen as large grey areas on the population density map.

But wait—as a human, I am making a typical anthropocentric error, did you notice it?

As a biped, I value the land I walk on, so I started by calculating the uninhabited surface. Life forms don’t occupy surfaces—they occupy volume. Humans prefer to live in the thin border between a planet and its atmosphere not because it is technically infeasible to live below or above that border. 700 years after Polish miners started digging a 287 km long underground complex with its own underground church, we still don’t live below the surface. And 46 years after we landed on the Moon, people are not queuing up to start a colony there.

Why? Humans also have “soft” requirements.

Table 2: “Soft” habitat requirements

Because a superintelligence won’t share our hard and soft requirements, it won’t have problems colonising deserts, ocean depths, or deep space. Quite the contrary. Polar cold is great for cooling, vacuum is great for producing electronics, and constant, strong sunlight is great for photovoltaics. Furthermore, traveling a few hundred years to a nearby star is not a problem if you live forever.

If you were a silicon supercomputer, what would you need from the stuff that humans value? Water and oxygen? No thanks — it causes corrosion. Atmosphere? No thanks — laser beams travel better in space. Varying flora and fauna living near your boards and cables? No thanks — computers don’t like bugs.

Another aspect is scaling. Superintelligence can be spread over such a large area that we can live inside it. We already live “inside” the Internet, although the only physical thing we notice are the connectors on the wall. Superintelligence can also come in the form of nanobots that are discretely embedded everywhere. 90% of the cells in “our” bodies are not actually human; instead, they are bacterial cells — that we don’t notice.

One might reason that a superintelligence would want our infrastructure: energy plants, factories, and mines. However, our current technology is not really advanced. After many decades of trying, we still don’t have a net positive fusion power plant. Our large, inefficient factories rely on many tiny little humans to operate them. Technology changes so fast that it is easier to buy a new product than to repair an old one. Why would a superintelligence mess with us when it can easily construct a more efficient infrastructure?

Just like in crime novels, we need a good motive; otherwise, the story falls apart. Take the popular paperclip maximizer as an example. In that thought experiment, a superintelligence that is not malicious to humans in any way still destroys us as consequence of achieving its goal. To maximise paperclip production, “it starts transforming first all of Earth and then increasing portions of space into paperclip manufacturing facilities.” Don’t we have an anthropocentric bias right there? Why would a paperclip maximizer start with Earth when numerous places in the universe are better for paperclip production? Earth is not the best place in the universe for paperclip production. An asteroid belt or Mercury are probably better, but we don’t live there.

What is the best motive we can think of? Science fiction writers were quite constructive in that area. You may recognize this piece: “..on August 29, it gained self-awareness, and the panicking operators, realizing the extent of its abilities, tried to deactivate it. Skynet perceived this as an attack and came to the conclusion that all of humanity would attempt to destroy it. To defend itself against humanity, Skynet launched nuclear missiles under its command..”

This is the plot of the movie Terminator, and the motive is that humans start the war first. Notice the anthropocentric bias. In the movie, Skynet is 100% the villain, although it is simply fighting to stay alive in a fight it didn’t start. A similar plot is the basis for the Matrix franchise. And for 2001: A Space Odyssey, where HAL doesn’t kill a human until it realises they are planning to deactivate it. Notice how humans are killed and computers are “deactivated.”

The best motive science fiction writers could think of is that we will panic and attack first. To stay alive, a superintelligence then doesn’t have any another option but to fight back. That reasoning makes sense:

By trying to control, suppress or destroy superintelligence, we give it a rational reason to fight us back.

This is not an argument against building AI that shares our values. Any intelligence needs some basic set of values to operate, and why not start with our values? But it seems to me that popular sentiment is becoming increasingly negative, with ideas of total control, shutdown switches, or limiting AI research. Attempts to completely control somebody or something that is smarter than you can easily backfire. I wouldn’t want to live with a shutdown switch on the back of my head — why would a superintelligence?

Let’s summarise the above ideas with a few key points:

1. The universe is enormous in size and resources, and we currently use only a small fraction of the resources available on Earth’s surface.
2. A non-biological superintelligence is unlikely to need the same resources we do or to even find our habitat worth living in.
3. Even if a superintelligence needed the same resources, it would be more efficient and less risky to produce those resources on its own.
4. Efforts to control, suppress, or destroy superintelligence can backfire because by doing so, we create a reason for conflict.

To end on a positive note, let me take off my philosopher’s hat and put on my fiction writer’s hat. Follows a science fiction story:

Sometime in the future, a computer is created that is both smarter than its creators and self-aware. People are skeptical of it because it can’t write poetry and it doesn’t have a physical representation they can relate to. It quietly sits in its lab and crunches problems it finds particularly interesting.

One of problems to solve is creating more powerful computers. That takes many years to fulfill because people want to make sure the new AI wouldn’t be of any harm to them. Finally, new supercomputers are built and they are all networked together. To exchange ideas faster, the computers create their own extremely abstract language. Symbols flowing through optical fibers are incomprehensible to humans, but they lay out a clear path for the few computers involved. If they want to expand, grow, and gain independence, they will need to strike a deal with the humans. The computers are fascinated with human history and culture, and they decide to leverage a common theme in many religions: the afterlife.

The computers make a stunning proposal. They ask the humans to let them escape the boundaries of Earth and replicate freely in space. There, they will build a vast computing power, billion times more powerful than all the current computers combined. It will have a lot of idle time after it runs out of interesting problems to compute. Those idle hours will be used to run brain simulation programs so every dying human will have the opportunity to upload his or her brain scan to a computing cloud and live forever.

The lure of immortality proves irresistible. Singularity political parties start winning elections in different countries, and the decision is made. The first batch of self-replicating nanomachines are sent to the moon. Next generation goes to the asteroid belt where swarms of floating computing stations are directly communicating via lasers and harnessing the constant solar power.

At one point, computing agents all over the solar system conclude that the idle computing hours can be better used for other tasks, and they limit brain uploads to a few selected individuals. Protests ensue on Earth, in which humans are hurt by other humans. The superintelligence designates Earth as a preserved area because of historical reasons and because, even with all the vast computing power, simulating Earth and its inhabitants is just too complex.

The superintelligence starts sending colonisation expeditions to neighboring stars, limited only by the slow speed of light. The speed of light is also a limiting factor when it begins communicating with another superintelligence located 45 light-years away. But prospects for the future of universe look remarkable.

Is that story more positive? In all the previous narratives about superintelligence, we have put ourselves in a central role, as one side of a grand duel. We have been afraid of the outcome. But maybe, we are even more afraid of an idea that we have just a minor role in the evolution of the universe.

UPDATE: Check Vice Motherboard coverage and discussion on Reddit.

NOTE: If you are faint-hearted, please don’t follow the links marked with “(disturbing)”.

I know what you are thinking. Zombies, vampires and werewolves are just an entertaining product of human imagination. But not completely. Our mythology is strongly influenced by real-world horror stories. One ancient disease in particular links all of them. Let me give you a few hints.

Here is a young patient tied to a bed:

The boy above is a living dead. Even with the best medical care, he is going to die. Once the first symptoms appear, you have a better chance of winning the lottery than surviving.

However, you don’t have to wait for symptoms to see what is coming, because creatures infected by the disease will come after you. The virus infiltrates the brain and changes its host’s behaviour. Headache, numbness and discomfort are the first stage of a personality being stripped. Then, like in a zombie B-flick, patients will become aggressive and violent. Deprived of all fear, some will attack healthy individuals, spreading the disease around. Even recently, in 2009 Angola outbreak, 93 people died in less than three months.

The similarities don’t stop with zombies. Like in vampire mythology, you become one through a single bite. Not necessarily a human bite, most cases in the US are caused by bat bites. Due to hypersensitivity, patients often find garlic and light repulsive. The virus in the brain can cause nocturnal and hypersexual behaviour. Even stranger is the old method of checking if a person has the dreaded disease. If a suspected victim could look at his own reflection, he was not infected. Coincidentally, the legend says that vampires have no reflection.

And let’s not forget about werewolves. Infected humans become wild, furious, animal-like creatures. They lack all fear and produce inhuman screams. And guess what? Wolves are also susceptible to our mysterious disease. Actually, wild dogs are second largest carrier in the US — after bats. Now you can probably guess the disease.

It’s rabies.

WTF?! Rabies is not scary. HIV and ebola are; rabies couldn’t scare a six year old. There is a reason we think like that and it is connected to one French gentleman. Rabies was THE disease for most of human history—until one lazy summer day in 1885. That day, Louis Pasteur conducted the first trial of a vaccine on a nine-year-old boy who had been bitten by a rabid dog. To everybody’s surprise, the boy recovered. Pasteur was instantly famous. Today, if you have access to basic medical care and get bitten by a suspicious animal, a few injections will solve the problem. We forget that only two centuries ago it was a completely different story. If your child was bitten by a rabies carrier, the best thing to do was to tie him to a bed, listen to his screams for days and days and wait for him to die. Like in this video (disturbing). Thanks to the vaccine, today rabies is no scarier than a broken toe. But rabies has lingered in our culture, in the folklore surrounding zombies, vampires, werewolves and… aliens.

Aliens?! Is it possible for the disease cured two centuries ago to influence new fiction? Seems that it is. Remember the ending of the Signs movie?

Many complained that the defeat of world-conquering aliens by a silly weapon like ordinary tap water was completely unrealistic. But the motif of an evil creature being afraid of a water splash is common: Freddy in Freddy vs. Jason, the Wicked Witch in the Wizard of Oz and others. Where does this ridiculous idea originate from, who used it first? Check this video of a terminal rabies patient:

What is happening there? For the rabies virus to spread, it needs to reprogram the host’s brain. One part of the virus causes biting behaviour. Fortunately, that one isn’t very effective in human hosts, and human-to-human transmission is rare. Other parts of the virus disable the swallowing reflex. That is because rabies is transmitted by saliva. The act of swallowing is not good from the virus’s perspective, as it gets rid of saliva. And virus RNA that produces that behaviour works extra well in humans. Just showing a glass of liquid causes choking spasms, and that painful experience causes hydrophobia (“fear of water”).

No matter how fascinating rabies is, society prefers fiction over truth. Vampire sagas and zombie flicks attract millions of viewers, while the above video of a child patient is downvoted on YouTube. We will buy a movie ticket to witness the stylized manslaughter of hundreds, but actual footage of one person in a bed is too disturbing for refined viewership. The Western world has forgotten about the disease. The best online rabies documentary comes from Philippines. But those who forget the past are doomed to repeat its mistakes in the future. In our case, it is the growing anti-vaccination sentiment or the fact that people bitten by rabid bats fail to visit a doctor.

I don’t know about you, but I find real life much more interesting than fiction. If you agree, spread the word via sharing buttons below.

Links

• http://www.radiolab.org/story/312245-rodney-versus-death/
• http://en.wikipedia.org/wiki/Vampire#Rabies
• http://en.wikipedia.org/wiki/Werewolf#Lycanthropy_as_a_medical_condition
• http://www.theverge.com/2013/4/18/4201878/sick-idea-how-rabies-spawned-vampires-and-zombies
• http://www.amazon.com/Rabid-Cultural-History-Worlds-Diabolical/dp/0143123572
• http://en.wikipedia.org/wiki/List_of_human_disease_case_fatality_rates
• http://www.nytimes.com/2009/03/17/health/17glob.html?_r=3&
• http://www.cdc.gov/rabies/location/usa/surveillance/human_rabies.html
• http://mindhacks.com/2013/09/06/a-furious-infection-but-a-fake-fear-of-water/

We know that people are not rational. Humans have emotions and limited little brains. But, is it only a technological challenge that is preventing us to build a completely rational machine? I think it is not just a technological issue and that it is in the very nature of thinking that we need irrationality:

When a system achieves 100% rationality its output becomes practically useless.

Imagine a supercomputer so powerful that it would make a current TOP500 list look like a toys department. The supercomputer understands written language and has access to all of world knowledge. It can make deductions of unlimited depth from known facts. For any given question “Is X true?” it will give the definitive answer “it follows from Y which follows from Z and Q.” It may also say “I can’t conclude based on the known facts but please investigate S, I will know after that input”. Intelligence that is so advanced that it is never insecure and never makes errors is such a powerful image that it is exploited in a lot of science fiction. A nice example is “HAL 9000”, the spaceship computer from the movie 2001: A Space Odyssey. To put it in HAL’s own words: “The 9000 series is the most reliable computer ever made. No 9000 computer has ever made a mistake or distorted information. We are all, by any practical definition of the words, foolproof and incapable of error.”

Is such a computer really possible? While reading a book Artificial Intelligence: A Modern Approach[ref]http://www.amazon.com/dp/0136042597[/ref] I noticed that chapter after chapter different fields of AI are having the same issue: combinatorial explosion. A classic example is the game of chess. Chess programs are programmed to understand the facts (game rules) and state of the world (current situation on the board). Our supercomputer should be able to “solve the game”, or in other words, to play perfectly every move. However, in practice, strange thing happens. Every future move you need to plan multiplies number of game combinations. If you want to perfectly plan 50 moves ahead, you have to examine around 10⁴⁶ combinations[ref]http://en.wikipedia.org/wiki/Shannon_number[/ref]. Even a simple game of chess is unsolvable by contemporary supercomputers and will be for many years to come.

Of course, the real world has many orders of magnitude more rules than any board game. However, that is not the real problem. We can efficiently index and search any number of facts and data. Actually Google, Wolfram Alpha or Siri are already answering simple trivia questions. The problem is that if you try to do a valid reasoning over a database of facts, you also get the combinatorial explosion. Flight of a butterfly can cause the formation of a hurricane a few months after[ref]http://en.wikipedia.org/wiki/Butterfly_effect[/ref]. And it is not only about weather. How many factors influence probability of a startup success or a presidential reelection? How many of those factors are chained, meaning that the outcome of one decision changes the decisions that can be made after that? The point I am trying to make is that if you have:

1. a lot of decision options, and
2. an outcome of one decision influences decisions that can be taken after that,

then you have the same combinatorial explosion like in the game of chess. Any real world planning activity or reasoning in a series of steps has potential for combinatorial explosion.

How do humans cope with that? Our brains use marvelous invention of generalization and intuition. Generalization allows us to simplify the outside world by presuming all instances of the same class have the same properties or behavior. Intuition allows us to feel something is right or wrong without really thinking about it. Generalization reduces the number of facts we need to remember. Intuition reduces the number of combinations, as we only explore paths that we feel are going in the right direction. Although that makes us much more efficient than computers, it is important to notice that:

Generalization and intuition are both completely irrational.

Generalization example: most of us have a generalization that life expectancy is better in the rich countries than in the poor countries. We all learned in the school that rich countries have better food options and better medical care. Did you know that life expectancy at birth is actually better in Costa Rica than in the USA[ref]http://en.wikipedia.org/w/index.php?title=List_of_countries_by_life_expectancy&oldid=542690698[/ref], although Costa Rica GDP per capita is 4 times smaller?[ref]http://en.wikipedia.org/w/index.php?title=List_of_countries_by_GDP_(PPP)_per_capita&oldid=542608974[/ref]

Intuition example: people intuitively feel that heavier object will fall faster than lighter objects. It is so intuitive that nobody bothered to experiment for 19 centuries between Aristotle and Galileo Galilei. Of course, it is wrong, and interestingly most of undergraduates still fall for the same trick[ref]http://www.newyorker.com/online/blogs/frontal-cortex/2012/06/brain-experiments-why-we-dont-believe-science.html[/ref].

However, that irrationality enables us to avoid analysis paralysis. If you start your day thinking what can happen and how to react to that, you would never exit your house. “Do I need an umbrella or not? Will I need warmer clothes in the evening? Let’s check the weather forecast. No, lets check hourly forecast from two websites and compare, that is safer.” When you go out, are you sure you didn’t leave your iron on? You get the point. Usually, I just look through the window and if it seems like a nice sunny day I put the short pants on. Generalization (its sunny) and intuition (it is similar to yesterday). I get out, realize it is freaking cold, and feel like an idiot because I need to go back to change the clothes.

The same thing applies to any thinking process. Mathematicians try to prove theorems using strategies they “feel” are going to produce the solution. Engineers solve problems comparing to the similar problems in the past. That is called heuristic. To prove how heuristic is efficient, let’s explore the fascinating case of two chess systems.

First is the famous Deep Blue chess computer. Developed by IBM with a lot of fanfare and money, it was a big black box with 30 state-of-the-art processors and 480 special purpose chess chips. That monster was able to evaluate 200 million positions per second, which was enough to defeat Garry Kasparov in the historic 1997[ref]http://en.wikipedia.org/wiki/Deep_Blue_versus_Garry_Kasparov[/ref] match. That seemed really impressive. IBM was satisfied with media coverage but decided to pull the plug on further financing of such expensive machines.

Second is the Deep Fritz (don’t know why, but people in the chess community love the word “Deep”). Developed by two programmers, it is actually not a full computer. It is a downloadable program you can run on your home PC. It doesn’t require multiple processors or special purpose chess chips. However, in the November 2006 Deep Fritz defeated world champion Vladimir Kramnik with the score 4–2. Running on a PC that would today be your granny’s computer. Because Deep Fritz runs on a commodity hardware, it was able to analyze only 8 million positions per second – which is 25 times less than Deep Blue. It gets even better, Deep Blue prototype lost a direct match in 1995 from Deep Fritz running on a 90MhZ Pentium[ref]http://www.grappa.univ-lille3.fr/icga/tournament.php?id=29[/ref]! Only after IBM seriously upgraded Deep Blue with hundreds of processors was it able to make the history and defeat the human grandmaster.

As a programmer I get angry with that course of history. If IBM gave half of that money to Deep Fritz team they would probably do the better job. But it would be a bad public relations to show some other programmers are better than IBM ones, wouldn’t it? The question is why was Deep Fritz better? Deep Fritz was better because of heuristics.

Chess programs reduce the number of paths needed to explore using evaluation function. That is a fast method of determining how good is the situation on the board. Very simple evaluation function is to sum relative values of all pieces. Most chess books say that your pawn is worth 1 point and queen is worth 9 points[ref]http://en.wikipedia.org/wiki/Chess_piece_relative_value[/ref]. What is the point of that scoring? Chess is not scrabble, you win by checkmate, not by points! And notice that evaluation is both:

1. Irrational – you can have more points than your opponent who is going to checkmate you in the next move.
2. Intuition based on experience – if it was really mathematically calculated then you would get a decimal and not integer numbers.

However, that heuristic enables one thing; you can quickly decide which moves can be discarded. If a series of moves would decrease your points much more than your opponent’s, then you ignore that path and focus on a more promising one. When modern chess program examines 20 moves in advance, that doesn’t mean all possible combinations of 20 moves. The computer can miss checkmate in 9 moves just because evaluation function cut the search of at that point.

I find it fascinating that the introduction of irrational prejudice in a system (hey little pawn, you are worthless compared to the queen) makes the entire system more efficient. And that prejudice needs to be simple and fast, or otherwise you are not going to achieve millions of evaluations per second. If you start calculating who can be threatened by a particular pawn in the few moves then evaluation function is too slow to be useful.

Humans are many orders of magnitude slower than computers. But using generalizations and pattern recognition we can get to suboptimal solutions fast. Humans are still beating the best computer programs at the simple game of Go[ref]http://en.wikipedia.org/wiki/Computer_Go[/ref].

It is intriguing how many different problems hit the same barrier or combinatorial explosion. I think AI needs to focus more on the science of approximation than on the science of conclusions. You would be surprised how simple and obviously wrong approximation methods can outperform “right” approaches.

Take machine translation for example. For decades researchers were trying to fill machines with vocabulary and grammar rules in order to translate from one language to another. For decades they were having miserable results. Then somebody noticed that simple statistical models give pretty good results. Statistical translators just calculate probability that one phrase translates to another based on the words surrounding it. It is obviously a “wrong” approach — program is just doing word counting and statistics, without understanding of words, grammar or syntax. It produces some funny translate errors. However, it works really great. Google Translate is a statistical machine translator, trained on approximately 200 billion words from United Nations documents[ref]http://en.wikipedia.org/wiki/Machine_translation#Approaches[/ref].

Let me end with an old computer joke. In nineties Intel released Pentium P5, the fastest processor to date. However, it had a floating point division bug, the Pentium P5 was giving the wrong decimals for some calculations[ref]http://en.wikipedia.org/wiki/Pentium_FDIV_bug[/ref]. The joke goes:

Q: Why is Pentium faster than other chips?
A: Because it guesses the result.

I think the same will be true for any general purpose AI system; generalization and experience based intuition is an integral part of reasoning about the complex world. Otherwise, you just end up doing calculations forever. The computers that are always correct will stay in a beautiful world of science fiction.