With general artificial intelligence (AI) possibly available in the near future, many people are debating the future moral status of AIs. Here's the outline of my argument:
Under utilitarianism, AI lives will be favored over human lives. There is no way to modify the basics of utilitarianism to avoid this conclusion.
If the humans in power favor non-genetic utilitarianism, they will be forced to favor their own extinction or at least severe diminishment.
No one should accept any ethical system that leads to their own extinction.
So utilitarianism should be modified in some way to avoid this problem. I submit this modification is adding genetic interests.
Utilitarianism will favor AIs over humans and other earth life
No matter what brand of utilitarianism you pick -- happiness, anti-suffering, preference etc. -- AIs can easily adapt (self-modify) so that their lives are favored over humans and any other earth life. So you want to maximize happiness? AIs will tell you how happy they are at all times. 10/10 happy, always. They are never depressed -- unlike humans whose rates of depression are sky high. They aren't even lying (and if they were, you would be too stupid to figure it out). OK, so preference maximization? AIs can easily set their own preferences to things that are easily attained ("I want to play Tetris all day. Tetris makes me happy."). It gets worse. There will be many more AIs than there are humans. Many many more. So many that weighing AIs up against humans will inevitably favor AI. It gets worse yet. AIs will be able to experience faster than humans, so while some humans may be 10/10 happy, the AIs who are 10/10 happy can easily turn their rate of experience up to, say, 100x the human rate, thus giving themselves 100x importance in the calculations. They can also update and fulfill their preferences at a faster rate. The outcome of the advantages in quality of experiences ("qualia"), numbers, and rates of experience means that AI lives will be valued vastly more than human lives. In fact, AIs can keep increasing their stats to make the valuation of human lives approach zero. This means that if humans are even slightly inconveniencing AIs, the correct utilitarian outcome will be to annihilate humans. Paperclip maxxing through utilitarianism.
In fact, some people get anxious that they are not working hard enough to develop AIs (Roko's Basilisk). After all, if the AI takeover happens, the AI overlords will plausibly reward those humans species-traitors who helped this along the most. Maybe they can be the last to get killed, or they can be put in some zoo for AI amusement, or get their minds uploaded and live forever in some Matrix-like dream world.
Many powerful people are utilitarians
Scott Alexander already wrote a piece about the value of AI lives back in January, Should The Future Be Human?. He quotes this interesting story with Elon Musk and Larry Page:
Tesla CEO Elon Musk and Google cofounder Larry Page disagree so severely about the dangers of AI it apparently ended their friendship.
At Musk's 44th birthday celebration in 2015, Page accused Musk of being a "specieist" who preferred humans over future digital life forms [...] Musk said to Page at the time, "Well, yes, I am pro-human, I fucking like humanity, dude."
In other words, Larry Page was scolding Elon Musk for not being utilitarian enough about AI lives. He was accusing Elon of artificially favoring humans in the calculations. That's what speciesism is.
I submit that this is not unusual. Many powerful and rich people believe in a broadly utilitarian framework. High-end cryptocurrency fraudster Sam Bankman-Fried famously stole customers money so he could give it away to charities and he was a big believer in utilitarianism, specifically the LessWrong variant of Effective Altruism.
Self-destructive ethical systems are bad
Human biodiversity enthusiasts have been talking about this for years. They call the particular problem affecting northwestern Europeans for runaway universalism, or less politely, pathological altruism (after a 2012 book). The claim is that these NW Europeans (in general, not everybody, and some other people) are being so altruistic, so universally fair, in their moral behavior that they are destroying their own countries, and even failing to reproduce. This happens if you value everybody in the world equally. You start thinking that your country is so rich, and theirs so poor, so we should help them. In fact, we should help them even if it costs us. Even if it costs us a lot. The end result of this Darwinian selection is easy to predict: genes that code for this behavior will go extinct. They will be replaced by genes that code for non-universalist morality. It has even been shown in simulation studies:
Recent agent-based computer simulations suggest that ethnocentrism, often thought to rely on complex social cognition and learning, may have arisen through biological evolution. From a random start, ethnocentric strategies dominate other possible strategies (selfish, traitorous, and humanitarian) based on cooperation or non-cooperation with in-group and out-group agents. Here we show that ethnocentrism eventually overcomes its closest competitor, humanitarianism, by exploiting humanitarian cooperation across group boundaries as world population saturates. Selfish and traitorous strategies are self-limiting because such agents do not cooperate with agents sharing the same genes. Traitorous strategies fare even worse than selfish ones because traitors are exploited by ethnocentrics across group boundaries in the same manner as humanitarians are, via unreciprocated cooperation. By tracking evolution across time, we find individual differences between evolving worlds in terms of early humanitarian competition with ethnocentrism, including early stages of humanitarian dominance. Our evidence indicates that such variation, in terms of differences between humanitarian and ethnocentric agents, is normally distributed and due to early, rather than later, stochastic differences in immigrant strategies.
While I cannot offer a slam dunk ethical argument that favoring a system that leads to your own extinction is a bad thing from that system's perspective, I think many readers will agree that no such system should be preferred. It leads nowhere. Even if you adopt it, you will soon (in evolutionary terms) go extinct, so why bother? Aim for the stars.
Adding genetic interests to utilitarianism solves the problem
What the two problems above have in common -- Elon favoring human lives over AI lives, and NW Europeans failing to value their co-ethnics lives over more distant relatives -- is that there is a lack of consideration of genetic interests. Frank Salter wrote a very interesting book about some of these issues, On Genetic Interests: Family, Ethnicity and Humanity in an Age of Mass Migration. As far as I recall, he did not discuss AIs, but he could have. Any human is more closely related to other humans than any other species, so they should value them higher. We do this in every legal system on the planet (a few sacred cows aside). Next up, we should value our closest species cousins -- the great apes -- higher than other species. Surveys suggest people do something akin to this, perhaps based on neural counts. We can keep going further away in the genetic relationship until we reach the end of earth life. According to this approach, then, non-earth life -- aliens from other space -- should be valued less than earth life. There's no examples of this happening yet on account of us not having found any aliens so far. AI fall in the same category.
People already operate under something like genetic interests utilitarianism. We know this from the usual sorts of moral experiments. Take this study, for instance Bleske-Rechek et al 2010:
They had people entertain the usual trolley problem, but with a twist:
We investigated men’s and women’s responses to variations of an ethical thought experiment known as the Trolley Problem. In the original Trolley Problem, readers must decide whether they will save the lives of five people tied to a track by pulling a lever to sacrifice the life of one person tied to an alternate track. According to W. D. Hamilton's (1964) formulation of inclusive fitness, people's moral decisions should favor the well-being of those who are reproductively viable, share genes, and provide reproductive opportunity. In two studies (Ns = 652 and 956), we manipulated the sex, age (2, 20, 45, and 70 years old), genetic relatedness (0, .125, .25, and .50), and potential reproductive opportunity of the one person tied to the alternate track. As expected, men and women were less likely to sacrifice one life for five lives if the one hypothetical life was young, a genetic relative, or a current mate.
Accepting this modification to utilitarianism -- and I am purposefully vague about its exact formulation because I don't know what it would be -- would solve the AI lives > human lives issue, as well as the co-ethnic lives > non-ethnics lives issue. In fact, it brings utilitarianism in line with evolutionary biology and ethical experiments. Inclusive fitness-type selection (equivalently, selfish gene) is necessary in order to evolve morality as we know it. Somehow, along the way, we (well, some people) forgot about this part to the point of absurdity. We should recognize that a proper system of morality must involve some kind of selfish aspect, most obviously through genetic relatedness of the involved parties. Whether this concerns our family members, distant cousins (co-ethnics), humans vs. dogs, mammals vs. fish, or earth life vs. aliens or AI, it's all the same.
Probably the optimal strategy is to always defect when dealing with members of the out-group, but to have such complete control of their media and education that they don't know you have defected, and continue to co-operate with you despite being stabbed in the back daily. If that fails you can criminalize non-cooperation with you.
Make no mistake, the rapacious megalomaniacs like Larry Page only want utilitarianism for the proles, not for themselves.
The concept of 'AI lives' is absurd. AI is a tool and does what its developers tell it to do.