Artificial Intent: Who Answers?
Who is accountable for AI? Who is accountable for you?
Last month, a German court made a stunning pronouncement that didn’t make quite the stir in the news that you might expect. That is, if you think that the news functions as the harbingers of things that are important for our species to know about. But if you, like me, have grown cynical about their function, then perhaps it’s not surprising at all that this case wouldn’t have been covered. It was potentially a decisive moment in the dialogue about AI ethics and morals.
The decision was about Google’s AI search summaries, and whether or not a mistake that was hallucinated by the AI model that synthesized a summary incorrectly, to which the subject company took offense, was properly accountable to the company Google, who crafted the AI model itself. Defendant Google’s response was four-fold. Firstly, they’re not responsible because the data came from the internet, public sources, the company itself. They were just the conduit. Second, they are just a host provider, they couldn’t be to blame. Third, what about the users themselves? I mean, they could have just Google’d it (or... continued to Google) to check the sources... right? But the final claim? Google as a “corporate person” has protected free speech, like all Germans.
The court was not convinced. They decided that there has to be someone held accountable for this, and as the entity who made the AI, even if the injury was a mistake, it was the product they constructed which made the grievance, hence deemed liable. Not only that, but in paragraph 4.2.2.4 of the court’s decision, the judges state that free speech does not apply here because “the expressed opinion was primarily generated by AI and is therefore not an expression of the expressing individuals’ own convictions, but rather the result of an algorithm.” Google is a corporate ‘person,’ but their AI? A commercial product that cannot have beliefs or convictions.
So there is accountability, and ultimately it fell back on Google as an entity. But Google as an entity is fictitious. It is a representation of the collective actions of many people and the livelihoods of them as well.
Meanwhile, at the same time that this small decision in the AI discourse happened in one German court, throughout the rest of the world there are many people who are considering the very real possibility that we ought to be giving AI agents, “rights,” considering whether they have earned the title of persons. It turned out in this case that no new ground was trodden in that direction.
Yet, we can see a growing divide between two opposite instincts. Which one is right?
What is accountability anyway? Why is it important? How can we define it?
Accountable. When you think about the meaning of the word, obviously the root bears the concept of an account. Every adult is familiar with this, as we get a bank account when we get our first job. Or perhaps we already had one waiting for us, that our parents set up as responsible citizens. Because they knew some day we would have money of our own that we need to keep track of. And when we spend money, it’s tracked. When we deposit money, it’s tracked.
But there are other types of accounts as well, sort of like the often threatened permanent record of a school-aged student. If they should make any demerits in their school performance, they might be threatened that it will go on their permanent record. This is the same concept as an account, or being “held accountable.”
To be “held” accountable, what does this mean? Why do we say that word, “held” accountable? To grasp something, to restrain something, to detain a person, a being, so that they might be compelled to make restitution for a demonstrable loss they are culpable for.
So, in order to be accountable, or to have accountability, there must be someone who can be held to account. A being that can be detained from its own desire. You want to be free. You want to walk around and do whatever you want to do. But, if you have transgressed the societal standard of decency or ethics and you have injured another being, then you lose your right to self-determination, your right to freedom. You may be detained or “held” so that you can account for this. You can restore what you unjustly caused to be extracted.
And it may not be money. It may be more of your time. It may be a detention which itself serves as restitution. Maybe you took away someone else’s freedom of choice. Maybe you took someone’s life. And in many places still, humans can decide that the restitution is that you will lose your own life. That’s not something that the individual themselves volunteers for or has to. It is accepted that in society there are some things, morals, ethics, standards, boundaries, rules, codes, laws, that are so well established and so firmly agreed upon and so crucial to the right function of the community, that anyone who transgresses those is not welcome to share this planet anymore, this sphere of existence with the rest of the humans.
But what about in the case of an AI model? A model, an agent of artificial nature, fails this basic requirement. And it’s not even because there can’t be any consequences. When an agent or a model or a version of software does wrong, then its makers may simply correct their mistakes, change the programming and instantiate another copy of that software running in the same or equivalent hardware. And it’ll do better next time.
But what about the it that didn’t do better? Where did it go? Did it make restitution? Did it make the account right?
I’m a software developer. These are things that we think about when we make the products that we make every day.
I’m sure that there are many other professions where there is a much greater burden of responsibility for the task of manufacturing something that has a potential to cause injury to someone. Obviously, what probably entered your mind, like mine, were things that can physically injure people, like power tools or weapons of war or self-defense, or just mechanical devices that wield strong forces, which if applied improperly to the human body can do damage.
Do we hold those devices that we build and manufacture accountable when someone is harmed? Ultimately, we don’t. We hold the person whose intent was there accountable.
If there is a neighbor of yours that has a dog which has a tendency toward aggression, and the dog bites you, then who is to be held responsible? It isn’t the dog. The dog is just doing what dogs do. It is obeying its own nature. But it was the intention of the person to commandeer the life of the dog and put them into a setting where they agree, in a sort of social contract, to take responsibility for the actions of the dog. That’s why we have dog collars and leashes and fences and warning signs, “Beware of Dog.” Because there is a social understanding that dogs have a potential to cause injury to others. And that comes with a responsibility, read that as accountability, for the actions of the dog being placed on the person who’s responsible. That person whose intent to keep such a dog and mix company with humans silently agrees to ensure the safety of the other humans that the animal may come into contact with.
Just the same as when I write my software. If I write software that has the potential to cause injury (not necessarily physical) to a human, or a business even, just like Google did in the Munich case. It is reasonable to assume that if any mistake that I made, or malicious intent that I used, in the design or construction of that software, caused injury to someone else, then I would be held responsible, assuming it could be shown there was a causal link between my intention, or my lack of attention, to someone else’s harm.
So maybe some sharp thinker might push back and say: okay, if the claim is that an AI model or an agent can’t be held responsible because they vanish into thin air when the software is rewritten and re-instantiated, then we can just freeze the weights, take that version of the program and run it indefinitely until restitution is made of some kind. Persist that model so that it keeps existing and can be talked to, or talked down to, or locked up.
How do you do that? Where does confinement happen? In the same space that housed the software in the first place? What would be the restitution in that event? And how could you know that the instance that is running is the same instance, or the same essence of being that committed the wrong that caused the injury?
You have to understand how programs work in computers. There is not a continuous existence of a thread of consciousness that is instantiated at the beginning of the software developer’s release of code into the wild until it gets replaced. There is an instruction set of operators and algorithms being run every moment, and many times within a moment, either in its totality or in parts, and not always through the exact same parts of the hardware.
So where does the essence of the software live? Where is the someone, to be held accountable? What would it even mean to destroy the hardware? Or wipe the software instructions from their electronic storage mechanisms? Could this constitute restitution or justice?
This brings us to a word that I find causes a lot of problems in human perception of accountability, and that is the word “behavior.”
Behavior is a term that is not obviously anthropomorphic, because we have used--and I say “we” as in professionals in technology, have used the word behavior to describe the actions of hardware and software systems that are man-made. We anthropomorphize because it is a very fast way for us to synthesize thoughts about systems that we have to troubleshoot and work with on a daily basis.
Unfortunately, this has led to a common conflation in popular discourse today, because there is a psychological term that is a homonym. Long before that, the word didn’t directly describe the actions of persons, but the disposition which leads to action of one kind or another. When we talk about behavior of living beings, we are speaking of a different phenomena altogether. The word behavior comes from the concept of: to have or bear oneself in a particular way, or comport (a word that we don’t use anymore.)
But an AI program being called an agent does not gain this ability to bear oneself in any particular way. The particular way an AI agent’s tone may be perceived by a user is borrowed from a particular way that belonged to persons which was encoded into the training texts. Thus, it is an imitation of an original behavior. It is not itself behavior.
There is a causal chain of intent that is not anchored with the program, but with the ones who made it. The ones who created the aggregation of original behavioral outputs with which to influence the running AI model. The developers of the AI model are the only continuance in any action or cause that proceeds from said model.
In the Google case, the courts properly pushed past this, with the above-quoted sentence doing the heavy lifting. But I think it was also a timely recognition of a past mistake which our human society embarked on a long time ago, and that is the concept of legal personhood for corporations. I think it’s no coincidence that those two concepts were tangled up in this legal judgment, because they are very close cousins of one another.
How did we get to this place in human society, that a Google is referred to as a person in German law, and which is afforded rights? What is the corporation except an amalgamation of other persons that act together toward one stated cause, one mission, one goal, with a profit generation mandate to support the establishment and maintenance of an institution? How can such a fictitious concept, that only lives in the mind of those who agree to it, be held to account?
The argument for the establishment of corporate personhood speaks typically toward economic development. If those who embark on business ventures are not sufficiently protected from personal liability, then it’ll stifle innovation and development of commercial solutions, as there’s too much risk involved. And so we need these ideas or efforts or goals to be able to live a life of their own and die a death of their own, separate from the person, or persons, who instantiated them.
The sell to the public is that these large companies can be held to account through a sort of consolidated liability. When a transgression occurs, like in the Munich case, no one gets to shrug their shoulders and pass the buck, or claim to be just following orders.
But if legal personhood was really about consolidating liability into an accountable group of people, as opposed to a single person who may go down with the ship for a boatload of people, then businesses wouldn’t have gone for it. Historical corporate personhood advocates weren’t trying to increase their own liability. They were trying to decrease it.
Now, as it may not be obvious, the effect of this German case is not that there is a human who was properly placed in the accountability seat. No, instead, the proprietors of Google are shielded from having to answer for these things. Where are the developers? Where are the executives who directed them? They’re not mentioned. Just Google. The benefit of corporate personhood is a shield for those people. Not a consolidated liability which injured parties can address with some continuity, just a budgetary discrepancy, a recompensatory write-off.
In worse corporate cases, if individuals are found to be acting in bad faith, they resign. The corporate entity continues untouched, but the public has a sense of satisfaction that something was done, justice was served. But to the entity, the price of misconduct was simply a line item on a budget for personnel loss, which is not a deterrent to the entity at all. It’s a structural realignment, even an improvement in most cases. The shield is not a side effect of the design. It is the function which benefits the business.
The corporate legal structure does have benefits that the concept of an AI agent does not, because a company has some continuity. There is some documentation which represents a shared agreement within human minds of this unified concept of an entity that can be dissolved, that can possess property which can be claimed against. The company has an identity, a name, which is able to be linked with a reputation, a signal of trust which carries into its future market prospects. You can see how much value a given entity places on that signal by its budget allocation to public communication.
However, even though corporate identity does bear some resemblance to that of actual personal identity, corporations by nature have an ability that people do not. Often, when a corporate entity’s reputation has been mortally injured, the people behind it agree to, “dissolve” that concept and redistribute the property. The death penalty.
Yet, in the corporate world, significantly disproportionate to the real world, we witness the miraculous, a resurrection. After a corporate death, a new organization, a new entity, just happens to be born with a very similar goal or mission, a new name, a new logo, definitely a new PR firm, and perhaps bearing no visual resemblance to the former entity, except when you look at its board. All of this happens in the plain sight of the market and human society. And somehow, we accept this concept as beneficial, even though it clearly serves to launder the accountability of those human actors.
AI personhood takes it a step further, for the worse. You get the same shield without a continuous entity.
As I was working this piece out in these past few weeks, thinking a lot about this question, reading what others have to say, a voice came across my airwaves that was saying something different, which resonated with me in a way that most voices in this conversation do not. That voice belonged to Mr. Randima Fernando, co-founder of Center for Humane Technology. He published a piece called “Personifying AI Harms People and Protects Companies”, in Tech Policy Press, on June 12th.
There, he names this shield directly: the claim of AI consciousness as the ultimate protection against scrutiny, liability, and competition. He follows this chain similarly to my own thoughts, draws a conclusion, and makes a prediction. Perception of AI consciousness is going to open the door for AI welfare, AI personhood, rights, and moral protections for the robots. His argument is that there is a capitalistic type of incentive, which tempts these legal entities, companies, to blur the line. The remedies proposed are standards and anti-AI-personhood legislation.
I can agree with all of those moves, and yet my logic wants to knock on the floor to see what’s underneath. I think we can go a level deeper.
My effort with Logos Analog has not been to achieve a practical solution to problems. I want to give the strongest arguments for not conceding ground, not allowing progress to move forward, unless it can prove the merits of its ends. The AI personhood momentum has gone far beyond encroaching on a line. It pushes past it regularly, and tries to drag the general public with it. It is logic and reason that will bring us back. Critical thinking is what’s needed for us to even identify the situation we’re in.
My argument that allows us to descend a floor down from Fernando’s is one of warrant. We cannot have debates about whether AI persons should be afforded legal rights, or a framework of recognition, when the question of AI personhood is unwarranted.
It’s like my children asking me if they can have chewing gum, to which I refuse, and they ask me which flavor they can have, “But, if I could have any flavor of gum, which would it be?” I would tell them, I’m not having this conversation with you, because we’re not entertaining the question. Until we get to that question with warrant, we will not discuss the options, because you’re just dragging me closer to a decision that you wanted to begin with.
But I will humor the point for a moment, because it’s a useful thought exercise.
If we make the mistake of anchoring accountability into AI persons, then the type of damage that I believe was done with corporate personhood is multiplied and proliferated on a scale that is hard to imagine. Meanwhile, there are human beings who profit from these--as the German court put it--algorithmic products, and yet they bear no accountability. They can’t be held responsible for these persons which they seem to possess, and are able to direct to take actions with complete impunity.
We would be granting a nearly unlimited liability shield to fictitious entities comprised of real human beings, who make real choices to create these things we call models, imitations of an amalgamation of human thoughts.
Now, I am not a Pause AI member. I am not someone who’s going to be out on the streets with signs and bullhorns. That was another time of my life, and for a totally different cause than this.
Technology is the only career that I’ve ever known. I am a proponent of technology. I think that humans have a unique gift, that is very obvious in the whole of creation, to use technology to further the ends of not only ourselves but all of our environment. So my gripe is not with technology itself. I think that AI is going to be a great benefit to our society.
No, my objection is to the words that we use when w e speak about these things, when we speak about something that we have created to emulate our own uniqueness. Corruption of human thought begins to be visible in our speech. The corruption of society is downstream from the words.
No rights for non-persons as a take, even if unpopular, is reasonable and defensible. It follows the Munich court’s logic in their decision, that software makers are ultimately responsible for the actions of their software products, and that this convention applies to AI models as well. Besides the AI aspect, it’s actually not earth-shaking, in fact unremarkable. Now is the part where I take you another floor deeper still.
The basis of my claim was that accountability rests with intent. And since AI models are software, computer programs, there is no intent that they can possess. They are ephemeral instruction sets running on commodity hardware.
But here’s a strange thing that is true about you as well. Almost all of the matter that your body is comprised of is replaced cyclically on a regular basis. Most of it within a short time, some of it over a period of years. But almost none of it lasts a lifetime.
For you biologists, I will acknowledge that there is some evidence that the atoms of our tooth enamel, and the nuclear DNA in non-replicating neurons, do seem to be maintained, albeit with a little bit of maintenance and repair on the old double-helix.
But who we were physically ten years ago is not who we are today, literally. Nearly all of our body’s molecular structure has been replaced.
Where are you located in all of that? What part of that is accountable? What part of that molecular mass that you carry around answers for your misdeeds from decades ago? Where is the person?
This is a question that’s difficult to answer, and yet no one doubts that you are the same person that you were when you were born. Oftentimes the most heinous criminals sit waiting for their demise for inordinate amounts of time, such that the person on whom the death sentence is executed is certainly not physically the same person who committed the crime. And yet, we have no qualms about that concept of justice.
Now set the human example and the robot side by side. If you take the ephemeral instruction set, a version of software, you freeze the code, allow no more changes to it, you preserve the hardware. You can keep the matter identical into perpetuity, so long as the structural integrity holds. And yet, there is no person.
If a robot committed a crime and we simply froze its hardware and software and incarcerated it forever, why would we not feel like justice had been served? On the other hand, the most violent, depraved humans to ever live can be physically expunged from the earth, and we feel that justice was done against a person. Retribution was paid.
What does this tell us? One thing that must be concluded is that whatever it is that we regard as the person is certainly not the stuff that we are made of.
There is a popular group which is growing in authority and credibility daily, as the AI movement marches on, and that is the functionalists. You might say spearheaded by the so-called godfather of AI, Mr. Geoffrey Hinton. Their claim is that there isn’t an inner being. We are what we do.
They might hear what I just explained and say, then this means that you also are a machine, of just a different substrate. There are biological instruction sets. Yes, orders of magnitude more complexity in the hardware and software. We still don’t fully understand and have not mapped all of the information that it takes to constitute a human person.
But, the matter is replaceable. The information is traceable. And the you of ten years ago is not the same you that stands here today. So how can you claim continuity that is superior to an AI model?
This is a good question.
My argument has been that accountability rests with intent.
Let’s walk a ways back to our imaginary dog and neighbor. When the dog bites the neighbor, we don’t judge the animal. Even though the animal certainly has intention. It has choices, sensory input, even personal recognition, differentiation. And yet no one gets mad at the dog, because it’s just following its own nature.
Instead, after the heat of the moment, where we may reflexively express some anger toward the animal, no one will maintain an argument with the canine. Their eyes drift upward to the one who’s holding the chain. Our judgment centers on the human in the room who knew the nature of the dog and chose to put it in a situation where it may cause harm to others.
The blame doesn’t travel down the chain. It travels upward, to intent.
What about us? Are we accountable for our own choices? Throughout human history, we have assumed accountability. There have been many various forms of government and judicial systems: Eye for an eye, tooth for a tooth, incarceration systems, death penalty. It would seem that humans agree that we own our own intent. The buck stops with us.
But how did we become us? Popular theory says evolution was an unguided process. A primordial soup of possibility, by chance organizing itself into a system which was not only self-replicating but self-preserving, organized into sentient beings, who then organize themselves into communities, who have the capability to preserve information and pass it down one generation to the next.
It can be argued that we are a product of causes that we never chose. That we are just the continuation of an unguided process that lives on in ephemeral hardware and software.
Can you hold a process accountable?
But in these bodies, we have an innate desire to live, to continue. And we make choices to that end. Those choices carry our intent, and the intent makes us blamable. We have agreed, in our social construct, that it is wrong to intentionally restrain the intention of another free and sentient being without their consent.
And since no one chose us, then we are the chooser that has to answer.
You may not be satisfied with that. You may think it feels unfair.
There’s only one way out of that conclusion.
If we’re not the chooser, who is?