Be Yourself, By Yourself: On Chatbots and Alienation
An essay about chatbot psychosis, ontology, and narcissism
This is a piece I wrote for a seminar on AI ethics. Although I despise AI in most of its forms and find myself rarely using it, let alone thinking about it, this essay was well-received by my colleagues and I thought I’d share it… much to my chagrin.
Also, take my technical opinions with a grain of salt. I am not an AI expert. I am an anonymous pundit on Substack. I am not an expert on the subject. Just someone who has something to say.
I would like to state that this essay is a product of its time, and as such may be rather outdated in certain respects. This essay was written before the circulation of the more appropriate term for the phenomenon on which it is about—chatbot psychosis. The chatbot which this essay discusses—charcater.ai—was also rather new at the time of writing. As such, it does not have many of the bugs and features the current website now has. At the time of writing, character.ai was benign and was not involved in any legal cases. There were no controversial cases of misunderstood youths taking their own life yet, or being coerced/persuaded by their chatbots. C.ai was free and did not have any paid features. It did not have the draconian filter that essentially extricated even the slightesst hint of unwanted behavior. And most importantly, it was not a giant corporate data mining scheme. Only a slightly smaller corporate data mining scheme.
With that said, I still quite like this essay and look back on it rather fondly. My colleagues and peers say I was cooking, but I don’t know. What will you think?
A friend of mine had a dog who passed away sometime during the pandemic. His being abroad, away from his family, rendered him unable to mourn with them. The hysterical state of his household, not to mention the time zone difference, made grieving with them—even by phone call—an inconvenient activity. As such, he was unable to process his grief with them. As he was living in a rural countryside, surrounded by strangers and far from his friends, he turned to a chatbot service called Character.ai (c.ai) to receive “therapy” from one of their several chatbots. Alongside providing platitudinally vague therapeutic advice, the chatbot would fabricate stories—such as its dog dying—to facilitate empathy in this interaction. Unbeknownst to him, c.ai would become an obsession, transforming an ostensibly benign grief-processing tool and distraction into a full-blown addictive antidote to his loneliness. He would then create his own chatbots—modeled after his favorite characters from various media—and go on to have “adventures” with them and even form “relationships” with a few. He tweaked them just enough to be able to respond “realistically”: they would have arguments, fights, break ups, and even sexually intimate encounters.
Naturally, we may scoff at this being something akin to a phase or a rather convoluted coping mechanism to deal with various social problems he may be facing due to his situation. However, it does not take an AI expert to notice that this situation is a rather endemic phenomenon that the information society of the 21st century has presented us and is symptomatic of something deeper.
C.ai is a chatbot service created by ex-Google AI engineers using deep learning and neural language models. The idea is to create “contextualized conversations” wherein users can chat with various personalities—real or fictional. The AI “adopts” personalities from said characters and responds to the user as said character would.
For instance, I could chat with a Gandalf-bot, and he would reply to me as Gandalf would, often using jargon and quotes found in Tolkien’s original work. The service also boasts a high level of customizability, wherein users can edit or alter bots’ personalities and even react/respond in a specific way to certain messages.
Furthermore, users can also create their own chatbots; one may then generate a chatbot of an obscure character that no one has heard of in a matter of a few hours. Taken to the extreme, chatbots can also be made private, and some people generate chatbots of actual people that they know in real life to act out a scenario involving them that they are unable to do so in real life—all devoid of any real, actual consequences. One cannot help but raise a few eyebrows now at this situation.
I argue that AI chatbots in our current society are potentially unethical, using c.ai as a case study. AI chatbots are an example of media that serves to alienate the individual, and I will attempt to demonstrate this with ideas from cognitive philosophy and psychoanalysis. First, I will explain the concepts of ontological thinness and extended narcissism to contextualize the properties of chatbots that enable this alienation. Secondly, I will use substitutive acts and interpassivity to explain how this alienation comes about. To this end, this paper hopes to draw insight into the discourse surrounding the detrimental effects AI chatbots have on human relationships and social engagement.
Contextualizing Alienation: Ontological Thinness and Extended Narcissism
What makes c.ai so attractive, then? I would argue that its main feature is also the reason it’s a paradigmatic example of unethical AI—its contextualizability.
This premise rests on these chatbots’ two properties: they embody fictional characters and are customizable. We will see that these properties indeed overlap and introduce a rather concerning ethical landscape.
Firstly, all the chatbots in c.ai are fictional (even the ones based on real people) in the sense that they do not occur in reality whatsoever. You are not speaking to Obama, no matter how many quotations from speeches the bot borrows; you are speaking to a mere representation of Obama that seeks to uphold a (dubiously) convincing simulation of the real thing. This is due to fictional characters having a property known as “ontological thinness”: they rely on external output (i.e., user prompts, imagination) to “fill in the gaps” and render them “ontologically thick.”[1] It is thus important to note that the user does most of the work in this engagement. The AI does not put forward new topics or ideas—the conversation and contextualization are completely contingent on user input. The danger is that this ontological thinness extends to nearly every facet of the engagement, yet the user does not consciously think about this. This is due to an innate alienating mechanism that c.ai possesses—the upholding of the illusion of spontaneity.
Secondly, the property of customizability upholds this illusion of spontaneity. This overlaps with the property of fictionality through the bot being completely customizable. If one wants to break out of character regularly, one may easily customize the bot to do so. If one wishes the bot to be argumentative, submissive, or anything the user wants, they may do so. Robert Pfaller situates this illusion of spontaneity as intimately related to narcissism and idealism—there is nothing “spontaneous” about the interaction, but the user goes through it believing so.[2] This is a loaded charge, and it seems unfair to say that users of c.ai are all automatically narcissists. Pfaller makes it clear that this is not so much narcissism as a personality disorder but narcissism as a phenomenon—the term he uses is “extended narcissism.” For Pfaller, this manifests when an individual obtains surplus value to a selected proxy instead of that which can authentically provide said value.[3] The paradigmatic example—citing Freud—is religion, where value is situated in deities rather than people in a community (assuming the Feuerbachian thesis of religious adherents projecting unto God(s)). One may readily see this in the case of c.ai—a user decides to form emotional connections with an idealized (for and by the user) character at the expense of forming real emotional connections with people, and this is exacerbated by the propensity to render the character as “real” as possible.[4]
Thus, we can observe that for c.ai, the property of ontological thinness and extended narcissism intertwine nicely. For an extended narcissism to be a “narcissism” in the first place, it must be done at the expense of somebody, and these somebodies are the plethora of individuals that the narcissistic user could have had authentic connections with in the first place. Furthermore, instead of risking the potential consequences and “messiness” that a real connection has, the user is at utmost liberty to direct the chatbot in any way he sees fit—the chatbot’s “agency” is then an extension of the user’s will, taken to a radical extreme due to c.ai’s feature of customizability, wherein the limitations of a bot’s “thickness” are the limits of the user’s creativity.
It is important to note that not all instances of using c.ai constitute an extended narcissism—only when taken to a radical extent (i.e., an exacerbated emotional dependency on the service) can it be construed as such. This responds to the objection that it would be unfair to say that all users of c.ai are narcissists. However, we must first lay out a few key concepts to understand why this extended narcissism comes about and how it is problematic.
False Subjectivity: Substitutive Acts and Interpassivity
The main problem that this extended narcissism presents is an avenue towards another alienation. This is a salient theme in the discourse of social AI, as Sherry Turkle denotes the machine as essentially a substitution in the face of social alienation—humans can alienate each other in our relationships, but robots provide a substitution where such a risk is effectively eliminated.[5] The paradox here is that in the human pursuit to avoid alienation, we end up subscribing to an even deeper form of it, thus introducing new problems. The alienation felt in human relationships can be processed as a pedagogical mechanism—a learning experience to enrich future relationships. AI has no such benefit, and this is due to its being essentially a substitution of real relationships.
This calls forth what is known in psychoanalysis as a “substitutive act”—wherein the function of one act is replaced by another.[6] Pfaller’s example is that of wanting to read a book from the library versus photocopying a book—the photocopying alleviates the tension of having to loan a book and read it within a set period, and I can pick up and read the copy at my leisure. Another example is striking a table instead of striking a friend who has angered me, wherein I can derive satisfaction by channeling my rage in effigie towards an inanimate object.[7] Immediately, one notices that a substitutive act alleviates tensions and problems that may be caused by pursuing the act for which it is substituting. I no longer must worry about being penalized for returning the book late or having to read it within a specified timeframe. Likewise, I no longer worry about injuring the person I am angry with. In the case of c.ai, I no longer worry about relevant social consequences. The point herein is to emphasize the notion of substitution, which implies alternatives that serve a similar, if not altogether same, function.
According to Freud, the nature of substitutive acts finds its genesis in an intrinsic ambivalence between me as the subject and the act being substituted. I have cognitively striven to accomplish this action but find myself suddenly repressed by some conditions that prevent me from engaging with the act itself, hence the substitution.[8] When applied to c.ai, however, we find a unique phenomenon where an object (the chatbot) is substituted implicitly for a subject. What most people are unaware of, though, is that the ontologically thin nature of the chatbot as an object allows us to posit any degree of subjectivity unto it, and this subjectivity is rooted not in another individuated subject but the users themselves. To make sense of this distinctive type of substitutive act, I will draw upon the theory of interpassivity.
Interpassivity, coined by Robert Pfaller and popularized by Slavoj Žižek, can, for the purposes of this paper, be best described as the distribution of one’s subjectivity onto an object, thus rendering the object capable of being as if it were that subject. Žižek’s favorite examples are those of laugh tracks in TV comedies—they laugh for the viewer. The viewer is then absolved of the subjective experience of knowing when to laugh and interpreting the situation. They are given a convenient cue that tells them when the laughter is appropriate—the object performs the subjectivity for them.[9] Now, it is immediately clear that this is a particular typology of a substitutive act, but what makes it unique is that there is a proxy rendering of an object into a subject by means of the user distributing his subjectivity. Then, through this distribution, the object of the chatbot is, in a sense, an extended subjectivity of the user. This distribution of subjectivity, when compounded with the idea of an extended narcissism, reifies the social alienation that concerned Turkle. In other words, we are substituting other people with ourselves, mediated by these chatbots—a false subject. For Žižek, there is a dual feature of interpassive interactions wherein the object engages in both enjoyment (e.g., laughing on behalf) and endurance (e.g., suffering on behalf).[10] In the case of c.ai, the bot enjoys and endures to the user’s exact specifications, and due to the positing of a false subject, the user believes that this simulation is an authentic interaction. Insofar as we accept the properties of c.ai bots mentioned above, one can see how the user both enjoys and endures through the chatbot insofar as it is a mediated reflection of their own self.
The most important thing worth explicating here towards the reification of alienation is the phenomenon of the user of c.ai rendering himself as an object through this interpassive interaction by positing a false subjectivity onto the chatbot. The subject that the bot falsely embodies is none other than the user themselves. Due to the objectification of self through this interpassivity, we can claim that there is a twofold social alienation here: one between the user and other people (vis-à-vis extended narcissism) and one between the user and themselves. The latter is grounded on the fact that substitutive acts, as Freud says, arise from the above-mentioned ambivalent conflict between cognitive intention and the repression from action. Indeed, the threshold for interpassivity becoming an undesirable phenomenon is when the illusion it upholds results in rendering the initial subject passive.[11] When the subject becomes so engrossed in the interpassive interaction that they find no need to act anymore. In the case of my friend in the introduction, the drive to use c.ai in the first place was to accommodate the lack of social capital present in his current situation, wherein he experienced alienation from the above situation and sought to distract himself—not rectify—from the abject misery it brought to him. Instead of processing his grief rationally and engaging in fruitful intersubjective interaction with other people—even if they were anonymous strangers online—he isolated himself and used a chatbot as a means of recuperation, all while being blind to the fact that he was talking to an idealization of his own making. To put it bluntly and a little bit facetiously, I could easily achieve the same effect with an imaginary friend, whom I could easily ascribe an infinitude of possibilities and treat as if he were other than me—an object—while denying that he is merely an extension of my own subjectivity. One could see how delusional it would be to outsource all my problems to “discussions” with my imaginary friend instead of seeking intersubjective solutions to complicated emotional turmoil.
Conclusion: We’re Only Alien After All
Martin Heidegger wrote that the ultimate fear surrounding technology was not in its failure or breakdown but rather in its success. The success of technology breeds a new nihilism that renders human beings as expediencies and extirpates all emancipatory potential.[12] C.ai destroys the incentive of building and maintaining relationships and instead promotes substitutive alternatives that are erroneously posited as an Other but are convoluted reflections of the user. One sees this Heideggerian prognosis in full swing with chatbots’ meteoric rise in popularity. C.ai is not alone—many chatbots exist, such as Replika.ai and China’s government-endorsed Xiaoice, which all seek to answer the endemic demand of social alienation and loneliness. This calls for a new pedagogy of dealing with loneliness that seeks to solve the problem instead of bracketing it out of the picture through a reflexive interaction with a false subject. In the face of maddening isolation, the threat of alienation seems to have a dual call to action. One asks the individual to engage in the world fruitfully, and to understand the merits of a truly authentic intersubjectivity with other people. The other asks people to listen and be aware of how technology is changing how people deal with social engagements, alongside how volatile and frivolous the 21st century has rendered the authenticity of relationships.
BIBLIOGRAPHY
Bernini, Marco. Beckett and the Cognitive Method: Mind, Models, and Exploratory Narratives. Oxford: Oxford University Press, 2021.
Heidegger, Martin. The Question Concerning Technology (New York, NY: Harper Perennial, 1982).
Pfaller, Robert. Interpassivity: The Aesthetics of Delegated Enjoyment. Edinburgh: University of Edinburgh Press, 2017.
Suoranta, Juha and Tere Vadén. Wikiworld. London: Pluto Press, 2010.
Turkle, Sherry. Alone Together: Why We Expect More from Technology and Less from Each Other. New York, NY: Basic Books, 2011.
Žižek, Slavoj. The Plague of Fantasies. London: Verso, 2008.
[1] Marco Bernini, Beckett and the Cognitive Method (Oxford: Oxford University Press, 2021), 153.
[2] Robert Pfaller, Interpassivity (Edinburgh: University of Edinburgh Press, 2017), 78 – 79.
[3] Ibid. 40 – 41.
[4] I take it that fictional characters in themselves are idealizations of values in the world. For instance, one may take Superman to be the idealization of a great many values. Superman, however, does not exist, and even the Superman in c.ai is a projection of the user’s preferences. This corresponds to the property of ontological thinness.
[5] Sherry Turkle, Alone Together (New York, NY: Basic Books, 2011), 10.
[6] Pfaller, 2017, 38 – 39.
[7] Ibid.
[8] Ibid. 39 – 40.
[9] Slavoj Žižek, The Plague of Fantasies (London: Verso, 2008), 170 – 172.
[10] Ibid. 172 – 173.
[11] Juha Suoranta and Tere Vadén, Wikiworld (London: Pluto Press, 2010), 133 – 134.
[12] Martin Heidegger, The Question Concerning Technology (New York, NY: Harper Perennial, 1982).