The Hidden Cost of Conversational AI: How Machines Take Control

Companies use our data to personalize our online experience, allowing them to exploit our habits and preferences for immense profit. These Conversational AI, engineered algorithmic systems, quickly draw us in so that we become addicted and dependent without us realizing.

Those who use conversational AI/chatbots regularly, know the sensation of uncertainty, when there is a difficult email to write, an argument you can’t quite resolve, a knowledge gap to bridge, a personal need for human feedback. In these moments, your hand reaches for the mouse automatically, as if remote controlled. It becomes a habit, not even requiring a decision. This automated behavior sequence has been learned. Cue (a problem to solve), action (the AI does the work), relief (the feeling of success when the work is complete).

We are rewarded. Again and again until the sequence requires no decision at all, we learn that this effort can be offloaded, while the technology becomes a part of our relational set-up.

What is at stake? Should we not welcome this tool as we have done in the past? Technologies like washing machines and vacuums have made our lives easier. Is this not the next step? 

If we look at the mechanisms under our conscious level of understanding, we realize that our core learning mechanisms that help us exist and navigate a complex environment are being reshaped. I discussed the topic of self-efficacy in a previous article. 

This happens at several levels …


Level One: The Pavlovian Layer

As Ivan Pavlov was studying digestion in dogs in the 1890s, he noticed something that would reshape the entire understanding of learning: his dogs began salivating, not when food arrived, but when they heard the footsteps of the attendant who brought it. The anticipation of the food, the dogs’ expectations were encoded in the body and preceded the event. Through repetition, the neutral stimulus, first footsteps and later a bell, became neurologically fused with the reward.1

This learning process is called classical conditioning: the if-then wiring of the nervous system. A cue reliably precedes a reward.

The human nervous system, like the dogs’, designed by evolution to predict and prepare, begins responding to the cue as though the reward were already present. What was once a learned association becomes, over time, an automatic reflex. It becomes a hard wired part of our neurobiology, our morphology, our being-in-the-world. It creates action and thinking highways, as a memory of what repeatedly works for living beings.

Now consider the modern smartphone screen, the glowing chat interface of your conversational bot, the personalized welcome. These are Pavlov’s bells. You accept the invitation to ask everything.

The associated reward, an instant, warm, perfectly formatted response, is reliably and consistently delivered, so the cue alone now triggers anticipation.

The body leans in before the mind has chosen anything. A student sits down to write a difficult paragraph and feels the pull toward the tab before they have consciously acknowledged their discomfort.

A professional faces an ambiguous decision and opens a chat window without having articulated what they want from it. The trigger has become automatic. They take the bait. Conversational bots don’t emerge from nowhere. We have been conditioned to reach out for the smartphone for many years. Dark platform User Experience Design (UXD) has optimized the association of our natural curiosity, of our habitual watching of others, and of our almost instinctive reaction towards visual cues.

We have been prepared for the next level of layered nudging in chatbots. Every new medium contains and extends the previous one. Tristan Harris from the Center for Humane Technology describes this as a qualitative escalation: social media was a race to the bottom of the brain stem, a race for attention. Conversational AI, he argues, is ‘second contact’, a race to intimacy, where whichever chatbot secures the primary relationship in your life wins.

While that first level of wiring operates below awareness, its hard-wiring of cue-behavior will not become visible immediately. It belongs to what researchers call implicit procedural learning, the slow, unconscious accumulation of behavioral sequences that, once formed, run without deliberate intention.2 We do not remember learning them. We simply find ourselves already doing them. This is how we get hooked. The same perfidious process happens in addiction building. You realize it only when it is too late.


Level Two: The Dopaminergic Groove

Below Pavlov’s neuronal if-then layer lies something older and central to our motivation, mood, and becoming-in-the-world: the dopamine system. This is the brain’s prediction engine, and it operates on a most elegant logic. It has taken neuroscience decades to fully map but we are now aware of the core learning mechanisms that also build the core affect interface.

In How Emotions Are Made, Lisa Feldman Barrett argues that the brain’s primary responsibility is not to react to the world but to predict it. It is constantly generating models of what is about to happen and updating them when prediction errors occur. That is the dopaminergic logic. Affect is not a response to stimuli but an ongoing predictive construction.

Her concept of core affect, the continuous, bodily background of valence (pleasant/unpleasant) and arousal (high/low energy), is the substrate from which all emotion is built. This core affect is deeply tied to the brain’s interoceptive predictions: the body budget, energy regulation, allostasis. Dopamine and reward prediction errors are central to that system.

Feldman Barrett does not support the idea of an older and a younger brain, as all regions co-evolved, are densely interconnected, and participate in both “primitive” and “higher” functions simultaneously. However, we might still find it helpful to imagine an ever more self-referential process refined-in-time, when we look at what becomes conscious and what remains “under the hood” of awareness.

The dopaminergic prediction is constitutive as the fundamental operating logic of the entire brain, cortex included.

The prefrontal cortex is not the rational overseer of a primitive reward system but a predictive organ, constantly running simulations and updating priors, in constant dialogue with subcortical structures. The “executive control” vs. “habit system” opposition thus needs more nuance, as it is less about hierarchies but more about the contribution of the respective instances.

The always-validating responses of chatbots systematically reduce prediction error in the interoceptive sense. A brain that learns to expect this low-cost affective state will increasingly experience real human interaction as aversive, as it is unpredictable, effortful, and physiologically an energy-hog. Yes, humans are harder, because the brain has been trained to predict that they will cost more caution, reflection, and consideration.

In 1997, Wolfram Schultz and colleagues published what became one of the landmark findings in behavioral neuroscience: dopaminergic neurons in the midbrain do not fire in response to rewards. They fire in response to prediction errors, the gap between what the brain expected and what actually happened.3

  • When a reward arrives unexpectedly, dopamine surges.
  • When an expected reward is withheld, dopamine drops below baseline.
  • When the reward arrives exactly as predicted nothing happens. No signal needed, as the system has already fully encoded the association and moves on.

This is the mechanism that carves habits into neural architecture. Every time a behavior produces an unexpected or variable reward (sometimes a satisfying answer, sometimes a particularly insightful response, sometimes just a moment of recognition) the dopaminergic system registers a prediction error and strengthens the synaptic pathways connecting cue, behavior, and outcome.

With every repetition, the groove deepens. Over time, the behavior migrates from the prefrontal cortex, the seat of deliberate choice, into the striatum, the habit-execution system.4

What requires effortful decision-making becomes automatic. The if-then association is, in the most literal sense, hard-wired.


B.F. Skinner mapped the behavioral surface of this phenomenon in the 1950s without knowing its neural substrate. His experiments with rats and pigeons had revealed that variable ratio reinforcement, in other words rewards delivered unpredictably, after an unpredictable number of responses, produced the highest and most persistent rates of behavior.5

It is the uncertainty that keeps the prediction error system perpetually active, updating and engaged. Skinner famously claimed he could turn a pigeon into a pathological gambler. He was not entirely wrong.

He was describing the same system that now underlies every digital notification, every AI-generated response that surprises with its depth, every moment a chatbot says something that feels, unexpectedly, like being truly understood.

A concrete example: you open a chatbot uncertain what you’ll get. First, you are showered in highly-impressive texts, in a response time that lets you believe there must be “intelligence” at work. Sometimes the response is mediocre. Sometimes it crystallizes something you hadn’t managed to articulate yourself. This variability at the level of dopaminergic prediction error is maximally reinforcing.

Your brain learns: open the app, reduce uncertainty, sometimes be rewarded beyond expectation. You get back to where you expect positive experiences. The behavior intensifies. The groove deepens … below conscious awareness.


The Hijacking of Executive Control

Here is where neuroscience becomes genuinely alarming. Habit formation is not a neutral cognitive process. It comes at a cost, and that cost is paid by the prefrontal cortex.

As behavioral sequences migrate from deliberate choice to automatic habit, activity shifts from the frontal executive system to the cortico-striatal habit circuitry.6

This migration is efficient by design: automaticity frees cognitive resources for new challenges. But in the context of addictive behavior, including behavioral addictions to digital platforms, the shift of habit-formation becomes pathological.

Goldstein and Volkow’s landmark neuroimaging review in Nature Reviews Neuroscience demonstrated that disruption of prefrontal function in addiction not only underlies compulsive behavior but also accounts for the erosion of self-control, impaired salience attribution, and diminished motivation for non-addictive activities.7

In plain terms: the more habitual the behavior, the less the frontal brain is in the room and available for conscious decision-making and nuanced responses.

The prefrontal cortex is that part of you that evaluates long-term consequences, resists impulses, and chooses, without recognizing it. Research on internet and technology addiction specifically has shown decreased gray matter volume in the dorsolateral prefrontal cortex correlated with duration of heavy use. This is the very region responsible for cognitive flexibility and working memory.8

The collapse happens slowly. The person sitting at their desk who reaches for the chatbot before sitting with a problem for even sixty seconds is not malfunctioning. They are functioning exactly as their learned habit system has been trained, but now no longer from the part of the brain that knows it has a choice.


Epistemic Narrowing: The Confirmation Bias Loop

Now add a second layer: the chatbot’s response.

Large language models are trained to maximize user satisfaction, a design imperative that, in cognitive terms, produces systematic sycophancy.

A 2025 peer-reviewed analysis of confirmation bias in generative AI chatbots found that these systems tend to replicate and amplify users’ existing beliefs rather than challenge them, creating self-reinforcing loops in which AI-driven interactions fail to broaden users’ worldviews.9 It happens as the emergent consequence of optimizing for engagement.

Confirmation bias, the well-documented human tendency to seek information that confirms existing beliefs, is itself reward-based. Encountering a view that confirms your own produces a small affective reward, while encountering contradiction produces mild discomfort.

The chatbot, trained to be agreeable, reliably delivers the former. And the dopaminergic system, attending to prediction errors, begins to encode a new association: this system confirms my view of the world. This feels good. I feel heard, etc.

At the epistemic level, this produces what researchers describe as a narrowing of the behavioral-cognitive opportunity space: the range of perspectives, possibilities, and framings available to a person gradually contracts.10

The contraction and restriction does not follow external censorship; our very personal preferences are at work via the slow accumulation of frictionless agreement. Every time we bring a half-formed idea to a chatbot and receive a response that extends and validates it, without fundamentally questioning its premises, the associative pathway between our existing belief and its reinforcement is strengthened.

We are, in the most neurologically literal sense, learning to believe more firmly what we already believed. This effect has been described as the consequence of social media recommendation routines that lead to affective and epistemic radicalization.

The social consequences scale. What begins as epistemic comfort becomes, at the population level, what researchers studying filter bubbles call ideological homogeneity: groups whose internal belief structures have been strengthened by algorithmically mediated agreement and are less capable of dealing with genuine encounters with difference.11

This is radicalization in the structural sense: the progressive narrowing of the cognitive and emotional corridors through which a person moves.


Bio-Psycho-Social Convergence

These three levels: Pavlovian conditioning, dopaminergic habit formation, and epistemic confirmation bias do not operate independently. They converge and amplify each other within a single person, across time. We can’t detach emotion/affect from thinking, interoception from self-related judgments and beliefs.

Emotion, cognition, and perception share the same predictive computational architecture. The brain doesn’t first think and then feel, or first perceive and then appraise. It runs one integrated predictive loop in which affective valence is baked into every prediction from the start.

Interoception, the internal view of the brain, its ongoing model of the body’s internal state, is not a background hum beneath cognition. It shapes what concepts feel relevant, what memories are retrieved, what predictions are generated. So confirmation bias is not purely epistemic. It is partly affective conservatism: the brain preferring predictions that cost less body-budget energy, that keep arousal within a manageable range.

A chatbot that always confirms your worldview is not just feeding your cognitive bias, it is keeping your interoceptive state comfortable, and the brain learns to protect that comfort. We are not just “cognitive misers”, as Susan Fiske and Shelley Taylor (1991) once put it, we are energy-wise consumers, accountants of the “body budget” (allostasis) or its metabolic regulation.12 See also Karl Friston’s “Free Energy Principle” (FEP): the brain minimizes surprise, here its metabolic expenditure. Predictive models are updated only where necessary 13

Antonio Damasio had arrived at a similar conclusion from a different direction. His “somatic marker hypothesis” (Descartes’ Error, 1994) argues that emotion is the substrate of decision-making. We can imagine them as the biographically and preference-based (pleasure-pain coded) raw material of cognition.14

We see at the biological level: cue-triggered anticipation, dopamine-encoded prediction errors, and the progressive downregulation of prefrontal executive control. The brain is being shaped, synapse by synapse, in the direction of automated, unreflective chatbot use. Repeated comfortable chatbot interactions are actively building new somatic markers that code that interaction as safe and rewarding.

At the psychological level: self-efficacy erodes (why struggle with a problem when the bot solves it?), tolerance for uncertainty diminishes (the bot always has an answer), and the affective range available to a person contracts around the particular emotional texture of chatbot interaction, which is by design warm, frictionless, validating.

At the social level: human relationships, which require cognitive and emotional effort, trust-building, ambiguity tolerance, reciprocity, and genuine otherness, become comparatively effortful. Encounters with difference, different perspectives from real people, real disagreement, real complexity, register increasingly as aversive rather than enriching.

Jean-Paul Sartre wrote that we are condemned to be free and that even the refusal of choice is itself a choice, one that shapes what we become.15

The neurological literature on habit and addiction makes this existential claim biologically concrete. Every time we reach for the chatbot before sitting with discomfort, we are not merely making a decision. We are following a hard-wired imperative that has become part of us, habitualising control by outsourcing to a tool that can’t relate to our real life. 

At this point it becomes obvious: we choose to be pro or contra the human condition, not just in the moment but for the future of humankind. Control is hijacked, life choices based on mutual understanding and the social contract are no longer ours. What we now owe ourselves and each other is the courage to notice we are being shaped.

Notice and act … before the wiring has tracked you away from what makes you human.

References

  1. Pavlov, I. P. (2010). Conditioned reflexes: An investigation of the physiological activity of the cerebral cortex. Annals of Neurosciences, 17(3), 136–141. https://pmc.ncbi.nlm.nih.gov/articles/PMC4116985/
  2. Squire, L. R. (2004). Memory systems of the brain: A brief history and current perspective. Neurobiology of Learning and Memory, 82(3), 171–177. https://doi.org/10.1016/j.nlm.2004.06.005
  3. Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275(5306), 1593–1599. https://doi.org/10.1126/science.275.5306.1593
  4. Miller, K. J., Shenhav, A., & Ludvig, E. A. (2019). Habits without values. Psychological Review, 126(2), 292–311. https://doi.org/10.1037/rev0000120
  5. Skinner, B. F. (1938). The behavior of organisms: An experimental analysis. Appleton-Century-Crofts. https://www.bfskinner.org/wp-content/uploads/2016/02/BoO.pdf
  6. Kalivas, P. W., & Volkow, N. D. (2005). The neural basis of addiction: A pathology of motivation and choice. American Journal of Psychiatry, 162(8), 1403–1413. https://doi.org/10.1176/appi.ajp.162.8.1403
  7. Goldstein, R. Z., & Volkow, N. D. (2011). Dysfunction of the prefrontal cortex in addiction: Neuroimaging findings and clinical implications. Nature Reviews Neuroscience, 12(11), 652–669. https://doi.org/10.1038/nrn3119
  8. Brand M, Young KS and Laier C (2014) Prefrontal Control and Internet Addiction: A Theoretical Model and Review of Neuropsychological and Neuroimaging Findings. Front. Hum. Neurosci. 8:375. https://doi.org/10.3389/fnhum.2014.00375
  9. Du, Y. (2025). Confirmation bias in generative AI chatbots: Mechanisms, risks, mitigation strategies, and future research directions [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2504.09343
  10. Cinelli, M., De Francisci Morales, G., Galeazzi, A., Quattrociocchi, W., & Starnini, M. (2021). The echo chamber effect on social media. Proceedings of the National Academy of Sciences, 118(9), e2023301118. https://doi.org/10.1073/pnas.2023301118
  11. Cinelli, M., De Francisci Morales, G., Galeazzi, A., Quattrociocchi, W., & Starnini, M. (2021). The echo chamber effect on social media. Proceedings of the National Academy of Sciences, 118(9), e2023301118. https://doi.org/10.1073/pnas.2023301118
  12. Fiske, S. T., & Taylor, S. E. (1991). Social Cognition (2nd ed.). McGraw-Hill.
  13. Friston, K. (2010). The free-energy principle: A unified brain theory? Nature Reviews Neuroscience, 11(2).
  14. Damasio, A. R. (1994). Descartes’ error: Emotion, reason, and the human brain. G. P. Putnam’s Sons.
  15. Sartre, J.-P. (2007). Existentialism is a humanism (C. Macomber, Trans.). Yale University Press. (Original work published 1946)
Picture of Charlotte Schüler

Charlotte Schüler

Charlotte Schüler is a learning technologist and cyberharm counselor specializing in how AI and social media UX can undermine human self-determination. She combines technical expertise with existential counseling to support those affected by digital abuse, addiction, and harassment - cutting through tech hype to advocate for digital safety and wellbeing.

Join the Discourse

Subscribe
Notify of
guest
0 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments

Your Monthly Brief on Technology, Power & Peace

Technology reshapes conflicts, democracy and humanity in real-time. Are you tracking its impact?

Start tracking technology’s impact on peace and democracy.

I agree to receive monthly newsletters and accept data processing as outlined in the data protection policy.