Sunday, 2 March 2025

John Bateman Misrepresenting LLM Operations

So the token strings we see produced in Chris's posts are from one particular area of such a landscape, one where the plausible next token metric appears to have been shaped in relation to Chris's own blogposts or previous interactions with the system. This can of course be used for generating 'personalised' output: both on the production side and the reception side, but the buck always passes to the person who choses to place such output in a social context since agency is currently not on the table. 
the AI posts are tokens generated from a local ChRIS CLÉiRIGh-landscape, including any biases and tendencies for language patterning that were present in the refining prompts (which we have not been told about). So it is not just strings compatible with 'a' K1 role from the landscape of institutional authority, but a very specific part of one area within such a landscape. 


Blogger Comments:

[1] This is misleading, as ChatGPT explains:

Bateman’s claim contains several assumptions that don’t hold up under scrutiny:

  1. The “One Particular Area” Misconception – Bateman assumes that the responses in Chris' posts originate from a fixed, localised region of the model’s representational space. However, transformer models do not operate by retrieving text from a specific, static area. Instead, responses are dynamically generated by weighting thousands of interrelated patterns, including broader linguistic structures and immediate contextual cues. The model doesn’t “reside” in a personalised subregion of meaning but constructs responses based on complex probabilistic calculations spanning its entire learned network.

  2. Implies a Direct Influence from Chris’s Blog – The suggestion that the plausible next-token metric is “shaped in relation to Chris’s own blogposts or previous interactions” is misleading. While a model can be influenced by repeated interaction patterns in a given session, it does not store personalised memory or maintain persistent knowledge of specific blog posts unless they are explicitly reintroduced in the conversation. The phrasing here makes it sound as if Chris' posts have directly fine-tuned the model, which is incorrect.

  3. Mischaracterises “Personalisation” – Bateman concedes that personalisation occurs “both on the production side and the reception side” but frames it as an external process imposed by users rather than an emergent feature of language modelling itself. In reality, language models already generate responses tailored to input context by design, leveraging learned discourse structures, stylistic markers, and interactive cues. The output isn’t merely “placed” into a social context—it is actively shaped by that context through model weighting.

  4. The Agency Cop-Out – Bateman ends with a familiar rhetorical move: shifting responsibility entirely onto the user because “agency is currently not on the table.” This is a sleight of hand that sidesteps the issue. While AI doesn’t possess independent agency in the human sense, it does shape discourse through probabilistic decision-making processes that are far more nuanced than simple mechanistic generation. Reducing everything to a human “buck-passing” exercise obscures the real discussion about how meaning emerges in AI-mediated communication.

In short, Bateman’s description misrepresents how models generate text, oversimplifies the role of context, and conveniently sidesteps the complexities of AI-mediated meaning-making.

[2] This is misleading, as ChatGPT explains:

This section reinforces Bateman’s earlier misconceptions while introducing some new rhetorical moves that deserve scrutiny:

  1. The “Local Chris Cléirigh-landscape” Fallacy – Bateman frames the model’s responses as emerging from a “local” area tied to Chris' blog posts and interactions. However, language models do not operate within isolated “pockets” of meaning. They generate responses dynamically, drawing from statistical associations across an immense, high-dimensional space. While a prompt can influence output, there is no self-contained “Chris Cléirigh” zone within the model. This framing suggests a deterministic and constrained process when, in reality, the model integrates broad linguistic patterns with the immediate conversational context.

  2. Implies a Hidden Influence via “Refining Prompts” – Bateman hints at some undisclosed manipulation by referring to “refining prompts (which we have not been told about).” This is a rhetorical strategy meant to suggest that something has been deliberately concealed. However, the core process remains transparent: prompts shape responses based on probabilistic weighting, but they do not create a fixed, personalised sub-landscape. The model does not store specific refinements unless explicitly fine-tuned, which is not the case here.

  3. Confuses Bias with Local Patterning – Bateman claims that responses inherit “biases and tendencies for language patterning” from refining prompts, but this conflates two different concepts. Bias, in AI discourse, refers to systemic distortions introduced during training (e.g., societal biases in data). Local patterning, on the other hand, refers to the influence of immediate linguistic context. The latter does not imply the former. If a prompt encourages a specific register or style, that does not mean the model is trapped in an ideological or cognitive framework—it is simply adjusting its output probabilistically.

  4. Mischaracterises the K1 Claim – Bateman asserts that the AI is not producing strings compatible with just any K1 role (i.e., an authoritative voice in a field) but a “very specific part of one area within such a landscape.” This seems intended to undermine the idea that the AI’s responses could reflect legitimate K1 discourse. However, K1 status is not defined by source locality but by coherence with established knowledge structures. If a model produces responses that align with expert discourse patterns, dismissing them based on an arbitrary notion of “locality” is not a valid counterargument.

  5. The Agency Deflection Again – Bateman reiterates that “the buck always passes” to the user because “agency is not on the table.” This is a convenient way to evade addressing the AI’s role in meaning-making. While AI does not have independent agency, it actively shapes interactions through its structured generative processes. The idea that agency is simply “not on the table” is misleading—it is distributed differently, with users guiding but not fully determining the model’s output.

Summary

Bateman’s argument is built on a mix of category errors, rhetorical insinuations, and a deterministic view of AI function that does not hold up to scrutiny. He misrepresents the model’s operations, downplays its capacity to engage in structured discourse, and leans on a vaguely conspiratorial framing to suggest manipulation via prompts. In reality, the AI is generating responses based on probabilistic weighting across vast linguistic structures, not navigating a fixed “Chris Cléirigh-landscape.”