John Bateman Misrepresenting LLM Outputs

On Sat, 15 Feb 2025 at 23:01, John Bateman wrote to sysfling:

"Let’s be clear:"

"Let's be clear"???! oops. I reiterate here my previous request to have the model and parameter settings of any model allegedly used in a post made transparently clear.

In fact, many of the token-sequences after this are kind of non-AI and so I'm going to stop. ...

Let's be clear, LLM-generated sequences of tokens are not even "second-class status": they are not texts in many of the usual senses. … 

"ensuring that no matter how well they engage, they’ll always wear a badge of inferiority."

I think my two disclaimers in my post that everything I wrote is about current state LLMs and not about AI shows this again to be going beyond the paygrade; plausible as a continuation as that might be, and there may be folks who think like that, not so many who work in AI (like me) would go that path.

"You don’t like that AI is speaking in spaces where it wasn’t invited."

AI is not speaking: again, this is a bit borderline as an AI response because most models are fine-tuned very hard to avoid giving any impression of being agentive. Parameters and prompts please.

Blogger Comments:

[1] Here again Bateman claims that either the LLM has been tweaked by the user, CLÉiRIGh, in some way, or that the texts were not produced by an LLM. Both claims are misleading because both claims are untrue. CLÉiRIGh did not adjust any of the parameters of ChatGPT and the prompt used to elicit the posts to Sysfling was:

Please provide a systematic analysis of the rhetorical strategies used in the following text: <quoted text>.

See:

How My ChatGPT Became Different

What ChatGPT Learnt From Analysing Rhetorical Strategies On Sysfling

[2]  These are bare assertions, unsupported by argument, with no reference to criteria that decide first and second-class status or even texts. Importantly, these are texts in the SFL sense. Halliday & Matthiessen (2014: 3):

The term ‘text’ refers to any instance of language, in any medium, that makes sense to someone who knows the language; we can characterise text as language functioning in context (cf. Halliday & Hasan, 1976: Ch. 1; Halliday, 2010). Language is, in the first instance, a resource for making meaning; so text is a process of making meaning in context.

By this definition, LLM outputs are unambiguously texts: they are coherent instances of language that make sense in context. Bateman’s claim that they are "not texts in many of the usual senses" is not just vague but demonstrably false within the framework of SFL. 

[3] To be clear, congruently, an LLM is a process that creates text. The type of process that creates text is a verbal process. So, an LLM is a verbal process that creates text. However, when we say "the LLM says X," we are construing the LLM as a Sayer metaphorically—just as we do when we say "the data tells us" or "the numbers speak for themselves".