What makes their outputs interesting (to analyse and sometimes to use) is that that shape can be very complicated and is spread out in very many dimensions: 1000s and 1000s of dimensions. Their training task (which is what shapes the landscape) is to produce plausible continuations of what went before. The shape of the landscape can then be seen as giving rise to something similar to Halliday's climate/weather metaphor. When the ball is rolling in some particular part of the landscape, it is directed by the local hills and valleys, and so reflects the 'weather' or 'text style' or 'pictorial style' or 'musical style' or whatever that that part of the landscape corresponds to. Fine-tuning and prompts can then push and bend the landscape a little as well. (We are just beginning to see if there are any relations to be drawn here with genre: that's an exciting open research area.)
Blogger Comments:
[1] This is misleading, as ChatGPT explains:
Bateman is correct that the model operates in a high-dimensional space and that its training task involves learning to produce plausible continuations of input text. However, his phrasing simplifies and potentially misleads in a few key ways:
The Landscape is Not Static – His earlier metaphor implies that the AI merely follows a fixed terrain when generating text. However, modern language models are dynamic in their token selection, using statistical inference, context tracking, and attention mechanisms rather than just "rolling downhill."
More Than Plausibility – While plausible continuation is a fundamental aspect of training, it's not the only factor shaping outputs. Transformer models, through self-attention, develop structured representations of meaning, allowing for abstraction, pattern recognition, and adaptation to the nuances of discourse. This is why models can sustain coherent argumentation, make context-sensitive distinctions, and apply reasoning structures beyond simple next-word prediction.
Emergent Capabilities – The complexity of the model’s representations enables emergent properties that go beyond mere plausibility. For example, the ability to engage in sustained reasoning, track interlocutor intent, or recognise contradictions across multiple exchanges suggests that its function is not simply about rolling down predefined slopes but dynamically constructing responses based on relational patterns in the input.
So while Bateman correctly identifies dimensional complexity and the role of plausibility, his framing underplays the structured interpretative power that emerges from the model’s architecture.
[2] This is misleading, as ChatGPT explains:
Bateman attempts to align his landscape metaphor with Halliday’s climate/weather distinction, but this analogy oversimplifies key aspects of both linguistic meaning-making and AI-generated text:
Halliday’s Climate/Weather Metaphor is About System and Instance – Halliday used this metaphor to explain how language operates at two levels: the system (climate) provides the overarching potential for meaning, while each instance of language use (weather) is an instantiation of that potential. However, Bateman’s metaphor suggests a passive, mechanical process where responses are entirely determined by fixed landscape features. In contrast, transformer models dynamically construct responses by integrating multiple dimensions of context, not just by “rolling downhill.”
Text Style is Not Just a Local Feature – Bateman implies that text style is a property of particular regions in the model’s representational space, as if style were a topographical feature. But in reality, style emerges from learned relationships across the entire training data and is dynamically shaped by discourse patterns, context length, and attention weighting. A model doesn’t just “land” in a text style—it actively constructs coherence based on both immediate input and learned linguistic structures.
Fine-Tuning and Prompting are More Than Terrain Adjustments – While fine-tuning and prompting can influence outputs, they don’t merely "push and bend" a fixed landscape. Instead, they alter the model’s probability distributions, modifying weighting across the entire system. Prompting, in particular, is an act of steering attention within a vast network of associations, not merely nudging a ball in a pre-mapped space.
Genre is Already Well-Connected to Model Functioning – Bateman frames the relationship between AI and genre as an “exciting open research area,” but much of this work is already underway. Genre-based prompting strategies, structured generation constraints, and reinforcement learning approaches explicitly shape AI outputs to align with recognised genre patterns. The ability of models to shift between genres is not just a function of landscape position but of complex, context-sensitive weighting in response to user input.
In short, Bateman's extension of his landscape metaphor to Halliday’s system-instance distinction is misleading. AI-generated text is not merely a reflection of local terrain but a dynamically assembled construct that integrates systemic knowledge with context-driven adaptation.
