Large Language Models
Large Language Models (LLMs), like ChatGPT, leverage neural networks and machine learning to generate human-like text.
Evolving from the 1960s chatbot ELIZA, LLMs excel in reasoning and prediction, powering automated home assistants and computer vision.
Following the advent of transformer models in 2017, which employ an attention mechanism that ‘consists of several attention layers running in parallel’ (Vaswani et al. 2017: 4), generative AI – including LLMs – can now produce text, images, other media in response to a prompt.
Focussing on ‘a word in the context of a sequence’, LLMs generate ‘probability for the importance of a word relative to other words in the phrase or sentence’ (Hayles, 2022: 639), essentially seeking to compute ‘human context, meanings, patterns of behaviour and possible futures’ (Amoore, 2020: 89).
For the computer scientist and literary theorist N. Katherine Hayles, generative AI acquires ‘a kind of intuitive knowledge’ derived from ‘the intricate and extensive connections that it builds up from the references it makes from its training dataset’ (Hayles, 2022: 648–649).
Although the nature of the external prompt will significantly shape an LLM’s output, the systems’ algorithmic “hunch” about how to respond will manifest in a novel or ‘unrepeatable’ form, which is dependent on ‘how the neurons are weighted’ among other factors (Hayles, 2022: 645).
While the prompt constitutes a provocation or affective relation to the machine, the LLM ‘exercises considerable creativity in fashioning responses that can be remarkably complex in style and conceptual structure’. ChatGPT-3 has, for instance, been observed to “flip the script” when it ‘senses a note of antagonism in the prompt’ (2022: 659, 658).