Coding a Co-Writer: Creating and Collaborating with a Bespoke Language Model

In the last decade, developments in the fields of machine learning and natural language processing have resulted in the production of increasingly complex text-generation programs like OpenAI’s ChatGPT and Anthropic’s Claude. These programs, often referred to as large language models (LLMs), are driven by complex layers of algorithms that are trained upon enormous sets of textual data to create statistical models that can output text which closely resembles the human writing they have been trained upon. While creative writers including Kate Mildenhall and Sean Michaels have experimented with the use of LLMs in their work, there is an increasing awareness of the ethical issues posed by LLMs, many of which are trained on copyrighted work without consent or compensation. As the transformer models that LLMs are built upon necessarily require ever-increasing amounts of text for marginal improvements, the ethical problems associated with generative AI language models are unlikely to be resolved in the near future. Yet LLMs are not the only way to experiment with machine learning and writing; more basic language models provide the opportunity to explore the relationship between language and narrative and to trouble and interrogate creative texts. This paper will outline and reflect upon the process of coding a bespoke neural network and training it on a small dataset consisting only of drafts of a novel manuscript in an iterative, cyclical process in which the output of the language model informs the creative work, and the creative work shapes the development of the language model.

N/A

30th Annual Australasian Association of Writing Programs Conference: Movement & Stasis

N/A

N/A

N/A

N/A

2