On Generative AI
40 The Degenerative, part 2
The dangers that are particular to Generative AI include:
Inaccuracies and hallucinations: generative models are a marvel in churning out coherent, fluent, human-like language. In all that glibness are hidden factual errors, limited truths, fabricated references and pure fiction – referred to as “hallucinations”1,2. At the bottom of the ChatGPT interface, underlining all conversations, is the notice that ‘ChatGPT may produce inaccurate information about people, places, or facts1. The accuracy of ChatGPT could be around 60% or worse, depending on the topic2,3.
To make things worse, ChatGPT has a tendency to present truths without evidence or qualification. When asked specifically for references, it can conjure sources that do not exist or support no such truth as presented in the text4,2. Yet, many users tend to use it like an “internet search engine, reference librarian, or even Wikipedia”5. When a teacher or student uses it to get information on which they have no prior knowledge, they run the risk of learning the wrong thing or presenting false knowledge to others1,5.
The success of today’s LLMs lies in the sheer number of parameters and amount of training data, which they use to model how words are stitched together in human communication. Teachers and students should always keep in mind that the text generated by conversational models is not connected to understanding of this text by those models, or even a notion of reality1. While they can manipulate linguistic form with varying degrees of success, they don’t have access to the meaning behind this form6. “Human-style thought is based on possible explanations and error correction, a process that gradually limits what possibilities can be rationally considered… Whereas humans are limited in the kinds of explanations we can rationally conjecture, machine learning systems can learn both that the earth is flat and that the earth is round”7 .
Shifting or worsening power and control: generative AI is dependent on huge amounts of data, computing power and advanced computing methods. Only a handful of companies, countries and languages have access to all of these. Yet, as more people adopt these technologies, much of humanity is forced to toe their line, and thus is alienated and forced to lose their expressive power1.
While the creators keep the power, they outsource the responsibility.
Copyright and intellectual property infringement: Much of the technological know-how of generative systems is guarded behind corporate walls. Yet, the data is taken from the general public1. Is it ok to take pictures that were made public on some platform and use them without the knowledge or consent of the subject? What if someone’s face is used for racist propaganda, for example8? Is the only way to block Gen AI to make content private?
Beyond public data, language models can take content behind paywalls and summarise them for the user. Image models have been known to