ChatGPT and other LLMs produce bull feces, not hallucinations


In communication around popular interfaces such as LLM and ChatGPT the term 'hallucinations' is often used to refer to the false statements made in the output of these models. This leads to the conclusion that there is some coherence and attempt by the LLM to know the truth as well as suffering moments of (mild) insanity. Thus the LLM is effectively treated like a small child or a person suffering from disorders such as Alzheimer's, giving him agency in the process. This is utter nonsense and so obviously wrong, it is the subject of a treatise [Michael Townsen Hicks] and published by colleagues, Ethics and Information Technology,

Much of the difference lies between lies and bullshit, as has been so clearly described. [Harry G. Frankfurt]'s 1986 essay and 2005 book On the nonsenseLying is intended to deceive and conceal the truth, while bullshit is done without any regard for or concern for the truth. Bullshit is done simply to make the immediate situation better, which is reminiscent of the worst aspect of sound bite culture.

When we consider the way LLM works, in which the input query is used to provide a probability fit to the weighted nodes that make up its vector space, we can see that the generated output is effectively that of an oversized word prediction algorithm. This eliminates any possibility of intelligence and thus prevents cognitive awareness of the ‘truth’. This means that even if there is no intention behind LLM, it is still nonsense, albeit of the soft (unintentional) kind. However, when taking into account the agency and intentions of those who create the LLM, those who train it, and those who create the interface (such as ChatGPT), we enter hard, intentional nonsense territory.

Incidentally, this is the nonsense that has already partially killed the LLM, with Retrieval Augmented Generation (RAG) turning word prediction algorithms into fancy search machines. After all, venture capitalists can only tolerate so much nonsense.

Leave a Comment

“The Untold Story: Yung Miami’s Response to Jimmy Butler’s Advances During an NBA Playoff Game” “Unveiling the Secrets: 15 Astonishing Facts About the PGA Championship”