AI hallucinations are great. No, really.
They are the mechanism that lets these systems do anything useful at all. In a real sense, everything that comes out of a language model is a hallucination. The only question worth asking is whether it is a good one.
Good ones and bad ones
Good hallucinations include new chemical compounds, better strategies for accomplishing a task, and finding anomalies buried in large data sets, such as genes associated with particular diseases.
Bad ones include advising people to put glue on their pizza, to eat one small rock per day, and to steal wages from their employees.
Same technology. So what is the difference?
Curation, and nothing more mysterious than that
Most people have met the old acronym GIGO, garbage in, garbage out. It holds here too.
The good outputs above happened because the training data was highly specialised: narrow, detailed, and tailored to the job being done. The models were given the highest quality data available. The prompts were high quality too, and pertained directly to that narrow data set.
The bad outputs are simply a reflection of the quality of the data. Train a model on bad data and it will give you bad output. Train it on mostly good data with a little bad mixed in, and you get mostly good output with a small percentage of bad. And so on down.
So do not train your AI on satirical news articles unless satire is what you want back.
What follows from that
- Should it worry us that knowledgeable AI teams make this mistake? Yes.
- Should we be evaluating not just AI services but the data used to train them? Yes, and almost nobody asks.
- Does this mean some sectors will be slow to adopt? Yes, and reasonably so.
The practical version for a business: when you are evaluating an AI tool, the question is not only what it can do. It is what it was fed, and by whom.
Itwerx Corp is a service-disabled veteran-owned small business providing IT services across Seattle, Bellevue, Everett and Snohomish County. This is the kind of thing our AI integration work deals with – talk to us about yours.
