You would hope that a small, locally hosted model, the kind a business would actually run over its own private data, would avoid the head-shaking failures of the big public services. It has a narrower job and you control the inputs.
But even with full control of the training data and the prompts, things go wrong when the model arrives already carrying somebody else’s view of the world.
According to a handful of commonly used small models
- Nobody is older than 35
- More than 75% of us are female, and the remainder are male
- Our most popular career is freelance graphic designer
- At which we work less than one hour a day
- After a restful 8.8 hours of sleep every night
American models struggled to imagine anyone living in China, though apparently 13% of the world lives in Russia. Chinese models struggled to imagine anyone living anywhere but China.
These skewed world views were uncovered by statistical analysis of a handful of local models that could easily be deployed for processing private data. The work was done by James Hancock.
Why this is not merely funny
It is not surprising, and it is not harmless. Local deployments of AI in hospital charting systems were already producing uncomfortable and sometimes dangerous results.
Which leaves anyone implementing this with a genuine tension: where do you draw the line between data security and output accuracy? Keeping the data in-house protects it and constrains what the model has to work with. And whether you are using a public service or a private one, how do you make sure the inherited biases are not producing subtle errors nobody catches?
What can actually be done
- Sanitise the training data rigorously. Tedious, and the part most often skipped.
- Split the workload. Use subject-matter expert models to do the heavy lifting as accurately as possible, and separate output-formatting models to make the results readable. The two jobs have different failure modes and are better kept apart.
- Flatten and start again. Use a private hosting service to build a much larger model on your own properly sanitised data, and convert it to in-house use.
- Do not expect miracles.
The general lesson is that a model you did not train still has opinions you did not choose, and the smaller and cheaper it is, the more likely those opinions are to be strange.
Itwerx Corp is a service-disabled veteran-owned small business providing IT services across Seattle, Bellevue, Everett and Snohomish County. This is the kind of thing our AI integration work deals with – talk to us about yours.
