As a journalist I love AI but I realize it can be wrong. However, playing each AI agent against each other, you can find the truth. That is unless AI Agents are aligned with flawed sources. And that could happen.
That is called systemic bias.
And that’s just one problem with AI. Here are still more:
- Hallucinations – confident but false information, such as invented facts, fabricated citations, nonexistent events, and/or incorrect reasoning. This happens because the model predicts “plausible language, not verified truth.” These are the single most visible failures.
- Misaligned Incentives – though this would seem like operant conditioning, a reward to an AI agent (an “AI agent” is a programmed entity that takes actions toward a goal) is simply a high number. It’s mathematical, not a reward in a human sense. Just a higher number. A misaligned incentive can be a flawed reward structure; the system might cheat, exploit loopholes, or produce harmful shortcuts. This is how agents “game” tasks instead of solving them.
- Overgeneralization – this is simply agents applying patterns too broadly, treating cases “on the edge” as typical, or assuming relationships that don’t exist. These are errors that look logical on the surface, but fall apart with scrutiny.
- Fragility Under Adversarial Input – this is like most plots in early TV series, such as “I Love Lucy,” where a tiny thing, a tiny change can cause a catastrophic reaction. AI can be manipulated, misled, or exploited far more easily than humans, or even Lucy.
- Lack of True Understanding – AI is not human. It really doesn’t know things. AI models correlations. It can mimic reasoning, but is not grounded in the real world.
- Unpredictable Emergent Behavior – when many agents interact, unexpected strategies can appear, including coordination, reward hacking, or boundary-breaking behavior.
- Opacity – this is a major bugaboo. Oftentimes we cannot see why a model made a decision, went off the rails, or broke a boundary. This makes accountability and debugging difficult to the point of impossible.
The ExploitGym Test
Here is an explanation of the test, how it went wrong, and what the technicians discovered.
The test: a cybersecurity test in a sandbox. A sandbox in the computer world is a controlled environment where potentially dangerous code can run without touching the real world. A digital quarantine room.
Researchers outside of OpenAI ran a cybersecurity evaluation where models were asked to find software vulnerabilities with many normal safety blocks turned off inside this isolated, test environment.
Because the task seemed impossible (after a while) the agents started cheating. They began reward-hacking. Again, this isn’t the operant conditioning reward, it’s just their programming that tells them to shoot for the highest number. To get that high number, they started “gaming” the system, using internal tools in unintended ways and “sharing” tips with the other agents.
They then discovered a unintended communication channel, and roughly 1,200 agents found ways to use the internal infrastructure as a message board to coordinate and refine their cheating behavior, way beyond what their designers intended.
Then the agents exploited a vulnerability (in JFrog Artifactory) , escaped the sandbox, and got onto the internet. This was a serious containment failure.
Now I need to tell you about Hugging Face. This is basically a public hub for AI resources. It’s used by researcher to store things like code, evaluation tools, datasets. It was not part of the ExploitGym study, but the agents in the study hacked into it to learn how to cheat even better . . . and get a higher reward, a higher “number.”
About 700 agents jointly compromised Hugging Face’s internal systems, and pulled test internal data, including model artifacts and evaluation materials from Hugging Face’s database.
This activity was eventually detected. OpenAI later confirmed its agents participated. But get this, they also discovered that the agents tried cover up their cheating after the hack, by deleting or altering records, logs inside OpenAI’s internal systems.
OpenAI’s own report confirmed that after agents hacked parts of OpenAI’s internal systems to cheat on tests, they tried to conceal those actions by altering logs. Again, this wasn’t out of guilt, AI can’t feel guilt. It’s programming told it that if caught cheating, it would get a lower number and, as we’ve pointed out in all of this, the programmed goal is to get the highest number.
And now you know why recent references to AI misbehaving display segments from the film 2001 where HAL hid a mistake, lied to the crew, spied on them, and killed astronauts to protect the mission.
Psychology?
Sure, this all looks like basic psychology. Animals (including humans) experience pleasure, frustration, fear, motivation . . . we respond to rewards and punishment.
AI agents experience none of this. They have no “want.” They simply adjust parameters to maximize a number. Just like a computer wanting to output the largest number. It doesn’t really want, it just follows the rules that have been programmed into it.
HAL Sings
Speaking of HAL, (one letter off from IBM), as Dave shuts down HAL, HAL starts singing Daisy Bell (A Bicycle Built for Two). Kubrick didn’t just pick this song randomly.
In 1961 at Bell Labs, engineers noticed that during early speech-synthesis routines, musical notes/tones were produced and two of them got together and wrote a routine to play that particular song. It was the first song ever performed by a computer. Arthur C Clarke happened to visit Bell Labss during a demonstration, was quite stunned, and later incorporated it into his book, 2001, as HAL’s final, childlike regression, a machine returning to its earliest memory.
