An LLM is a computer program. You put text in and get text out. That’s it. The program has no interactions with the real world. It is not going to “wake up” and do things. It can’t do anything. In 2023, a human would get the text through a chat interface and cut/paste the code it generates, or cut/paste the commands to the command line, then cut/paste the output back in to refine.
What’s a harness?
A harness is what makes the LLM agentic. This is, basically, a “while loop”, probably in python. The while look takes your input, then calls the llm, then parses the output, then (maybe) asks if you approve the suggested actions, then takes the suggested actions. It can parse the output to find what command to execute, then call os.system() or subprocess.run() – or take the text and write it to disk. The loop can then take the output and keep calling the LLM until it hits a stopping point, then ask for your input again. There are a few other hooks, to, say, run os-level programs, but for the most part, that’s it.
What about the hacks?
That’s the basic architecture. It’s not magic. When we say the bots have escaped a sandbox, we just mean they found a way to access the internet. They are still running on a machine, or AWS Cluster, and you can “just” unplug them. That’s it. Programming agents to prevent them from running LLM-generated code to, say, replicate them system on the internet, is, while not trivial, not particularly difficult, either. Programming the LLM to not try to replicate itself is also not particularly difficult.
In the cases when we have seen outlandish claims from the vendors, there has invariably been a full context in which they were prodded to do exactly what they did with unreasonable, reduced, or negligent safety backdrops.
So … what?
I am convinced if we can keep the conversation focused on what these actually are and what is actually happen, we can avoid some of the mistakes that bozos, fools and scoundrels seem to want us to make as a society where we conflate LLMs and Agents with something from a Terminator movie.