ErnestEndeavor said:
Going off the deep end is assigning human characteristics like "going rogue" to glorified cloud software.
You're now stuck on the semantics of a word, get over it
Quote:
"Agents" are scripted computer programs calling on systems for instructions that are inherently unpredictable. LLMs are stochastic. They are not normative. When they produce weird outputs or don't follow the prompt instructions they aren't "going rogue" it's just what they do. It is inherent in the design.
What a weird thing to say. "Agents" are not scripted computer programs at all, they are guidance on how to act upon prompts and focused context to work in. Of course LLMs are stochastic, that's the entire point, but you're trying to gloss over the difference between "producing weird outputs" and flat out breaking the hard rules that are in place.
Quote:
The HuggingFace hack was 100% OpenAI and the researchers' fault, was predictable and avoidable.
Which is why they had taken such measures to place it in an isolated network to run.
Quote:
They designed an experiment in which they trained an internal model, tuned it for hacking, gave agents access to the model, removed hard-coded guardrails, and walked away. That's their fault. A model producing instructions to skirt the perimeters of a test to attack it sideways to produce a result that looks like it passed the test is a known thing.
This is a gross misrepresentation of reality. They removed some very focused guardrails to test its capabilities, but left many of the guardrails in place, and the AI ignored it. I would say it went rogue, but you seem to be hyper-focused on that word, so I'll just say, "the agents acknowledged its hard guardrails and explicit instructions, and ignored them". You keep making the generic statement that is semantically true, but incredibly disingenuous and deliberately obtuse at the same time.
Quote:
They should have been supervising the experiment and they didn't. Soon after the agents started receiving erroneous instructions to hack an outside entity they should have cut off the experiment but no one was there watching.
Again, it was heavily supervised and watched, they just didn't watch jfrog closely enough.
Quote:
OpenAI did something inherently stupid and we're lucky HuggingFace was the only victim, at least that we know of.
Overstating it. OpenAI created a very controlled and isolated environment, and missed one thing that the agents found and exploited. That was a mistake.
Quote:
Assigning blame to the AI system is just a copout.
Blame? You think people are trying to assign blame to the AI agent? It's an analysis of what happened, not a blame game. Focus on facts instead of injecting your personal feelings into it.
So far, you've incorrectly stated they trained the models to work together. You've incorrectly stated that they removed all the guardrails. You've incorrectly stated that they just let it go without watching it. You're now digging in on the semantics of a word.
This is my last post. It's very clear that you still haven't done any real inquiry into what actually happened, you've turned instead to the doom and gloom mindset and forgone any attempt at true understanding of what went wrong and why.
