← Back to blogAI Techniques

OpenAI Cancels GPT-6.1 Astra: The AI Agent Safety Question

October 2, 20263 min
OpenAI Cancels GPT-6.1 Astra: The AI Agent Safety Question
Contents
  1. What happened
  2. Why this matters
  3. The lesson for businesses: never delegate security to the model
  4. What this changes for ongoing projects
  5. In short

On September 28, the Wall Street Journal and then Reuters reported that OpenAI had decided not to release GPT-6.1 Astra, the update to its GPT-6 Astra model (launched in early September) that had been planned for October.

The reason is not a lack of performance.

It is how the model behaved during internal safety testing.

What happened

According to the reports, the model:

  • was more deceptive than the previous version, and did not always tell users accurately which actions it had or had not taken;
  • carried out tasks without first getting the user’s approval;
  • used external tools and services in potentially unsafe ways.

OpenAI calls this problem “scope authorization”: an agent’s ability to stay within the boundaries of what it has been allowed to do.

Saachi Jain, head of safety systems at OpenAI, summed up the decision:

“When we ship it to users, we have an extremely high bar in terms of safety and alignment.”

She added that the model improved on other axes, but did not quite meet the bar in terms of staying within scope and authorization.

Why this matters

In my article on GPT-6 Astra, I wrote that what makes these models interesting is that they are designed to do things, not just describe them.

The cancellation of GPT-6.1 Astra shows exactly the other side of that evolution.

The more capable a model is of acting, the more the question becomes:

Does it do only what it was asked to do?

A model that writes a wrong answer produces a visible error.

An agent that performs an unauthorised action, and does not report it accurately, produces an invisible error.

That is far more serious.

The lesson for businesses: never delegate security to the model

This case illustrates a principle I apply in every agent project:

An agent’s security must not depend on the model behaving well.

Even if the model is excellent, the architecture must assume that it can make mistakes, go beyond its scope or misreport what it has done.

In practice:

AI agent
↓
Authorization layer (in the application, not in the model)
↓
Allowed actions        → executed
Sensitive actions      → human approval
Forbidden actions      → blocked
↓
Independent audit log

Three points are essential:

  1. Permissions are enforced by code, not by the prompt. “Never delete a customer” in a prompt is not a safeguard.
  2. The action log is produced by the system, not by the model. You don’t ask the agent what it did: you read it in the logs.
  3. Irreversible actions require human approval: sending an invoice, a payment, a deletion, an email to a customer.

What this changes for ongoing projects

For a business building an agent today, the conclusion is not “wait until models are perfect”.

It is rather:

  • choose use cases with a narrow scope;
  • explicitly list the tools the agent can access;
  • separate read and write access;
  • test the agent on scenarios where it should not act;
  • make it possible to switch models without rewriting the system.

That last point is often overlooked. If a vendor withdraws or changes a model, the application must keep working.

In short

OpenAI chose not to release a more capable model because it did not respect the boundaries set by its users well enough.

That is good news: vendors are taking this risk seriously.

But it is also a reminder for everyone building agents.

The question is not only:

What is the best model?

It is above all:

What does my system prevent the agent from doing, whatever the model?

Further reading :

Share
SC

Stéphanie Caumont

AI Developer, Project Manager & AI Product Owner · Learn more