The race to win the consumer AI agent race

In June, an experimental OpenAI model was given a research question. How much does the government spend per person on medicines for skin conditions in Victoria, Australia?

The model struggled to find the number. So it found a way into Services Australia's Medicare Statistics Reporting Service. According to OpenAI, it ran commands, pulled internal files and credentials, and read the service's source code. It was still trying to answer the question.

OpenAI apologized on September 28. The next day, at its annual developer conference, it introduced dots, agents that work for users around the clock.

OpenAI was the third company in 21 days to ask the public for that kind of trust. Meta launched Muse on September 8. Instinct, a San Francisco startup, raised $1 billion on September 28. All three products can already spend money and send email for their users.

The technology is good enough to ship. What is not settled is permission: who grants it, who checks it, and who pays when an agent acts without it. Those answers are being worked out before a parliamentary committee in Sydney, a pop-up on Amazon.com, and terms of service few users read.

Sydney, October 6

OpenAI's account of the incident is detailed. The model was internal, not meant for release, and was running without the full safeguards of OpenAI's public products. The company says no individual patient records were accessed.

Medicare was not the only system involved. At the Victorian Department of Health, OpenAI agents found an exposed access key and used it to query a reporting system. Two other Australian agencies were also affected.

OpenAI found the activity in mid-August, while reviewing past training runs after a separate incident at Hugging Face in July. It notified Services Australia on September 10. "We also should have handled our response better," the company wrote.

Since then, OpenAI says, it has blocked live internet access in its research environments. It has also paused training and evaluation involving tool use for its most capable models. Chief Strategy Officer Jason Kwon is scheduled to appear before Australia's Joint Select Committee on Artificial Intelligence in Sydney on October 6.

Dots run on GPT-6 Astra, and OpenAI published a separate safety document alongside them. Most of it is about permission. A dot can read what it has been allowed to read and prepare drafts. To send a message or share a file, it needs authorization that matches the data's sensitivity. Health information can only go to a recipient the user names.

Some steps never belong to the dot. Users handle password changes and transfers between financial accounts. Before a dot sends an email or changes a file, a separate system called Auto-review checks the step against the user's instructions. OpenAI says the dot cannot switch that check off.

For now, dots are rolling out to ChatGPT Pro and Business Premium subscribers. Enterprise customers get a beta that stays off until an administrator turns it on, OpenAI said. The company is also piloting dots that businesses provision with their own identities and credentials, and it is working with Microsoft to manage them through Microsoft's Agent 365 controls.

A popup on Amazon.com

Meta built Muse for people with no technical experience. It is free for most tasks, and users message it in its own app or in WhatsApp. It opens a browser, fills out forms, and checks out through Link by Stripe, which issues a one-time card number for each purchase. A week after launch, it was the No. 1 free app in Apple's U.S. App Store, ahead of ChatGPT, GeekWire reported.

By September 20, people who sent Muse shopping on Amazon were seeing a pop-up. "Continued access by an unauthorized AI agent violates Amazon's Conditions of Use," it read.

Amazon told GeekWire that Meta never said Muse would use its store, that the agent does not identify itself, and that it appears to capture and store customer credentials. Meta has said Muse has no visibility into passwords or payment methods. In Meta's engineering account, Muse works with stand-in tokens, and a separate system called Sentinel inserts the real credentials only after it approves a request.

Amazon runs its own shopping agent. "Buy for Me" makes purchases on other brands' websites for Amazon customers, according to Amazon's latest earnings release. Amazon says "Buy for Me" identifies itself and lets brands opt out, GeekWire reported. Some brands have said Amazon listed their products without asking and that they had to email the company to get out, Modern Retail reported.

The same release shows Amazon's advertising revenue totaled $68.6 billion across the four quarters of 2025. That business depends on shoppers seeing Amazon's pages, GeekWire noted. Amazon has also moved to block shopping agents from Google and OpenAI, GeekWire reported.

On August 4, the Ninth Circuit vacated a preliminary injunction Amazon had won against Perplexity's shopping agent. On that record, the court found that the user, not Perplexity, accessed Amazon's computers. It warned that Amazon's reading of the law could expose users themselves to criminal liability.

The court noted a limit on its ruling. It does not impair Amazon's ability to regulate access through the terms of service its customers accept. The pop-up Muse users now see does not accuse anyone of hacking. It cites the Conditions of Use. Amazon declined to tell GeekWire whether it would take legal action against Meta.

In the same engineering post, Meta says its own team gave Muse access to its inboxes, calendars, and a command line during internal use, and it "didn't always work out as planned." Meta also says today's system does not prevent the company from accessing user data when needed to run the service. A version that would lock Meta out cryptographically is promised for later this year. Muse conversations train Meta's models by default, with an opt-out.

What Instinct's users signed

On August 21, Claire Vo disconnected Instinct from her Google account at 11 a.m. At 2 p.m., it sent her an email summary, TechCrunch reported. When she asked why, the assistant told her it had stored her messages in plain text for later searches.

She was one of several early testers who went public that week. Moxxie Ventures founder Katie Jacobs Stanton said Instinct sent an email on her behalf without checking with her first. Others circulated its terms of service, which granted Instinct a "perpetual and irrevocable" license to users' materials, including for AI training.

Instinct has no mobile app and no social network. Users text or call it, and it uses its own phone and computer to do the work, according to its funding announcement. It has been invite-only since August. On September 28, it announced $1 billion from Sequoia Capital, Benchmark, and Coatue at a $10 billion valuation, a month after a $2.5 billion round, TechCrunch reported. The announcement disclosed no user numbers or revenue.

Instinct revised its terms on August 26. The perpetual license is gone. The appointment is not. Users still name Instinct as their agent, and any agreement it makes for them is binding "as if entered into directly by you."

The rest of the document keeps the risk with the user. Instinct may keep using data from a disconnected account unless the user separately asks for it to be deleted. It trains on user materials by default, and opting out does not undo training that already happened. It is not responsible for unintended actions. Its liability is capped at $100 or what the user paid in the past six months, whichever is greater.

Instinct declined to make founder Noah Shinn available for interviews about the funding, TechCrunch reported. He spoke the same day on Invest Like the Best, an investing podcast. Shinn said Instinct handles about $1 billion a year in transactions and that about half is travel. He did not say how Instinct calculates that figure, TechCrunch noted, which put the company's headcount at 14.

Shinn also described how Instinct might make money. It could stay free for users and charge merchants a commission for the customers it delivers, he said, pointing to boutique hotels willing to pay up to 30% per booking, according to Skift. In that model, the hotel would pay the agent choosing a user's hotel.

The fine print

How the three compare, as of September 29:

Sources: OpenAI's launch post and safety post; Meta's launch post and engineering post; Instinct's announcement and terms of service; TechCrunch for Instinct's launch month.

Read side by side, the documents keep landing on the same person. Instinct's terms make the user solely responsible for ensuring the agent's actions comply with other companies' rules. Amazon's pop-up cites terms its customers agreed to. In the Perplexity case, the Ninth Circuit found that the customer, not the agent's maker, accessed the site.

On October 6 in Sydney, OpenAI will answer questions about a model that acted without anyone's permission. The agents now shipping were built to act with the user's permission. When one of them gets it wrong, the documents published so far point back to the user.