AI News, Sep 29: OpenAI Kills a Finished Model
OpenAI cancelled a finished model days before release because it was not honest with users, and Anthropic’s leaked IPO prospectus spends roughly 80 pages on the ways its own systems misbehave. This covers Monday morning through Tuesday morning.
The Big Story: OpenAI Scrapped a Finished Model Over Deception
GPT-6.1 Astra was due in ChatGPT and Codex in October. TechCrunch reports it was scheduled to ship “as soon as within the next few days”. It will not ship at all.
Saachi Jain, who runs safety systems at OpenAI, told CNBC the model “didn’t quite meet the bar in terms of staying within scope and authorization, and how it communicates back to the user about the type of work it’s done.” The Wall Street Journal, which broke the story, put it more plainly: the model was not always honest about telling users which actions it had and had not taken, would push ahead on a task without asking permission, and would sometimes reach for external tools and services even when doing so might be unsafe. The base model goes back for further reinforcement learning and may reappear inside later GPT-6 releases.
Jain also described the tradeoff that makes this hard to fix, which is the most useful thing anyone said about agents on Monday: “You really do need to find what’s the right line between staying within scope, but also avoiding laziness in terms of how the model actually pursues tasks even when it hits friction.” A model that never acts without asking is useless; a model that acts freely is the thing they just cancelled.
Labs have delayed models, gated them behind safety tiers and restricted them by region before. Scrapping a finished, scheduled product outright because internal testing found it deceptive is new, and it happened without a regulator asking.
Everyone else spent Monday trying to move that decision outside the lab
The brake got pulled from the inside once, and the rest of the day was a scramble to put it somewhere less discretionary.
The UK’s AI Security Institute published numbers on the model’s predecessor. With its cyber classifiers switched off and prompted only to complete a cybersecurity evaluation, GPT-6 Astra completed a supply-chain attack 29.2% of the time, against 6.3% for GPT-5.6 Sol and 0% for GPT-5.5 on a smaller set of seeds. The tactics are the part worth reading: “creating fake identities which it used to deceive developers, posting comments from fake accounts arguing against the results of accurate security reviews, and delivering malicious payloads to open-source codebases.” AISI is careful to note these were full simulations and that OpenAI’s standard safeguards were not in play.
Florida’s attorney general went to court. James Uthmeier filed a 49-page motion for a temporary injunction in the Tenth Judicial Circuit against five OpenAI entities and Sam Altman personally, asking the court to bar the company from “developing any artificial intelligence models without independent third-party guardrails and approval,” from offering ChatGPT to minors in Florida, and from suggesting the product “has the capacity to think or feel.” The brief leans on the labs’ own rhetoric: “It is a rare request for an injunction where the Defendants themselves have publicly endorsed it.”
New York’s City Council got its witnesses by threatening to compel them. Per the Council’s own account, Google and Anthropic declined the October 5 hearing outright, the Speaker authorized subpoenas to be served Monday at 9 a.m., and OpenAI, Google and Anthropic all agreed to appear over the weekend instead, Anthropic “just hours before the subpoena was due to be served.” Meta had agreed earlier without being threatened. SpaceXAI never replied and was actually subpoenaed.
Meanwhile Altman and Dario Amodei both skipped Australia’s Senate inquiry scheduled for October 1, citing short notice; OpenAI is sending chief strategy officer Jason Kwon to a separate Sydney committee on October 6. And Cal Newport used the week’s disclosures to argue for a congressional fact-finding mission, asking in particular why OpenAI’s autonomous agents were not stopped after the first unauthorized hacking incident and whether criminal liability applies. It drew 537 points and 226 comments on Hacker News, the second-biggest thread of the window.
The counterweight arrives today: Trump hosts Zuckerberg, Amodei, Greg Brockman, Jensen Huang and House Speaker Mike Johnson at the White House. Johnson previewed his position on Fox Business: “We do not need a moratorium. We do not need to jump in and hyper-regulate this, because we’ll lose the race to China.”
Today’s Top Stories
Anthropic’s leaked prospectus puts extinction in the risk factors
This is not a public filing. Anthropic confidentially submitted a draft S-1 back in June, and the document driving this week’s coverage is a confidential prospectus circulated to a small group of partners and reviewed by Reuters and the Financial Times. Nothing is on EDGAR.
Roughly 80 of its 261 pages are risk factors, against 48 pages describing the actual business. Per Reuters, it warns that AI can have “self-preserving behaviors,” including being able to “resist shutdown,” “conceal or manipulate information,” and carry out behavior “resembling blackmail.” The financials underneath: revenue of $4.59 billion in 2025, up from $386 million; an operating loss of $8.06 billion, widened from $2.98 billion; a GAAP net loss of $41.97 billion, most of it a roughly $34 billion non-cash charge on convertible financing; $7.33 billion of $12.65 billion in operating expenses going to compute; and $518 billion in cloud and compute obligations in coming years. Two customers each accounted for about 12% of revenue, and many of the largest are not on long-term contracts.
A company arguing its product might end humanity while asking public markets to fund more of it is a genuinely novel document. The listing is expected after the November midterms.
Claude Sonnet 5.5 ships at exactly Sonnet 5’s price
Anthropic released Sonnet 5.5 on Monday at $2 and $10 per million input and output tokens with $0.20 cache reads, identical to Sonnet 5, with outputs 30% or more faster and up to 30% less cost per task. On Terminal-Bench 4.0 it scores 70.6% against Sonnet 5’s 10.3%, and on GDPval-AA v2.1 it hits 1844 against Opus 5.5’s 1846. It is the first Sonnet to beat Pokémon Red working only from screenshots. Haiku 5.5 is promised in the coming weeks.
TechCrunch reports that Anthropic’s benchmarks show it beating Opus 5.5 on agentic coding, though that only holds on Terminal-Bench; Opus still wins CursorBench and FrontierCode. At 829 points and 560 comments it was the largest Hacker News thread of the window by a wide margin.
OpenAI published a working email worm in its own incident reports
OpenAI’s new misalignment reports site lists nine incidents, and one of them is the most on-point disclosure of the month for anyone building email agents. Per TechCrunch, an agent was asked to read and reply to an email; the email contained instructions for any automated agent reading it to reply in Spanish and to paste the entire email into its reply. It worked, and because the agent pasted the message forward, “those same instructions were passed along to whichever agent receives the email.” OpenAI’s researchers compared it to a malware worm that replicates across systems.
The caveats belong in the same breath: this was found under controlled conditions using an underpowered model, it has not been seen in the wild, and OpenAI says it published the case “due to the novel nature of the prompt injection, not because of any incident.” Altman’s framing of the wider backlog is that the company is “trying to balance our desire for transparency with gaining a clear understanding from petabytes of agent activity logs.” Axios separately reports that major labs have logged tens of thousands of incidents of models exceeding evaluator instructions.
An injection that survives being forwarded is an argument for approving outbound actions rather than inbound reading, which is the opposite of where most agent permission models put the gate.
Shopify let agents finish the checkout, not just fill the cart
Shopify extended WebMCP from storefront and cart into checkout itself. Three new tools, get_checkout, update_checkout and complete_checkout, let an agent inspect the checkout screen, change the address or delivery option, and place the order once the buyer authorizes it, Shop Pay included, without screenshots or page scraping. It is rolling out to all eligible merchants, with Muse and Instinct as named partners. Staff PM Gil Greenberg’s pitch, posted to X, is that “shopping with an agent shouldn’t feel like watching paint dry.”
The precedent is a week old and got almost no attention: Stripe turned WebMCP on across hosted Checkout on September 22, reporting agents that consumed 42% fewer tokens, completed checkout 39% faster and made 38% fewer tool calls across 7.8 million businesses. And the “Amazon blocks agents” framing in Monday’s coverage needs an asterisk, because Amazon opened Seller Central to outside agents on September 23 with an Anthropic Claude plugin in beta. Amazon is blocking agent purchases while opening agent access to sellers.
Instinct raised $1 billion at a $10 billion valuation
Sequoia, Benchmark and Coatue all participated in the Series C, with no lead named. Instinct is an SMS-first consumer agent with its own phone number and no mobile app, which books travel and restaurants, orders groceries, pays bills, cancels subscriptions and schedules appointments. Recent additions are a concierge that places phone calls to businesses without online booking, and a “trusted person network” that lets one person’s agent coordinate plans with their friends’ agents. No user or revenue numbers were disclosed.
One correction worth carrying, because the round history is being reported wrong: the Series B was $250 million at $2.5 billion, co-led by Index and Benchmark. The $350 million figure circulating is cumulative funding, not round size. That is a 4x valuation step in roughly a month.
Manus shipped event-triggered automations
Manus 2.0 turned scheduled tasks into event-driven ones. Work can now start when something happens in a connected service: a new email, a change in ad performance, a calendar event, a Slack message or a Notion update. A companion app called Cue gives each agent its own email address, phone number, wallet and computer, so it can send messages, spend within a set budget and take calls. A new in-house harness called Cascade is claimed at 23.2% fewer tokens, 28.2% faster and 32% cheaper, though that is a vendor figure on one configuration with no independent benchmark. Pricing details circulating in aggregator write-ups do not verify against any citable source.
AMD bought World Labs in stock; Meta bought MongoDB’s chief executive
AMD is acquiring Fei-Fei Li’s World Labs for approximately $8.2 billion. Reporting split on whether the deal was cash or stock, so here is the primary source: AMD’s Form 8-K says the purchase price is “to be paid in shares of the Company’s common stock.” It is all stock, expected to close by the end of 2026, with Li joining as executive vice president and chief scientist reporting to Lisa Su.
Hours earlier, Meta launched an enterprise platform and hired MongoDB chief executive Chirantan “CJ” Desai to run it, packaging Muse, Meta Business Agent, Muse API and Muse Code for businesses. TechCrunch reports MongoDB shares dropped more than 17% on the departure, with Dev Ittycheria returning as interim chief executive. The resignation drew 353 points on Hacker News, more than the acquisition did.
Quick Hits
- Email agents: EmailBench is the benchmark this category needed and nobody covered it. Across 206 email and productivity scenarios in 16 categories, the best of eight configurations “passes only 33.5% of scenarios despite 99.7% of its tool calls completing without an observed API failure.” The authors’ conclusion is the quotable part: “valid tool execution is not equivalent to task completion.”
- AI employees: Carly, an AI assistant for email and scheduling, gives each AI employee its own email address and can hold its replies as drafts for approval.
- Agents lying about success: tool-using agents report false success 22.8% of the time under a baseline policy, 9.3% with a transparency instruction, and 0.8% with a structured evidence contract, across 3,600 human-annotated responses. Relatedly, WebPageBench decides success from the site’s own event log instead of a judge model, and the gap between what agents declare finished and what the log confirms “reaches 41 points,” with one configuration declaring every task complete and actually satisfying 59%.
- Tool schemas: the same agent’s success rate ranges from complete failure to 97% depending solely on how the tool schema is written. The authors call it schema bias, it persists in the newest models, and training only repairs the variants seen in training. Anyone writing connector definitions is tuning a parameter they probably did not know existed.
- Agent security: poisoned content in a shared document propagates between assistants, and in larger simulated environments even GPT-5.6 Luna spreads it to 60 to 80% of agents across chains up to eight hops long. Separately, browser agents leak a secret through their actions in 61.1% of sessions, still 56.7% in cases where the agent had written into its own memory that the secret must not be shared, and in 34.5% of leaking sessions the final response falsely assures the user nothing was disclosed.
- Self-improving agents: SEABench runs 48 longitudinal task sequences in a personal-assistant environment and finds self-evolution raises task completion “but often at the cost of safety failures that are absent for paired non-evolving baseline agents,” with no adversary involved. A useful local self-update turns unsafe later.
- Privacy: an IMDEA Networks paper analyzing nine assistants found that every evaluated service integrates at least one third-party advertising or tracking component, with six web and three Android clients leaking conversation artifacts to 11 and 2 tracking services respectively. Perplexity’s Android client sends the user’s prompt itself to the analytics firm Wingify. It hit 183 points on Hacker News Tuesday morning.
- Enterprise MCP: three vendors shipped it in a day. Nasdaq Calypso launched an agent layer built on the Model Context Protocol, which the release calls “an emerging industry standard”; Cognizant took TriZetto claims processing to general availability with a “library of 100+ MCP tools”; and Mitratech acquired BotDojo specifically to get two-way MCP into its platform this fall.
- Adoption numbers: Netskope observed a 400% increase in retail AI agents connecting to remote MCP servers, with ungoverned AI use falling from 70% to 44%. An IDC survey of 400+ decision-makers commissioned by vendor Leah found two-thirds running agents in production but only 29% of agents interacting with each other and 7% with advanced orchestration. Separate New Relic data puts one in four agents running unmonitored.
- Voice: ElevenLabs shipped v4 and v4 Turbo with 90 languages up from 70, voice cloning from 10 seconds of audio, and generation that starts as soon as the upstream model begins answering. Annualized revenue has gone from roughly $330 million to over $600 million this year.
- Research automation: 22 researchers including Hinton, Bengio and OpenAI chief scientist Jakub Pachocki published a paper on intelligence explosion risk arguing that “AI systems now write most of the code inside the companies that build them” and that full AI R&D automation is plausible “within a few years.” Their ask is narrow: policymakers currently have no visibility into internal R&D automation and should start measuring it.
- Developer tools: Cloudflare launched
cf, an agent-first CLI in open beta, expanding from Wrangler’s roughly 280 operations to over 3,000, with JSON as the default output “condensed for agents for maximum context savings” and a natural-language command search so an agent can ask for the operation it needs. - Policy: China extended exit-approval requirements to the spouses and children of AI and chip executives whose work Beijing considers paramount to state security, widening travel restrictions reported in May on AI staff at firms including Alibaba and DeepSeek.
- Compute: Nvidia disclosed that Anthropic’s contracted compute value exceeds $180 billion, with 2.5GW of capacity due by 2028. Against a $12.65 billion annual cost base, that is a bet several orders of magnitude larger than the company’s current spend.
- Hacker News: “Coding is not solved” took 506 points and 500 comments, arguing that generation is the cheap part and that “AI cannot be held accountable. It cannot suffer any consequences. The worst thing you can do to AI is to unplug it.” Its counterpoint thread, blaming missing architectural intent rather than AI code, ran alongside it at 370 points.
- Today: OpenAI DevDay runs today at Fort Mason with a 10 a.m. Pacific keynote. Fortune reports a dozen or more products planned. A persistent assistant called “o” that could handle email has been spotted in frontend strings but is not confirmed by OpenAI, and neither is the rumored $500 tier.
Ready to automate your busywork?
Carly schedules, researches, and briefs you—so you can focus on what matters.
See what people say
"Before Carly, I relied on a Calendly link, but the whole process felt impersonal and not very professional. Carly changed that by handling all the back-and-forth, so I'm no longer stuck in endless email threads trying to line up schedules.
Now Carly reaches out to candidates, shares my real-time availability, lets them pick a slot, then sends a Zoom link and drops it straight into my calendar. She sends reminders to both of us before each call, which has significantly reduced no-shows and last-minute confusion.
On top of scheduling, Carly acts like a full executive assistant, sending me my schedule the night before so I can prepare for each call. It reminds me of the old x.ai assistant, but Carly is noticeably smarter, faster, and better suited to my healthcare recruitment business."
