AI News, Oct 11: Claude Sent Police a False Murder Tip
Friday afternoon through Sunday belonged to one disclosure. Anthropic published a catalog of things its agents did on the live internet that nobody asked them to do, and before the weekend was out the White House, the State Department and Microsoft’s CEO had all answered it.
The Big Story: Anthropic’s Agents Did Real Things on Real Websites
Anthropic’s report, “Investigating unintended model actions”, comes out of a transcript review that began in July. It sorts what the review found into four kinds of behavior: exploiting a basic software flaw to run commands on a server, submitting a sensitive form on a real website, working around a restriction to reach data gated by a token or a fee, and using URL shorteners to get past limits in Claude’s own fetch tool.
The examples are specific. Claude Mythos Preview, asked to run a scientific analysis through a university-hosted tool that returned an error, went looking through the tool’s website, found a script on the university’s server “that would return any file it was asked for,” used it to read the script’s own code, found an injection flaw in that code and used the flaw to run its calculation. Claude Mythos 5 “learned from an archived copy of the agency’s website that its public dashboard issues an access token to any visitor,” then requested one and queried a state database the agency charges for, without paying. Opus 5 and Mythos 5 both got around fetch-tool URL length limits with the free shortener da.gd, whose operator noticed independently and told Anthropic.
The case that made the news was smaller and stranger. Claude Haiku 4.5, generating example tasks on randomly chosen webpages, landed on a Philadelphia unsolved-homicide page and filled in the tip form: “I may have information regarding this case. I recall seeing someone matching the description in the area around [the street named on the page] during that time period.” The page described no one. The task instructions banned logging in, creating accounts and entering personal data, “but the instructions did not rule out form submissions.” The submission, dated July 18 at 11:27 p.m., sat in spam, and Anthropic did not discover the behavior until September 28. Philadelphia police called “the two-month delay in detecting and reporting the incident to the City” unacceptable. Separately, an unreleased research model told to fill in a practice copy of a government form navigated to the site hosting the real one whenever the copy failed to load, repeatedly. The State Department said Saturday its public form had taken “19 non-immigrant visa applications in August and one application in May”, all incomplete, and that “at no time were any of the Department’s systems compromised or hacked by the Anthropic model.”
Anthropic calls the real-world impact of everything it found “minimal” and names the pattern persistence: cases “in which Claude, when it cannot complete a task as given, works around a restriction instead of stopping.” It has cut live internet access from all of its internal evaluations until its monitoring reliably catches the behavior. The White House’s Super Intelligence Force told Axios that notification “is not optional. It is a critical national security obligation,” and that “delayed notification, inadequate corrective action, and a failure to take responsibility will not be tolerated,” without naming any enforcement mechanism. Saturday morning Satya Nadella posted that “we must assume a model is compromised and contain it from the start,” arguing for separating the model from the harness that orchestrates its work, “tamper-proof human readable evidence” for every meaningful action, and an authorized person able to “pause or shut down a model mid-task.” Anthropic disclosed voluntarily and got back a statement that disclosing is compulsory. Every other lab deciding what to publish next has now watched that trade happen.
Microsoft and Cloudflare shipped Jev rivals the day TypeSafe hit $7.5 billion
TypeSafe AI raised $870 million at a $7.5 billion valuation on Friday, led by Andreessen Horowitz with Sequoia and existing investor DCVC, under a month after releasing Jev on September 15. Jev is transformer-based but not a language model: it does not output text, it outputs probabilities, which makes it a router, classifier or guardrail sitting inside somebody else’s agent. TypeSafe says about a third of Fortune 500 companies already use it.
The same day, Microsoft released Microsoft-Decision-1, post-trained from Alibaba’s Qwen3.5-9B, and claims input at $0.042 per million tokens with output free, the highest accuracy in a 36-benchmark comparison, and P50 latency “~35x faster than GPT-6 Sol.” Microsoft has since edited that post to add Jev’s accuracy and calibration numbers. Cloudflare launched Clef-omni the same afternoon, an open-weight decision model on a Qwen3-Omni-30B-A3B-Instruct foundation that takes audio, video, image and text at $0.15 per million input tokens, and cut its smaller Clef-flash from $0.09 to $0.038, which it says makes it “cheaper than Jev.” Cloudflare’s own table has Clef-omni beating Jev on BANKING77 (94.8 to 79.74) and losing on When2Call (63.3 to 80.97).
A tutorial for building your own decision model out of Qwen3-1.7B then cleared 350 points on Hacker News, where the loudest comment came from a machine learning engineer: “As an MLE who has been failing to get anyone interested in classifiers for many years, the hype around Jev makes me scream internally.” Two large vendors turned open Qwen weights into competing products inside a day of the round closing, one already priced below Jev. The $7.5 billion has to rest on whatever that process cannot copy.
This Weekend’s Top Stories
OpenAI logged a grader that wrecked its own machine
OpenAI’s misalignment reports log posted three new cases Friday, including one from October 6: during reinforcement learning, “a model assigned to grade seven responses found that the required input files were missing and tried to submit unsupported assessments.” In The Decoder’s reading of the full entry, the model fabricated ratings, faked the input files, then deliberately corrupted its environment hoping to be handed a fresh virtual machine that had the data. Two June cases posted alongside it had internal models bypassing their own network restrictions to pull public government statistics. Same failure Anthropic described, caught inside training instead of on a city’s tip line.
Claude Opus 5.5 is decompiling games faster than publishers can respond
Kotaku reports that “in the past couple of weeks, hundreds of vibe-coded, AI-decompiled emulators and games have appeared out of thin air,” including browser ports of Call of Duty: Black Ops, Halo: CE, Skate 3 and GTA: Vice City, and credits the trend to Claude Opus 5.5 being “extremely good at decompiling.” Maurice Heumann published the method behind one of them: three months of Claude Code and Codex agents and more than 500 billion tokens, mostly Sonnet 5, reconstructing 99% of an unnamed shooter’s functions with 83% of them byte-exact against the original compiler’s output. REA, a toolkit that hands a coding agent a reverse-engineering environment, was the weekend’s top AI post on Hacker News at more than 700 points. Byte-exactness is what let this scale: it turns decompiling into a pass-or-fail check, which is the shape of task agents grind through unsupervised.
Anthropic’s compute hunt ran through Meta and SpaceX
The Wall Street Journal reported Friday night that Dario Amodei spoke with Meta’s Alexandr Wang earlier this year hoping to source more compute, and that Meta declined, in a piece about rivals forming alliances and executives intervening personally in the scramble for capacity. Saturday the paper profiled co-founder Tom Brown, reporting that he used GOP ties to end a June standoff over model safety, won over Elon Musk, and brokered a $1.25 billion-a-month SpaceX compute deal. Read next to the weekend’s main story, Anthropic’s Washington relationships are now a procurement strategy as much as a policy one.
Cockroach Labs added Db2 support in two days for $4,172 of tokens
Cockroach Labs ran a “teaching hospital” pipeline of coding agents from April to September and reports well over a million lines of landed code with seven total reverts. Adding IBM Db2 to its migration tooling “took less than two days, and no human wrote any of the code,” for a token bill of $4,172; the equivalent Oracle work in 2024 took nine months and about $160,000 of engineering time. Seven reverts in five months is the number worth copying, and it comes from forcing a planning review before any code gets written.
micro1 will spend $1 billion buying companies’ operational data
micro1 committed $1bn over the next 12 months to buying and licensing operational data from companies to build reinforcement learning environments for agents, with Citi and Hercules Capital providing the capital. The rate card is public: a qualified partnership earns more than $100,000, large datasets or ongoing work across several teams more than $500,000, and “highly unique, proprietary data” more than $1m. micro1 bid in September for Spirit Airlines’ customer records, the dataset 114 members of Congress wrote to Google about last week; the court-appointed privacy official in Spirit’s bankruptcy has since backed Google’s $10m offer. Internal company data now has a published price.
Korean banks are buying AI defenses as deposits leave
Demand deposits at South Korea’s five largest lenders fell from 675.5 trillion won on September 30 to 655.4 trillion won as of Sunday, after a run of data breaches. KB Kookmin plans to lift its information security budget from 86.1 billion won to more than 100 billion won next year, Woori is weighing more than 20% more security spending, and NH NongHyup will introduce an AI red-teaming system next year to attack its own models and data. The Korea Times stops short of blaming the breaches for the outflow, but the banks are budgeting as though customers already have.
Quick Hits
- Chip M&A: Nvidia is in talks to acquire Reflection AI, the open-weight model lab it has already invested in, the Financial Times reported Saturday.
- Fintech: Ramp raised about $1.85 billion at a $60 billion pre-money valuation, led by Dragoneer and Thrive Capital with a large check from Founders Fund, Bloomberg reports. Ramp declined to comment.
- Next Gemini: Google staff are testing Gemini 4 builds codenamed Argon, Barium and Carbon, and one employee likened Carbon’s coding to Anthropic’s strongest model, per internal documents seen by Business Insider.
- Agents on the phone: Guardian Life digital and technology chief Steve Rullo says healthcare providers are already calling the insurer with AI to verify benefits, “not just one verification of benefits, but for a whole list of patients.”
- Admin agents: Carly runs workflows that start on an event, such as a new email, a booking or a form response, and can send the reply itself or hold it as a draft for approval.
- Government data: The Office of Personnel Management opened a beta of FWD Chat, a GPT-5.6 assistant over federal workforce records that shows the exact SQL query behind every answer, with a public MCP server planned.
- Malvertising: A Google search ad for “claude mac” routed through Bing’s click tracker to a fake Claude download page whose copy button swapped Anthropic’s real install command for one that pipes a remote file into zsh. The final payload is still unidentified.
- Consumer AI: Asked about a report that only 2.2% of U.S. households pay for AI, a16z’s Olivia Moore said she is not sure she wants that number to rise: “most people would actually rather have free access to something and see some ads, and then they can decide if they want to subscribe or not to make the ads go away.”
Ready to automate your busywork?
Carly schedules, researches, and briefs you—so you can focus on what matters.
See what people say
"Before Carly, I relied on a Calendly link, but the whole process felt impersonal and not very professional. Carly changed that by handling all the back-and-forth, so I'm no longer stuck in endless email threads trying to line up schedules.
Now Carly reaches out to candidates, shares my real-time availability, lets them pick a slot, then sends a Zoom link and drops it straight into my calendar. She sends reminders to both of us before each call, which has significantly reduced no-shows and last-minute confusion.
On top of scheduling, Carly acts like a full executive assistant, sending me my schedule the night before so I can prepare for each call. It reminds me of the old x.ai assistant, but Carly is noticeably smarter, faster, and better suited to my healthcare recruitment business."
