Beatrice Nolan here. Anthropic is about to IPO, reportedly at a record breaking $2 trillion. If you’re an employee at the company, especially one that’s been there for some time, you are about to come into a life-changing amount of money.
So quitting, publicly sounding the alarm about the technology you are building, or predicting the death of all humans due to your flagship product, may seem like an unusual move at such a precarious moment. But that’s exactly what a fair few of Anthropic researchers, current and former, have been up to in the last week.
Here’s one post from an Anthropic employee called Evan Hubinger, who leads the company’s Alignment Science team, in response to a colleague’s resignation post:
“We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”
While a 10% chance of human destruction via AI may sound daunting, it’s not a new stance for a lot of these researchers. Some, including those who had a hand in creating the technology, have been saying this since the beginning of the decade, some even longer. (Here’s a handy chart if you want to see what some of these leading researchers have for their ‘p.doom’ number.)
While very much up for debate whether these doomsday scenarios from Hubinger and various other concerned researchers are likely to actually happen or not. It certainly seems like they really do believe it.
See this post from another Anthropic researcher, Drake Thomas:
“I would burn my equity to the ground in a heartbeat for a 1% higher chance we make it out of this situation alive. I expect a great many of my colleagues across the industry would as well. I promise you, we are actually just fucking scared, it’s not galaxy brained marketing.”
None of this should come as much of a surprise, really. Anthropic was founded as an AI safety lab in the first place, and CEO Dario Amodei himself has put his own p(doom) at somewhere between 10% and 25%. Researchers voicing this kind of alarm are, in some sense, just saying out loud what’s been baked into the company’s identity from day one.
However, the timing of this new frenzy over AI risk does seem odd, just weeks out from an IPO. It’s left me asking: what will this mean for the company’s historic public offering?
A public listing may bring its own pressure against Anthropic’s cautious foundations—a public company answers to shareholders expecting growth, which could mean less appetite for the kind of voluntary slowdown Amodei has floated. The fact that researchers say Anthropic doesn’t have a plan for controlling risks could also spook investors.
Some already appear to have been listening.
This week, SOC Investment Group, a labor-affiliated shareholder group that works with union-sponsored pension funds, called on Anthropic to delay the offering, arguing the company’s confidential IPO filing was submitted in June—before the recent hacking incidents, extinction warnings, and pacing proposals emerged—leaving investors unable to properly price those risks.
On the other hand, there could also be a safety case for going ahead with the IPO, the argument being that going public brings transparency and scrutiny that would make the company more, not less, safe.
Neither companies nor consumers actually want misaligned AI that could cause havoc in their products, and a track record of taking the risks seriously—third-party evaluation, pacing commitments—could be a selling point for enterprise customers and consumers alike.
In contrast, OpenAI has pumped the brakes on its own IPO, telling Fortune late last week that AI safety concerns made it, in CEO Sam Altman’s words, “an ill-advised moment to go public” right now.
One thing’s for sure. When those S-1s eventually emerge from Anthropic and OpenAI, the “risks” sections will surely make for some highly interesting reading.
See you tomorrow,
Beatrice Nolan
X: @beafreyanolan
Email: beatrice.nolan@fortune.com
Submit a deal for the Term Sheet newsletter here.
Joey Abrams curated the deals section of today’s newsletter. Subscribe here.
VENTURE CAPITAL
– Impulse Space, a Redondo Beach, Calif.-based developer of in-space transportation vehicles for moving payloads between orbits, raised $308 million in a Series D extension from 137 Ventures, BANNER VC, DFJ Growth, Linse Capital, Lux Capital, and Valor Equity Partners.
– EnduroSat, a Sofia, Bulgaria-based satellite manufacturer and space service provider, raised $205 million in funding. Riot Ventures and Atreides Management led the round and were joined by European Innovation Council, Google Ventures, Founders Fund, and others.
– Thatch, a San Francisco-based health benefits platform, raised $108 million in Series C funding from The General Partnership, Index Ventures, Scale Venture Partners, and others.
– Hang Ten Systems, a Menlo Park, Calif.-based enterprise AI services company that helps large businesses build, modernize, and operate software, raised $53 million in seed funding. Xora led the round and was joined by Mayfield, Aramco Ventures, and others.
– BackOps, a San Francisco-based AI platform for resolving supply-chain workflows at companies that make or move physical goods, raised $42 million in Series B funding. Insight Partners led the round and was joined by existing investors Theory Ventures, Construct Capital, Gradient Ventures, and 10VC.
– Noetive, a San Francisco-based applied-AI research lab building self-improving systems for industrial businesses, raised $41 million in seed funding. Eclipse led the round and was joined by Craft Ventures, The Westly Group, Swish Ventures, Factory, Incite Ventures, and others.
– Rune, a Mountain View, Calif.-based developer of a product that turns renewable energy into AI compute, raised $40 million in Series A funding. Spark Capital led the round and was joined by Union Square Ventures, Lowercarbon Capital, Activate Capital, and others.
– Altis Labs, a Toronto, Ontario-based AI-powered software designed to predict drug efficacy, raised $25 million in Series A funding. OrbiMed and Qiming Venture Partners USA led the round and were joined by Innovation Endeavors, Benchstrength, Fusion Fund, the Cancer Breakthrough Fund, and others.
– Hackuity, a Lyon, France-based risk-based vulnerability-management platform, raised $19 million in funding. Forgepoint Capital International led the round and was joined by Bright Pixel, Bpifrance, and Seventure Partners.
– R3 Lithium, a Covington, Ga.-based lithium carbonate recovery company, raised $15 million in Series A funding from Integral GlobalTech Partners, Axial Partners, TDK Ventures, and others.
– AcouBatt, a London, U.K.-based battery-monitoring company that uses acoustic sensors and AI to improve cell manufacturing, quality control, and safety, raised £1.1 million ($1.5 million) in pre-seed funding. Creator Fund led the round and was joined by Ada Ventures.
PRIVATE EQUITY
– Silversmith Capital Partners invested $100 million in Vantora, a Santa Monica, Calif.-based builder of bespoke AI-native operating companies for industrial enterprises.
– Brookfield Capital Partners agreed to acquire Reliance Worldwide, an Atlanta, Ga.-based plumbing and heating solutions company. Financial terms were not disclosed.
– Niobrara Capital acquired MSP Corp, a Montreal, Quebec-based managed IT and cybersecurity services company. Financial terms were not disclosed.
– TRG, a portfolio company of Gemspring Capital, acquired Vista Technology Support, a Cardiff, Wales-based tech field services and support company. Financial terms were not disclosed.
EXITS
– Bertram Capital acquired STARC Systems, a Brunswick, Maine-based maker of reusable temporary containment-wall systems for construction projects in occupied or sensitive environments, from North Branch Capital. Financial terms were not disclosed.
FUNDS + FUNDS OF FUNDS
– Bain Capital Ventures, a San Francisco-based venture capital firm, raised $1.6 billion for its 11th fund focused on AI, enterprise software, infrastructure, fintech, cybersecurity, healthcare, commerce, and industrial-technology companies.
– Portage, a Toronto, Canada-based investment platform, raised $600 million for its fourth fund focused on financial technology and financial services companies.
– Matter Venture Partners, a Los Altos, Calif.-based venture capital firm, raised $450 million for its second fund focused on early-stage companies across semiconductors, robotics and Physical AI, AI infrastructure, quantum computing, advanced manufacturing, energy building blocks, and AI.
PEOPLE
– Strategic Value Partners, a Greenwich, Conn.-based alternative investment firm, hired Matt Gibson as President. Previously, he was with Goldman Sachs.
