Connect with us

Business

AI’s ability to ‘think’ makes it more vulnerable to new jailbreak attacks, new research suggests

Published

on



New research suggests that advanced AI models may be easier to hack than previously thought, raising concerns about the safety and security of some leading AI models already used by businesses and consumers.

A joint study from Anthropic, Oxford University, and Stanford undermines the assumption that the more advanced a model becomes at reasoning—its ability to “think” through a user’s requests—the stronger its ability to refuse harmful commands.

Using a method called “Chain-of-Thought Hijacking,” the researchers found that even major commercial AI models can be fooled with an alarmingly high success rate, more than 80% in some tests. The new mode of attack essentially exploits the model’s reasoning steps, or chain-of-thought, to hide harmful commands, effectively tricking the AI into ignoring its built-in safeguards.

These attacks can allow the AI model to skip over its safety guardrails and potentially open the door for it to generate dangerous content, such as instructions for building weapons or leaking sensitive information.

A new jailbreak

Over the last year, large reasoning models have achieved much higher performance by allocating more inference-time compute—meaning they spend more time and resources analyzing each question or prompt before answering, allowing for deeper and more complex reasoning. Previous research suggested this enhanced reasoning might also improve safety by helping models refuse harmful requests. However, the researchers found that the same reasoning capability can be exploited to circumvent safety measures.

According to the research, an attacker could hide a harmful request inside a long sequence of harmless reasoning steps. This tricks the AI by flooding its thought process with benign content, weakening the internal safety checks meant to catch and refuse dangerous prompts. During the hijacking, researchers found that the AI’s attention is mostly focused on the early steps, while the harmful instruction at the end of the prompt is almost completely ignored.

As reasoning length increases, attack success rates jump dramatically. Per the study, success rates jumped from 27% when minimal reasoning is used to 51% at natural reasoning lengths, and soared to 80% or more with extended reasoning chains.

This vulnerability affects nearly every major AI model on the market today, including OpenAI’s GPT, Anthropic’s Claude, Google’s Gemini, and xAI’s Grok. Even models that have been fine-tuned for increased safety, known as “alignment-tuned” models, begin to fail once attackers exploit their internal reasoning layers.

Scaling a model’s reasoning abilities is one of the main ways that AI companies have been able to improve their overall frontier model performance in the last year, after traditional scaling methods appeared to show diminishing gains. Advanced reasoning allows models to tackle more complex questions, helping them act less like pattern-matchers and more like human problem solvers.

One solution the researchers suggest is a type of “reasoning-aware defense.” This approach keeps track of how many of the AI’s safety checks remain active as it thinks through each step of a question. If any step weakens these safety signals, the system penalizes it and brings the AI’s focus back to the potentially harmful part of the prompt. Early tests show this method can restore safety while still allowing the AI to perform well and answer normal questions effectively.



Source link

Continue Reading

Business

A Thanksgiving dealmaking sprint helped Netflix win Warner Bros.

Published

on



The Netflix Inc. plans that clinched the deal for Warner Bros. Discovery Inc. started to shape up around Thanksgiving. 

deadline was looming: Warner Bros. had asked bidders, which also included Paramount Skydance Corp. and Comcast Corp., to have their latest proposals and contracts in by the Monday after the holiday, following a round about a week earlier. The suitors were told to put their best foot forward.

While most Americans were watching football and feasting on turkey, Netflix executives and advisers hunkered down to finalize a binding offer and a $59 billion bridge loan from banks, one of the biggest of its kind. That gave the streaming company the ammunition to make a mostly cash-and-stock bid that helped it prevail over Comcast and David Ellison’s Paramount, according to people familiar with the matter.

The resulting $72 billion deal, announced Friday, is set to bring about a seismic shift in the entertainment business — if it can survive intense regulatory scrutiny and a potential fight from Paramount. This account of Netflix’s surprise victory in the biggest M&A auction of the year is based on interviews with half a dozen people involved in negotiations. They asked not to be identified because the details are confidential.

The sales process had kicked off with several unsolicited bids from Paramount Skydance, itself a newly formed company after a merger this year orchestrated by Ellison. He’s now the studio’s chief executive officer and controlling shareholder, with backing from his father, Oracle Corp. billionaire Larry Ellison. 

Paramount’s early move gave it a head start in the bidding process weeks before other would-be buyers got access to information. But the post-Thanksgiving deadline for second-round bids became a turning point by giving Netflix time to catch up and assemble the documents it needed, some of the people said. And since the streaming giant was bred in the fast-paced ethos of Silicon Valley, it could move quickly. 

When the binding bids arrived that Monday, Netflix’s offer emerged as superior, the people said.

One issue was the Warner Bros. camp had doubts about how Paramount would pay for the company, which owns sprawling Hollywood studios, the HBO network and a vast film and TV library. Paramount’s offer included financing from Apollo Global Management Inc. and several Middle Eastern funds, and it had conveyed that its bid was fully backstopped by the Ellisons. Still, Warner Bros. executives were privately concerned about the certainty of the financing, people familiar with the matter said.

Representatives for Netflix and Warner Bros. declined to comment.

‘Noble’ vs ‘Prince’

In the weeks leading up to the finale, Warner Bros. advisers set up war rooms at various hotels in midtown Manhattan. A core group holed up at the Loews Regency, which has long been a convening spot for the city’s movers and shakers.

Inside Warner Bros., the situation was known as “Project Sterling.” The company called itself by the code name “Wonder.” The team referred to Netflix as “Noble,” while Paramount was “Prince” and Comcast was “Charm.”

At Netflix, Chief Financial Officer Spencer Neumann served as the point man while corporate development head Devorah Bertucci organized people day-to-day. Chief Legal Officer David Hyman and Spencer Wang, vice president of finance, investor relations and corporate development, also were key architects, with all of them reporting into co-CEOs Ted Sarandos and Greg Peters.

The contours of the deal were shaped in a way befitting of a tech company: mostly over video chat or phone rather than in person. Virtual war rooms were set up. While strategizing or discussing diligence on Zoom, participants would raise virtual hands or make suggestions over chat rather than unmuting and slowing down the meeting. Google Docs were used to review and edit documents together in real time.

Talks heated up this week, with Warner Bros. advisers in continuous dialogue with the bidders and negotiating contract language and value. Comcast said it would merge its NBCUniversal division with Warner Bros. Paramount offered to more than double its proposed breakup fee to $5 billion to sweeten its deal and outshine rivals. 

In the end, Warner Bros. determined Netflix had the best offer and the company was the most flexible on key terms. On Wednesday, Paramount lobbed an aggressively worded letter to Warner Bros. board saying the sales process was “tainted.” It also identified what it saw as regulatory risks in the Netflix proposal, one sign that a winning outcome was slipping away for Paramount. 

Netflix found out Thursday evening New York time that it had won. Executives and advisers were assembled on a video call when they got the official word, sparking a moment of jubilation before everyone snapped into action. By 10:25 p.m., Bloomberg News broke the news that a deal was imminent. 

Even Sarandos made it sound like the ending was a twist on a conference call with investors. “I know some of you are surprised that we’re making this acquisition, and I certainly understand why,” he said. “Over the years, we have been known to be builders, not buyers.”

Regardless of whether Paramount reemerges to try and top the bid, Netflix will have work ahead of it. It has agreed to pay a $5.8 billion breakup fee to Warner Bros. if the transaction fails on regulatory grounds. The company also has to digest its largest acquisition ever.

“It’s going to be a lot of hard work,” co-CEO Peters said on the conference call. “We’re not experts at doing large-scale M&A, but we’ve done a lot of things historically that we didn’t know how to do.”



Source link

Continue Reading

Business

‘Its own research shows they encourage addiction’: Highest court in Mass. hears case about Instagram, Facebook effect on kids

Published

on



Massachusetts’ highest court heard oral arguments Friday in the state’s lawsuit arguing that Meta designed features on Facebook and Instagram to make them addictive to young users.

The lawsuit, filed in 2024 by Attorney General Andrea Campbell, alleges that Meta did this to make a profit and that its actions affected hundreds of thousands of teenagers in Massachusetts who use the social media platforms.

“We are making claims based only on the tools that Meta has developed because its own research shows they encourage addiction to the platform in a variety of ways,” said State Solicitor David Kravitz, adding that the state’s claim has nothing to do the company’s algorithms or failure to moderate content.

Meta said Friday that it strongly disagrees with the allegations and is “confident the evidence will show our longstanding commitment to supporting young people.” Its attorney, Mark Mosier, argued in court that the lawsuit “would impose liabilities for performing traditional publishing functions” and that its actions are protected by the First Amendment.

“The Commonwealth would have a better chance of getting around the First Amendment if they alleged that the speech was false or fraudulent,” Mosier said. “But when they acknowledge that its truthful that brings it in the heart of the First Amendment.”

Several of the judges, though, seem to more concerned about Meta’s functions such as notifications than the content on its platforms.

“I didn’t understand the claims to be that Meta is relaying false information vis-a-vis the notifications but that it has created an algorithm of incessant notifications … designed so as to feed into the fear of missing out, fomo, that teenagers generally have,” Justice Dalila Wendland said. “That is the basis of the claim.”

Justice Scott Kafker challenged the notion that this was all about a choose to publish certain information by Meta.

“It’s not how to publish but how to attract you to the information,” he said. “It’s about how to attract the eyeballs. It’s indifferent the content, right. It doesn’t care if it’s Thomas Paine’s ‘Common Sense’ or nonsense. It’s totally focused on getting you to look at it.”

Meta is facing federal and state lawsuits claiming it knowingly designed features — such as constant notifications and the ability to scroll endlessly — that addict children.

In 2023, 33 states filed a joint lawsuit against the Menlo Park, California-based tech giant claiming that Meta routinely collects data on children under 13 without their parents’ consent, in violation of federal law. In addition, states including Massachusetts filed their own lawsuits in state courts over addictive features and other harms to children.

Newspaper reports, first by The Wall Street Journal in the fall of 2021, found that the company knew about the harms Instagram can cause teenagers — especially teen girls — when it comes to mental health and body image issues. One internal study cited 13.5% of teen girls saying Instagram makes thoughts of suicide worse and 17% of teen girls saying it makes eating disorders worse.

Critics say Meta hasn’t done enough to address concerns about teen safety and mental health on its platforms. A report from former employee and whistleblower Arturo Bejar and four nonprofit groups this year said Meta has chosen not to take “real steps” to address safety concerns, “opting instead for splashy headlines about new tools for parents and Instagram Teen Accounts for underage users.”

Meta said the report misrepresented its efforts on teen safety.

___

Associated Press reporter Barbara Ortutay in Oakland, California, contributed to this report.



Source link

Continue Reading

Business

Quant who said passive era is ‘worse than Marxism’ doubles down

Published

on



Inigo Fraser Jenkins once warned that passive investing was worse for society than Marxism. Now he says even that provocative framing may prove too generous.

In his latest note, the AllianceBernstein strategist argues that the trillions of dollars pouring into index funds aren’t just tracking markets — they are distorting them. Big Tech’s dominance, he says, has been amplified by passive flows that reward size over substance. Investors are funding incumbents by default, steering more capital to the biggest names simply because they already dominate benchmarks.

He calls it a “dystopian symbiosis”: a feedback loop between index funds and platform giants like Apple Inc., Microsoft Corp. and Nvidia Corp. that concentrates power, stifles competition, and gives the illusion of safety. Unlike earlier market cycles driven by fundamentals or active conviction, today’s flows are automatic, often indifferent to risk.

Fraser Jenkins is hardly alone in sounding the alarm. But his latest critique has reignited a debate that’s grown harder to ignore. Just 10 companies now account for more than a third of the S&P 500’s value, with tech names driving an outsize share of 2025’s gains.

“Platform companies and a lack of active capital allocation both imply a less effective form of capitalism with diminished competition,” he wrote in a Friday note. “A concentrated market and high proportion of flows into cap weighted ‘passive’ indices leads to greater risks should recent trends reverse.” 

While the emergence of behemoth companies might be reflective of more effective uses of technology, it could also be the result of failures of anti-trust policies, among other things, he argues. Artificial intelligence might intensify these issues and could lead to even greater concentrations of power among firms. 

His note, titled “The Dystopian Symbiosis: Passive Investing and Platform Capitalism,” is formatted as a fictional dialog between three people who debate the topic. One of the characters goes as far as to argue that the present situation requires an active policy intervention — drawing comparisons to the breakup of Standard Oil at the start of the 20th century — to restore competition.

data-srcyload

In a provocative note titled “The Silent Road to Serfdom: Why Passive Investing is Worse Than Marxism” and written nearly a decade ago, Fraser Jenkins argued that the rise of index-tracking investing would lead to greater stock correlations, which would impede “the efficient allocation of capital.” His employer, AllianceBernstein, has continued to launch ETFs since the famous research was published, though its launches have been actively managed. 

Other active managers have presented similar viewpoints — managers at Apollo Global Management last year said the hidden costs of the passive-investing juggernaut included higher volatility and lower liquidity. 

There have been strong rebuttals to the critique: a Goldman Sachs Group Inc. study showed the role of fundamentals remains an all-powerful driver for stock valuations; Citigroup Inc. found that active managers themselves exert a far bigger influence than their passive rivals on a stock’s performance relative to its industry.

“ETFs don’t ruin capitalism, they exemplify it,” said Eric Balchunas, Bloomberg Intelligence’s senior ETF analyst. “The competition and innovation are through the roof. That is capitalism in its finest form and the winner in that is the investor.”

Since Fraser Jenkins’s “Marxism” note, the passive juggernaut has only grown. Index-tracking ETFs, which have grown in popularity thanks to their ease of trading and relatively cheaper management fees, are often cited as one of the primary culprits in this debate. The segment has raked in $842 billion so far this year, compared with the $438 billion hauled in by actively managed funds, even as there are more active products than there are passive ones, data compiled by Bloomberg show. Of the more than $13 trillion that’s in ETFs overall, $11.8 trillion is parked in passive vehicles. The majority of ETF ownership is concentrated in low-cost index funds that have significantly reduced the cost for investors to access financial markets. 

In Fraser Jenkins’s new note, one of his fictitious characters ask another what the “dystopian symbiosis” implies for investors. 

“The passive index is riskier than it has been in the past,” the character answers. “The scale of the flows that have been disproportionately into passive cap-weighted funds with a high exposure to the mega cap companies implies the risk of a significant negative wealth effect if there is an upset to expectations for those large companies.”



Source link

Continue Reading

Trending

Copyright © Miami Select.