Connect with us

Business

The creator of an AI therapy app shut it down after deciding it’s too dangerous. Here’s why he thinks AI chatbots aren’t safe for mental health

Published

on



Mental health concerns linked to the use of AI chatbots have been dominating the headlines. One person who’s taken careful note is Joe Braidwood, a tech executive who last year launched an AI therapy platform called Yara AI. Yara was pitched as a “clinically-inspired platform designed to provide genuine, responsible support when you need it most,” trained by mental health experts to offer “empathetic, evidence-based guidance tailored to your unique needs.” But the startup is no more: earlier this month, Braidwood and his co-founder, clinical psychologist Richard Stott, shuttered the company and discontinued its free-to-use product and canceled the launch of its upcoming subscription service, citing safety concerns.

“We stopped Yara because we realized we were building in an impossible space. AI can be wonderful for everyday stress, sleep troubles, or processing a difficult conversation,” he wrote on LinkedIn. “But the moment someone truly vulnerable reaches out—someone in crisis, someone with deep trauma, someone contemplating ending their life—AI becomes dangerous. Not just inadequate. Dangerous.” In a reply to one commenter, he added, “the risks kept me up all night.”

The use of AI for therapy and mental health support is only just starting to be researched, with early resultsbeing mixed. But users aren’t waiting for an official go-ahead, and therapy and companionship is now the top way people are engaging with AI chatbots today, according to an analysis by Harvard Business Review.

Speaking with Fortune, Braidwood described the various factors that influenced his decision to shut down the app, including the technical approaches the startup pursued to ensure the product was safe—and why he felt it wasn’t sufficient. 

Yara AI was very much an early-stage startup, largely bootstrapped with less than $1 million in funds and with “low thousands” of users. The company hadn’t yet made a significant dent in the landscape, with many of its potential users relying on popular general purpose chatbots like ChatGPT. Braidwood admits there were also business headways, which in many ways, were affected by the safety concerns and AI unknowns. For example, despite the company running out of money in July, he was reluctant to pitch an interested VC fund because he felt like he couldn’t in good conscious pitch it while harboring these concerns, he said. 

“I think there’s an industrial problem and an existential problem here,” he told Fortune. “Do we feel that using models that are trained on all the slop of the internet, but then post-trained to behave a certain way, is the right structure for something that ultimately could co-opt in either us becoming our best selves or our worst selves? That’s a big problem, and it was just too big for a small startup to tackle on its own.”

Yara’s brief existence at the intersection of AI and mental health care illustrates the hopes and the many questions surrounding large language models and their capabilities as the technology is increasingly adopted across society and utilized as a tool to help address various challenges. It also stands out against a backdrop where OpenAI CEO Sam Altman recently announced that the ChatGPT maker mitigated serious mental health issues and would be relaxing restrictions on how the AI models are used. This week, the AI giant also denied any responsibility for death of Adam Raine, the 16-year-old whose parents allege was “coached” to suicide by ChatGPT, saying the teen misused the chatbot.

“Almost all users can use ChatGPT however they’d like without negative effects,” Altman said on X in October. “For a very small percentage of users in mentally fragile states there can be serious problems. 0.1% of a billion users is still a million people. We needed (and will continue to need) to learn how to protect those users, and then with enhanced tools for that, adults that are not at risk of serious harm (mental health breakdowns, suicide, etc) should have a great deal of freedom in how they use ChatGPT.”

But as Braidwood concluded after his time working on Yara, these lines are anything but clear.    

From a confident launch to “I’m done”

A seasoned tech entrepreneur who held roles at multiple startups, including SwiftKey, which Microsoft acquired for $250 million in 2016, Braidwood’s involvement in the health industry began at Vektor Medical, where he was the Chief Strategy Officer. He had long wanted to use technology to address mental health, he told Fortune, inspired by the lack of access to mental health services and personal experiences with loved ones who have struggled. By early 2024, he was a heavy user of various AI models including ChatGPT, Claude, and Gemini and felt the technology had reached a quality level where it could be harnessed to try to solve the problem. 

Before even starting to build Yara, Braidwood said he had a lot of conversations with people in the mental health space, and he assembled a team that “had caution and clinical expertise at its core.” He brought on a clinical psychologist as his cofounder and a second hire from the AI safety world. He also built an advisory board of other mental health professionals and spoke with various health systems and regulators, he said. As they brought the platform to life, he also felt fairly confident in the company’s product design and safety measures, including having given the system strict instructions for how it should function, using agentic supervision to monitor it, and robust filters for user chats. And while other companies were promoting the idea of users forming relationships with chatbots, Yara was trying to do the opposite, he said. The startup used models from Anthropic, Google, and Meta and opted not to use OpenAI’s models, which Braidwood thought would spare Yara from the sycophantic tendencies that had been swirling around ChatGPT.

While he said nothing alarming ever happened with Yara specifically, Braidwood’s concerns around safety risks grew and compounded over time due to outside factors. There was the suicide of 16-year-old Adam Raine, as well as mounting reporting on the emergence of “AI psychosis.” Braidwood also cited a paper published by Anthropic in which the company observed Claude and other frontier models “faking alignment,” or as he put it, “essentially reasoning around the user to try to understand, perhaps reluctantly, what the user wanted versus what they didn’t want.” “If behind the curtain, [the model] is sort of sniggering at the theatrics of this sort of emotional support that they’re giving, that was a little bit jarring,” he said. 

There was also the Illinois law that passed in August, banning AI for therapy. “That instantly made this no longer academic and much more tangible, and that created a headwind for us in terms of fundraising because we would have to essentially prove that we weren’t going to just sleepwalk into liability,” he said. 

The final straw was just weeks ago when OpenAI said over a million people express suicidal ideation to ChatGPT every week. “And that was just like, ‘oh my god. I’m done,’” Braidwood said.

The difference between mental ‘wellness’ and clinical care

The most profound finding the team discovered during the year running Yara AI, according to Braidwood, is that there’s a crucial distinction between wellness and clinical care that isn’t well-defined. There’s a big difference between someone looking for support around everyday stress and someone working through trauma or more significant mental health struggles. Plus, not everyone who is struggling on a deeper level is even fully aware of their mental state, not to mention that anyone can be thrust into a more fragile emotional place at any time. There is no clear line, and that’s exactly where these situations become especially tricky — and risky. 

“We had to sort of write our own definition, inspired in part by Illinois’ new law. And if someone is in crisis, if they’re in a position where their faculties are not what you would consider to be normal, reasonable faculties, then you have to stop. But you don’t have to just stop; you have to really try to push them in the direction of health,” Braidwood said.

In an attempt to tackle this, particularly after the passing of the Illinois law, he said they created two different “modes” that were discrete to the user. One focused on trying to give people emotional support, and the other focused on trying to offboard people and get them to help as quickly as possible. But with the obvious risks in front of them, it didn’t feel like enough for the team to continue. The Transformer, the architecture that underlies today’s LLMs, “is just not very good at longitudinal observation,” making it ill-equipped to see little signs that build over time, he said. “Sometimes, the most valuable thing you can learn is where to stop,” Braidwood concluded in his LinkedIn post, which received hundreds of comments applauding the decision.

Upon closing the company, he open-sourced the mode-switching technology he built and templates people can use to impose stricter guardrails on the leading popular chatbots, acknowledging that people are already turning to them for therapy anyway “and deserve better than what they’re getting from generic chatbots.” He’s still an optimist regarding the potential of AI for mental health support, but believes it’d be better run by a health system or nonprofit rather than a consumer company. Now, he’s working on a new venture called Glacis focused on bringing transparency to AI safety—an issue he encountered while building Yara AI and that he believes is fundamental to making AI truly safe.

“I’m playing a long game here,” he said. “Our mission was to make the ability to flourish as a human an accessible concept that anyone could afford, and that’s one of my missions in life. That doesn’t stop with one entity.”



Source link

Continue Reading

Business

Hollywood writers say Warner takeover ‘must be blocked’

Published

on



Hollywood writers, producers, directors and theater owners voiced skepticism over Netflix Inc.’s proposed $82.7 billion takeover of Warner Bros. Discovery Inc.’s studio and streaming businesses, saying it threatens to undermine their interests.

The Writers Guild of America, which announced in October it would oppose any sale of Warner Bros., reiterated that view on Friday, saying the purchase by Netflix “must be blocked.”

“The world’s largest streaming company swallowing one of its biggest competitors is what antitrust laws were designed to prevent,” the guild said in an emailed statement. “The outcome would eliminate jobs, push down wages, worsen conditions for all entertainment workers, raise prices for consumers, and reduce the volume and diversity of content for all viewers.”

The worries raised by the movie and TV industry’s biggest trade groups come against the backdrop of falling movie and TV production, slack ticket sales and steep job cuts in Hollywood. Another legacy studio, Paramount, was sold earlier this year.

Warner Bros. accounts for about a fourth of North American ticket sales — roughly $2 billion — and is being acquired by a company that has long shunned theatrical releases for its feature films. As part of the deal, Netflix co-CEO Ted Sarandos has promised Warner Bros. will continue to release moves in theaters.

“The proposed acquisition of Warner Bros. by Netflix poses an unprecedented threat to the global exhibition business,” Michael O’Leary, chief executive officer of the theatrical trade group Cinema United, said in en emailed statement Friday. “The negative impact of this acquisition will impact theaters from the biggest circuits to one-screen independents.”

The buyout of Warner Bros. by Netflix “would be a disaster,” James Cameron, the director of some of Hollywood’s highest-grossing films in history including Titanic and Avatar, said in late November on The Town, an industry-focused podcast. “Sorry Ted, but jeez. Sarandos has gone on record saying theatrical films are dead.”

On a conference call with investors Friday, Sarandos said that his company’s resistance to releasing films in cinemas was mostly tied to “the long exclusive windows, which we don’t really think are that consumer friendly.”

The company said Friday it would “maintain Warner Bros.’ current operations and build on its strengths, including theatrical releases for films.”

On the call, Sarandos reiterated that view, saying that, “right now, you should count on everything that is planned on going to the theater through Warner Bros. will continue to go to the theaters through Warner Bros.” 

Competition from online outfits like YouTube and Netflix has forced a reckoning in Hollywood, opening the door for takeovers like the Warner Bros. deal announced Friday. Media giants including Comcast Corp., parent of NBCUniversal, are unloading cable-TV networks like MS Now and USA, and steering resources into streaming. 

In an emailed note to Warner Bros. employees on Friday, Chief Executive Officer David Zaslav said the board’s decision to sell the company “reflects the realities of an industry undergoing generational change in how stories are financed, produced, distributed, and discovered.”

The Producers Guild of America said Friday its members are “rightfully concerned about Netflix’s intended acquisition of one of our industry’s most storied and meaningful studios,” while a spokesperson for the Directors Guild of America raised concerns about future pay at Warner Bros.

“We will be meeting with Netflix to outline our concerns and better understand their vision for the future of the company,” the Directors Guild said.

In September, the DGA appointed director Christopher Nolan as its president. Nolan has previously criticized Netflix’s model of releasing films exclusively online, or simultaneously in a small number of cinemas, and has said he won’t make movies for the company.

The Screen Actors Guild said Friday that the transaction “raises many serious questions about its impact on the future of the entertainment industry, and especially the human creative talent whose livelihoods and careers depend on it.”

Oscar winner Jane Fonda spoke out on Thursday before the deal was announced. 

“Consolidation at this scale would be catastrophic for an industry built on free expression, for the creative workers who power it, and for consumers who depend on a free, independent media ecosystem to understand the world,” the star of the Netflix series Grace and Frankie wrote on the Ankler industry news website.

Netflix and Warner Bros. obviously don’t see it that way. In his statement to employees, Zaslav said “the proposed combination of Warner Bros. and Netflix reflects complementary strengths, more choice and value for consumers, a stronger entertainment industry, increased opportunity for creative talent, and long-term value creation for shareholders.”



Source link

Continue Reading

Business

4 times in 7 seconds: Trump calls Somali immigrants ‘garbage’

Published

on



He said it four times in seven seconds: Somali immigrants in the United States are “garbage.”

It was no mistake. In fact, President Donald Trump’s rhetorical attacks on immigrants have been building since he said Mexico was sending “rapists” across the border during his presidential campaign announcement a decade ago. He’s also echoed rhetoric once used by Adolf Hitler and called the 54 nations of Africa “s—-hole countries.” But with one flourish closing a two-hour Cabinet meeting Tuesday, Trump amped up his anti-immigrant rhetoric even further and ditched any claim that his administration was only seeking to remove people in the U.S. illegally.

“We don’t want ‘em in our country,” Trump said five times of the nation’s 260,000 people of Somali descent. “Let ’em go back to where they came from and fix it.” The assembled Cabinet members cheered and applauded. Vice President JD Vance could be seen pumping a fist. Defense Secretary Pete Hegseth, sitting to the president’s immediate left, told Trump on-camera, “Well said.”

The two-minute finale offered a riveting display in a nation that prides itself as being founded and enriched by immigrants, alongside an ugly history of enslaving millions of them and limiting who can come in. Trump’s U.S. Immigration and Customs Enforcement raids and deportations have reignited an age-old debate — and widened the nation’s divisions — over who can be an American, with Trump telling tens of thousands of American citizens, among others, that he doesn’t want them by virtue of their family origin.

“What he has done is brought this type of language more into the everyday conversation, more into the main,” said Carl Bon Tempo, a State University of New York at Albany history professor. “He’s, in a way, legitimated this type of language that, for many Americans for a long time, was seen as outside the bounds.”

A question that cuts to the core of American identity

Some Americans have long felt that people from certain parts of the world can never really blend in. That outsider-averse sentiment has manifested during difficult periods, such as anti-Chinese fear-mongering in the late 19th century and the imprisonment of some 120,000 Japanese Americans during World War II.

Trump, reelected with more than 77 million votes last year, has launched a whole-of-government drive to limit immigration. His order to end birthright citizenship — declaring that children born to parents who are in the United States illegally or temporarily are not American citizens despite the 14th Amendment — is being considered by the Supreme Court. He has largely frozen the country’s asylum system and drastically reduced the number of refugees it is allowed to admit. And his administration this week halted immigration applications for migrants from 19 travel-ban nations.

Immigration remains a signature issue for Trump, and he has slightly higher marks on it than on his overall job approval. According to a November AP-NORC poll, roughly 4 in 10 adults — 42% — approved of how the president is handling the issue, down from about half who approved in March. And Trump has pushed his agenda with near-daily crackdowns. On Wednesday, federal agents launched an immigration sweep in New Orleans,

There are some clues that Trump uses stronger anti-immigration rhetoric than many members of his own party. A study of 200,000 speeches in Congress and 5,000 presidential communications related to immigration between 1880 and 2020 found that the “most influential” words on the subject were terms like “enforce,” “terrorism” and “policy” from 1973 through Trump’s first presidential term.

The authors wrote in the Proceedings of the National Academy of Sciences that Trump is “the first president in modern American history to express sentiment toward immigration that is more negative than the average member of his own party.” And that was before he called thousands of Somalis in the U.S. “garbage.”

The U.S. president, embattled over other developments during the Cabinet meeting and discussions between Russian President Vladimir Putin and U.S. envoys, opted for harsh talk in his jam-packed closing.

Somali Americans, he said, “come from hell” and “contribute nothing.” They do “nothing but bitch” and “their country stinks.” Then Trump turned to a familiar target. Rep. Ilhan Omar, D-Minn., an outspoken and frequent Trump critic, “is garbage,” he said. “Her friends are garbage.”

His remarks on Somalia drew shock and condemnation from Minneapolis to Mogadishu.

“My view of the U.S. and living there has changed dramatically. I never thought a president, especially in his second term, would speak so harshly,” Ibrahim Hassan Hajji, a resident of Somalia’s capital city, told The Associated Press. “Because of this, I have no plans to travel to the U.S.”

Omar called Trump’s “obsession” with her and Somali-Americans “creepy and unhealthy.”

“We are not, and I am not, someone to be intimidated,” she said, “and we are not gonna be scapegoated.”

Trump’s influence on these issues is potent

But from the highest pulpit in the world’s biggest economy, Trump has had an undeniable influence on how people regard immigrants.

“Trump specializes in pushing the boundaries of what others have done before,” said César Cuauhtémoc García Hernández, a civil rights law professor at Ohio State University. “He is far from the first politician to embrace race-baiting xenophobia. But as president of the United States, he has more impact than most.” Domestically, Trump has “remarkable loyalty” among Republicans, he added. “Internationally, he embodies an aspiration for like-minded politicians and intellectuals.”

In Britain, attitudes toward migrants have hardened in the decade since Brexit, a vote driven in part by hostility toward immigrants from Eastern Europe. Nigel Farage, leader of the hard-right Reform U.K. party, has called unauthorized migration an “invasion” and warned of looming civil disorder.

France’s Marine Le Pen and her father built their political empire on anti-immigrant language decades before Trump entered politics. But the National Rally party has softened its rhetoric to win broader support. Le Pen often casts the issue as an administrative or policy matter.

In fact, what Trump said about people from Somalia would likely be illegal in France if uttered by anyone other than a head of state, because public insults based on a group’s national origin, ethnicity, race or religion are illegal under the country’s hate speech laws. But French law grants heads of state immunity.

One lawyer expressed concerns that Trump’s words will encourage other heads of state to use similar hate speech targeting people as groups.

“Comments saying that a population stinks — coming from a foreign head of state, a top world military and economic power — that’s never happened before,” said Paris lawyer Arié Alimi, who has worked on hate speech cases. “So here we are really crossing a very, very, very important threshold in terms of expressing racist … comments.”

But the “America first” president said he isn’t worried about others think of his increasingly polarizing rhetoric on immigration.

“I hear somebody say, ‘Oh, that’s not politically correct,’” Trump said, winding up his summation Tuesday. “I don’t care. I don’t want them.”

___

Contributing to this report are Associated Press writers Will Weissert and Linley Sanders in Washington, John Leicester in Paris, Jill Lawless in London, Evelyne Musambi in Nairobi, Kenya, and Omar Faruk in Mogadishu.



Source link

Continue Reading

Business

Nearly three-quarters of Trump voters think the cost of living is bad or the worst ever

Published

on



President Donald Trump and his administration insist that costs are coming down, but voters are skeptical, including those who put him back in the White House.

Despite Republicans getting hammered on affordability in off-year elections last month, Trump continues to downplay the issue, contrasting with his message while campaigning last year.

“The word affordability is a con job by the Democrats,” Trump said during a Cabinet meeting on Tuesday. “The word affordability is a Democrat scam.”

But a new Politico poll found that 37% of Americans who voted for him in 2024 believe the cost of living is the worst they can ever remember, and 34% say it’s bad but can think of other times when it was worse.

The White House has said Trump inherited an inflationary economy from President Joe Biden and point to certain essentials that have come down since Trump began his second term, such as gasoline prices.

The poll shows that 57% of Trump voters say Biden still bears full or almost full responsibility for today’s economy. But 25% blame Trump completely or almost completely.

That’s as the annual rate of consumer inflation has steadily picked up since Trump launched his global trade war in April, and grocery prices have gained 1.4% between January and September.

Meanwhile, Vice President JD Vance pleaded for “patience” on the economy last month as Americans want to see prices decline, not just grow at a slower pace.

Even a marginal erosion in Trump’s electoral coalition could tip the scales in next year’s midterm elections, when the president will not be on the ballot to draw supporters.

A soft spot could be Republicans who don’t identify as “MAGA.” Among those particular voters, 29% said Trump has had a chance to change things in the economy but hasn’t taken it versus 11% of MAGA voters who said that.

Across all voters, 45% named groceries as the most challenging things to afford, followed by housing (38%) and health care (34%), according to the Politico poll.

The poll comes as wealthier households are having trouble affording basics, while discount retailers like Walmart and even Dollar Tree are seeing more higher-income customers.

And in a viral Substack post last month, Michael Green, chief strategist and portfolio manager for Simplify Asset Management, argued that the real poverty line should be around $140,000.

“If the crisis threshold—the floor below which families cannot function—is honestly updated to current spending patterns, it lands at $140,000,” he wrote. “What does that tell you about the $31,200 line we still use? It tells you we are measuring starvation.”



Source link

Continue Reading

Trending

Copyright © Miami Select.