Connect with us

Business

UK health service AI tool generated a set of false diagnoses for a patient

Published

on



AI use in healthcare has the potential to save time, money, and lives. But when technology that is known to occasionally lie is introduced into patient care, it also raises serious risks.

One London-based patient recently experienced just how serious those risks can be after receiving a letter inviting him to a diabetic eye screening—a standard annual check-up for people with diabetes in the UK. The problem: He had never been diagnosed with diabetes or shown any signs of the condition.

After opening the appointment letter late one evening, the patient, a healthy man in his mid-20’s, told Fortune he had briefly worried that he had been unknowingly diagnosed with the condition, before concluding the letter must just be an admin error. The next day, at a pre-scheduled routine blood test, a nurse questioned the diagnosis and, when the patient confirmed he wasn’t diabetic, the pair reviewed his medical history.

“He showed me the notes on the system, and they were AI-generated summaries. It was at that point I realized something weird was going on,” the patient, who asked for anonymity to discuss private health information, told Fortune.

After requesting and reviewing his medical records in full, the patient noticed the entry that had introduced the diabetes diagnosis was listed as a summary that had been “generated by Annie AI.” The record appeared around the same time he had attended the hospital for a severe case of tonsillitis. However, the record in question made no mention of tonsillitis. Instead, it said he had presented with chest pain and shortness of breath, attributed to a “likely angina due to coronary artery disease.” In reality, he had none of those symptoms.

The records, which were reviewed by Fortune, also noted the patient had been diagnosed with Type 2 diabetes late last year and was currently on a series of medications. It also included dosage and administration details for the drugs. However, none of these details were accurate, according to the patient and several other medical records reviewed by Fortune.

‘Health Hospital’ in ‘Health City’

Even stranger, the record attributed the address of the medical document it appeared to be processing to a fictitious “Health Hospital” located on “456 Care Road” in “Health City.” The address also included an invented postcode.

A representative for the NHS, Dr. Matthew Noble, told Fortune the GP practice responsible for the oversight employs a “limited use of supervised AI” and the error was a “one-off case of human error.” He said that a medical summariser had initially spotted the mistake in the patient’s record but had been distracted and “inadvertently saved the original version rather than the updated version [they] had been working on.”

However, the fictitious AI-generated record appears to have had downstream consequences, with the patient’s invitation to attend a diabetic eye screening appointment presumedly based on the erroneous summary. 

While most AI tools used in healthcare are monitored by strict human oversight, another NHS worker told Fortune that the leap from the original symptoms—tonsillitis—to what was returned—likely angina due to coronary artery disease—raised alarm bells.

“These human error mistakes are fairly inevitable if you have an AI system producing completely inaccurate summaries,” the NHS employee said. “Many elderly or less literate patients may not even know there was an issue.”

The company behind the technology, Anima Health, did not respond to Fortune’s questions about the issue. However, Dr. Noble said, “Anima is an NHS-approved document management system that assists practice staff in processing incoming documents and actioning any necessary tasks.”

“No documents are ever processed by AI, Anima only suggests codes and a summary to a human reviewer in order to improve safety and efficiency. Each and every document requires review by a human before being actioned and filed,” he added.

AI’s uneasy rollout in the health sector

The incident is somewhat emblematic of the growing pains around AI’s rollout in healthcare. As hospitals and GP practices race to adopt automation tools that promise to ease workloads and reduce costs, they’re also grappling with the challenge of integrating still-maturing technology into high-stakes environments. 

The pressure to innovate and potentially save lives with the technology is high, but so is the need for rigorous oversight, especially as tools once seen as “assistive” begin influencing real patient care.

The company behind the tech, Anima Health, promises healthcare professionals can “save hours per day through automation.” The company offers services including automatically generating “the patient communications, clinical notes, admin requests, and paperwork that doctors deal with daily.”

Anima’s AI tool, Annie, is registered with the UK’s Medicines and Healthcare products Regulatory Agency (MHRA) as a Class I medical device. This means it is regarded as low-risk and designed to assist clinicians, such as examination lights or bandages, rather than automate medical decisions.

AI tools in this category require outputs to be reviewed by a clinician before action is taken or items are entered into the patient record. However, in this case of the misdiagnosed patient, the practice appeared to fail to appropriately address the factual errors before they were added to the patient’s records.

The incident comes amid increased scrutiny within the UK’s health service of the use and categorization of AI technology. Last month, bosses for the health service warned GPs and hospitals that some current uses of AI software could breach data protection rules and put patients at risk.

In an email first reported by Sky News and confirmed by Fortune, NHS England warned that unapproved AI software that breached minimum standards could risk putting patients at harm. The letter specifically addressed the use of Ambient Voice Technology, or “AVT” by some doctors.

The main issue with AI transcribing or summarizing information is the manipulation of the original text, Brendan Delaney, professor of Medical Informatics and Decision Making at Imperial College London and a PT General Practitioner, told Fortune.

“Rather than just simply passively recording, it gives it a medical device purpose,” Delaney said. The recent guidance issued by the NHS, however, has meant that some companies and practices are playing regulatory catch-up. 

“Most of the devices now that were in common use now have a Class One [categorization],” Delaney said. “I know at least one, but probably many others are now scrambling to try and start their Class 2a, because they ought to have that.”

Whether a device should be defined as a Class 2a medical device essentially depends on its intended purpose and the level of clinical risk. Under U.K. medical device rules, if the tool’s output is relied upon to inform care decisions, it could require reclassification as a Class 2a medical device, a category subject to stricter regulatory controls.

Anima Health, along with other UK-based health tech companies, is currently pursuing Class 2a registration.

The U.K.’s AI for health push

The U.K. government is embracing the possibilities of AI in healthcare, hoping it can boost the country’s strained national health system.

In a recent “10-Year Health Plan,” the British government said it aims to make the NHS the most AI-enabled care system in the world, using the tech to reduce admin burden, support preventive care, and empower patients through technology.

But rolling out this technology in a way that meets current rules within the organization is complex. Even the U.K.’s health minister appeared to suggest earlier this year that some doctors may be pushing the limits when it comes to integrating AI technology in patient care.

“I’ve heard anecdotally down the pub, genuinely down the pub, that some clinicians are getting ahead of the game and are already using ambient AI to kind of record notes and things, even where their practice or their trust haven’t yet caught up with them,” Wes Streeting said, in comments reported by Sky News.

“Now, lots of issues there—not encouraging it—but it does tell me that contrary to this, ‘Oh, people don’t want to change, staff are very happy and they are really resistant to change’, it’s the opposite. People are crying out for this stuff,” he added.

AI tech certainly has huge possibilities to dramatically improve speed, accuracy, and access to care, especially in areas like diagnostics, medical recordkeeping, and reaching patients in under-resourced or remote settings. However, walking the line between the tech’s potential and risks is difficult in sectors like healthcare that deal with sensitive data and could cause significant harm.

Reflecting on his experience, the patient told Fortune: “In general, I think we should be using AI tools to support the NHS. It has massive potential to save money and time. However, LLMs are still really experimental, so they should be used with stringent oversight. I would hate this to be used as an excuse to not pursue innovation but instead should be used to highlight where caution and oversight are needed.”



Source link

Continue Reading

Business

Trump finally meets Claudia Sheinbaum face to face at the FIFA World Cup draw

Published

on



Their long-delayed first face-to-face discussion focused on next year’s World Cup — and included side discussions about trade and tariffs — but immigration was not the top issue. That’s despite Trump’s push to crack down on the U.S.-Mexico border being a centerpiece of his administration, and the driving force in the relations between both countries.

Trump has been in office for more than 10 months, and his having taken so long to see Sheinbaum in-person is striking given that meeting with the leader of the country’s southern neighbor is often a top priority for U.S. presidents.

Trump and Sheinbaum had been set to meet in June on the sidelines of the Group of Seven summit in Canada, but that was scrapped after Trump rushed back to Washington early amid rising tensions between Israel and Iran.

Soccer took center stage — but tariffs still loom large

Trump and Sheinbaum sat talking in the president’s box and also appeared onstage with Canadian Prime Minister Mark Carney at the Kennedy Center for Friday’s 2026 World Cup draw. The U.S., Mexico and Canada are co-hosting the tournament, which begins in June.

A senior White House official, who spoke on the condition of anonymity to discuss private meetings, said Trump, Sheinbaum and Carney met privately after participating in the draw.

Sheinbaum had said before leaving Mexico that she’d talk to Trump about tariffs that his administration has imposed on automobiles, steel and aluminum from Mexico, among other things. She said after appearing at the Kennedy Center that the three leaders “talked about the great opportunity that the 2026 FIFA World Cup represents for the three countries and about the good relationship we have.”

“We agreed to continue working together on trade issues with our teams,” Sheinbaum posted on X.

Mexico is the United States’ largest trading partner. The the U.S.-Mexico-Canada Agreement which Trump forged in his first term as a replacement for 1994’s North American Free Trade Agreement also remains in place. But U.S. Trade Representative Jamieson Greer has begun scrutinizing it ahead of a joint review process set for July.

In the meantime, the U.S. and Mexico’s priorities have been reshaped by the steep drop in the number of people crossing into the U.S. illegally along its southern border, as well as the White House’s — so far largely unrealized — threats to impose large trade tariffs on its neighbor.

Before speaking in-person, Trump and Sheinbaum had repeatedly talked by phone, discussing tariffs and Mexican efforts to help combat the trafficking of fentanyl into the U.S. But despite other world leaders, including Russian President Vladimir Putin and Chinese President Xi Jinping, having already met with Trump this term, the meeting with Sheinbaum hadn’t happened until Friday.

The Trump whisperer?

Waiting so long to meet in person hasn’t seemed to hurt Mexico’s president’s standing with Trump.

The two spoke by phone in November 2024, with the then-U.S. president-elect declaring afterward that they’d agreed “to stop Migration through Mexico” — even as Sheinbaum suggested her country had already been doing enough.

Trump soon after taking office threatened to impose a 25% tariff on goods imported from Mexico in an effort to force that country to better combat fentanyl smuggling, only to later agree to a pause.

The White House subsequently backed off tariff threats against most Mexican goods. Then, in October, Sheinbaum announced that the U.S. had given her country another extension to avoid sweeping 25% tariffs on goods it imports to the U.S. — even as many items covered by the USMCA trade deal remain exempt.

Mexico, though, hasn’t avoided all U.S. tariffs. Sheinbaum’s country continues to try to negotiate its way out of import levies Trump has imposed worth 25% on the automotive sector and 50% on steel and aluminum.

Sheinbaum’s success at mitigating many tariffs, and other successes in the bilateral relationship, has led some to wonder if she has a special gift for getting what she wants from him.

She’s largely pulled it off by affording Trump the respect the U.S. president demands from leaders around the world — but especially a neighboring country — and by deploying occasional humor and pushing back, always respectfully, when necessary.

Sheinbaum also defused another potential point of contention, Trump’s renaming of the Gulf of Mexico to the “Gulf of America,” by proposing dryly that North America should be renamed “América Mexicana,” or “Mexican America.” That’s because a founding document dating from 1814 that preceded Mexico’s constitution referred to it that way.

Still, Mexican officials continue to work furiously to lessen the trade blow from tariffs going into 2026 — levies that could wreck its already low-growth economy, particularly in its all-important automotive sector. Sheinbaum’s government has also sought to defend its citizens living in the U.S. as the Trump administration expands its mass deportation operations.

Sheinbaum’s government also lobbied unsuccessfully against a 1% U.S. tax on remittances, or money transfers that millions of Mexicans send home every year from the United States. It was approved as part of Trump’s tax cut and spending package and takes effect Jan. 1.

Trump’s push for mass deportations

Trump has directed federal officials to prioritize major deportation pushes in Democratic-run cities — an extraordinary move that lays bare the politics of the issues. He’s also deployed the National Guard in an effort to curb crime, which has led to a spike in immigration-related arrests, in places like Los Angeles, Chicago and Washington, as well as Memphis, Tennessee, and Portland, Oregon.

The Trump administration says its priority is targeting “the worst of the worst” criminals, but most of the people detained in operations around the country have not had violent criminal histories.

Such operations often meant targeting Mexican citizens who have lived and worked in the United States for years and may face deportation to a homeland they no longer know well. It also has meant serious threats of declining remittance income, which has fallen for seven consecutive months.

The lower number of illegal U.S.-Mexico border crossings has knocked immigration off its perch as the top agenda item for the U.S.-Mexico bilateral relations for the first time in recent memory.

Mexican officials now say conversations around immigration have shifted toward cajoling countries into taking back their citizens and reintegrating them to keep them from leaving again — a major Trump administration priority around the world.

Cooperation on security

Sheinbaum has blunted some of the Trump administration’s tough talk on fentanyl and drug smuggling cartels by giving her security chief Omar García Harfuch more authority.

Mexico has also extradited dozens of drug cartel figures to the U.S., including Rafael Caro Quintero, long sought in the 1985 killing of a DEA agent. That show of goodwill, and a much more visible effort against the cartels’ fentanyl production, has gotten the Trump administration’s attention.

That’s a significant improvement. Only a few years ago, the DEA struggled to get visas for its people in Mexico, and then-President Andrés Manuel López Obrador accused the U.S. government of fabricating evidence against a former Mexican defense secretary, though he never presented evidence to back up the allegation.

Not everything has gone so smoothly, though. Trump criticized Sheinbaum for rejecting his proposal to send U.S. troops to Mexico to help thwart the illegal drug trade.

Last month, Sheinbaum said there was no way the U.S. military would be able to make strikes in Mexico, after Trump said he was open to the idea. And she has denounced U.S. strikes on boats allegedly carrying drugs in the Caribbean and eastern Pacific.

“The president of Mexico is a lovely woman, but she is so afraid of the cartels that she can’t even think straight,” Trump said earlier this year.

Sheinbaum declined to take the bait — and avoided turning up the political pressure — by sidestepping Trump’s criticism.

___

Associated Press writer Chris Sherman contributed from Mexico City.



Source link

Continue Reading

Business

Former Amazon exec warns Netflix-WBD deal will make Hollywood ‘a system that circles a single sun’

Published

on



A Netflix-Warner Bros. merger would risk a monopsony where a single buyer wields enormous control over the marketplace, the former head of Amazon Studios warned.

Roy Price, who is now chief executive of the studio International Art Machine, wrote in a New York Times op-ed on Saturday that predictions of doom are nothing new in the film industry, pointing to the advent of TV, home video, streaming, and AI.

“But if Netflix acquires Warner Bros., this long-prophesied death may finally arrive, not in the sense that filmmaking will cease but in the sense that Hollywood will become a system that circles a single sun, materially changing its cultural output,” he added. “All orbits—every deal, every creative decision, every creative career—will increasingly revolve around the gravitational mass and imprimatur of one entity.”

To be sure, Netflix has said Warner Bros. operations will continue, and the studio’s films will still be released in theaters. Meanwhile, Warner’s TV channels will be spun off via a separate company, though HBO will be included in Netflix.

But Price said the danger “is not annihilation but centralization,” with the combined company accounting for an even bigger slice of overall content spending.

A reduction in bidders also means less content will be produced, while a separate development culture, set of tastes, and risk tolerances will be sidelined, he predicted.

“A Netflix merger with Warner Bros. would create a monopsony problem: too few buyers with too much bargaining power,” Price explained. “Writers, directors, actors, showrunners, puppeteers, visual effects artists—all are suppliers. The fewer buyers competing to hire them, the lower their compensation and the narrower their opportunities.”

Such reasoning sank Penguin Random House’s attempt to merge with Simon & Schuster that would’ve created a book publisher with too much leverage over authors, he pointed out.

Of course, the remaining players in Hollywood and content creation are giants in their own right as well. A KPMG survey of spending in 2024 put NBC Universal parent Comcast at the top with $37 billion, followed by Alphabet’s YouTube ($32 billion), Disney ($28 billion), Amazon ($20 billion), Netflix ($17 billion) and Paramount ($15 billion). Comcast and Paramount also made bids for Warner Bros.

Theater owners, producers and other creative workers have also voiced opposition to the deal. In addition to the business impact of a Warner Bros. takeover, other opponents raised even weightier concerns.

Oscar winner Jane Fonda sounded the alarm on a “constitutional crisis” and demanded that the Justice Department not use its regulatory power to “extract political concessions that influence content decisions or chill free speech.”

For its part, the Trump administration views the deal with “heavy skepticism,” sources told CNBC. The merger is expected to face exceptional antitrust scrutiny, and Netflix’s $5.8 billion breakup fee is among the biggest ever.

On Wall Street, analysts see a tech angle in the merger, namely the importance of content to train and power the next generation of AI models that will shape the entertainment industry’s future.

The acquisition of Warner Bros. would help Netflix stand out in an AI future, Divyaunsh Divatia, research analyst at Janus Henderson Investors, said in a note on Friday.

“They’re also levering up on premium entertainment at a time when competition on engagement from short form video is expected to intensify especially if AI models democratize video creation at an increasing rate,” he wrote.



Source link

Continue Reading

Business

25-year DEA veteran charged with helping Mexican drug cartel launder millions of dollars, secure guns and bombs

Published

on



A former high-level agent with the U.S. Drug Enforcement Administration and an associate have been charged with conspiring to launder millions of dollars and obtain military-grade firearms and explosives for a Mexican drug cartel, according to an indictment unsealed Friday in New York.

Paul Campo, 61, of Oakton, Virginia, who retired from the DEA in 2016 after a 25-year career, and Robert Sensi, 75, of Boca Raton, Florida, were caught in sting involving a law enforcement informant who posed as a member of the Jalisco New Generation Cartel, prosecutors said.

The cartel, also know as CJNG, was designated as a foreign terrorist organization by the U.S. in February.

U.S. Attorney Jay Clayton said Campo betrayed his DEA career by helping the cartel, which he said was responsible for “countless deaths through violence and drug trafficking in the United States and Mexico.”

Campo and Sensi appeared Friday afternoon before a magistrate judge in New York, who ordered them detained without bail. Their lawyers entered not guilty pleas on their behalf.

Campo’s lawyer, Mark Gombiner, called the indictment “somewhat sensationalized and somewhat incoherent.” He denied the two men had agreed to explore obtaining weapons for the cartel.

Prosecutors say pair talked of laundering money, obtaining weapons

Over the past year, Campo and Sensi agreed to launder about $12 million in drug proceeds for the cartel and converted about $750,000 in cash to cryptocurrency, thinking it was going to the group when it really went to the U.S. government, the indictment said. They also provided a payment for about 220 kilograms of cocaine they were told would be sold in the U.S. for about $5 million, thinking they would get a cut of the proceeds, prosecutors said.

The two men also said they would look into procuring commercial drones, AR-15 semiautomatic rifles, M4 carbines, grenade launchers and rocket-propelled grenades for the cartel, the indictment said.

Campo boasted about his law enforcement experience during conversations with the informant and offered to be a “strategist” for the cartel, authorities said. He began his career as a DEA agent in New York and rose to become deputy chief of financial operations for the agency, the indictment said.

Evidence in the case includes hours of recordings of the two men talking with the informant, as well as cellphone location data, emails and surveillance images, Assistant U.S. Attorney Varun Gumaste said in court Friday.

Sensi’s attorney, Amanda Kramer, unsuccessfully argued that Sensi should be freed while he awaits trial, saying he wouldn’t flee partly because he has multiple health problems, including injuries from a fall two months ago, early-stage dementia and Type II diabetes.

Sensi was convicted in the late 1980s and early 1990s of mail fraud, defrauding the government and stealing $2.5 million, said the prosecutor, Gumaste. He said evidence shows Sensi also was engaged in a scheme to procure military-grade helicopters for a Middle East country.

Other criminal cases have roiled the DEA

DEA Administrator Terrance Cole said in a statement that while Campo is no longer employed by the DEA, the allegations undermine trust in law enforcement.

The DEA has been roiled in recent years by several embarrassing instances of misconduct in its ranks. The Associated Press has tallied at least 16 agents over the past decade brought up on federal charges ranging from child pornography and drug trafficking to leaking intelligence to defense attorneys and selling firearms to cartel associates, revealing gaping holes in the agency’s supervision.

Starting in 2021, the agency placed new controls on how DEA funds can be used in money laundering stings, and warned agents they can now be fired for a first offense of misconduct if serious enough, a departure from prior administrations.

Campo and Sensi are charged with four conspiracy counts related to narcoterrorism, terrorism, narcotics distribution and money laundering.

____

Collins reported from Hartford, Connecticut. Associated Press writer Joshua Goodman in Miami contributed to this report.



Source link

Continue Reading

Trending

Copyright © Miami Select.