Rebecca Johnson, PhD, Clinical Research Recruitment and Inclusion Executive and Strategist

Efforts to make sustainable change toward more inclusive clinical research have gained momentum in recent years. The enactment of legislation through the Food and Drug Omnibus Reform Act of 2022 (FDORA), which requires sponsors to submit diversity action plans (DAPs) for pivotal studies, highlights this commitment.1 Industry sponsors are making concerted efforts to ensure trial populations reflect the diversity of the populations they aim to treat, ensuring research is more inclusive and accessible.2 In addition to race and ethnicity, clinical trial diversity efforts are now increasingly focusing on other factors that characterize populations often underrepresented in clinical research, such as sexual orientation, gender identity, socioeconomic status, and age, among others.3

However, the recent executive orders (EOs) by the Trump administration banning all US diversity, equity, and inclusion (DEI) and gender identity initiatives in the federal government have consequently impacted clinical trial diversity, putting in jeopardy programs designed to promote equitable access, particularly for LGBTQIA+ populations and other groups underrepresented in research.4-8 This development raises critical questions about what clinical trial diversity-related initiatives and programs are still “allowable” under the new policies.

If clinical trial diversity continues to be caught in the crossfire of the DEI ban, it could have significant repercussions for drug development.

Consequences for R&D

It is well documented that the failure to achieve appropriate representation in clinical trials limits the generalizability of results and hinders understanding of drug response variability across different subgroups.4 Without inclusive recruitment, sponsors risk delays and higher costs—either through slowing enrollment or the need for postmarketing studies.9,10 Public trust in an approved treatment, among both physicians and patients, is crucial for widespread adoption. For example, physicians treating Black patients are less likely to prescribe drugs approved based on unrepresentative trials, while Black patients are more likely to trust a drug that was studied in a trial representative of participants in their racial group.11

Specifics of executive orders

The EOs titled “Ending Illegal Discrimination and Restoring Merit-Based Opportunity” and “Ending Radical and Wasteful Government DEI Programs and Preferences” mandate the termination of all “diversity,” “advancing equity,” and “diversity, equity, inclusion, and accessibility” principles, programs, positions, and workforce balancing in federal agencies, contractors, and grantees.6,7 These EOs also seek to deter DEI programs within the private sector.6 The Department of Justice will “investigate, eliminate, and penalize illegal DEI (and accessibility) preferences, mandates, policies, programs, and activities in the private sector.”12

The EO titled “Defending Women from Gender Ideology Extremism and Restoring Biological Truth to the Federal Government” prohibits the recognition of gender identity apart from biological sex.8

While specifics related to what constitutes a DEI program are not outlined in the EOs, Diversio, in a report released in February, conducted an analysis of the recent directives published by the administration and deemed the following practices, among others, as “illegal DEI”:

Race-based or gender-based quotas for hiring or promotions. 13

Training or messaging that focuses on addressing privilege, implicit bias, systemic inequalities, or oppression based on race or gender.13

The Trump administration’s actions have led to a banned word list, including terms such as “disability,” “inclusion,” “excluded,” and “racism.”14

Multi-stakeholder perspectives

The clinical trial ecosystem includes various stakeholders, such as community members, clinical research sites, contract research organizations, academia, advocacy organizations, government agencies, and sponsors. If executive actions continue as currently outlined, government contractors and the private sector will be impacted, with many stakeholders at risk of violating federal laws if their DEI practices conflict with new policies. Implications of the DEI ban are explored ahead from a multi-stakeholder perspective across the clinical trial ecosystem.

Patients: Will efforts to build trust in the medical system be disrupted?

Mistrust in the medical community is a prominent barrier to clinical research participation. Clinical research stakeholders have made significant strides to help build trust, with efforts such as establishing long-term community relationships and providing summary information after a trial. The hyper-focus on anti-DEI initiatives and publicity surrounding anti-immigration and gender identity sentiments threatens to undermine trust, especially among Hispanic and sexual and gender diverse (SGD) populations.15

Paul Evans, CEO of Velocity Clinical Research, an integrated site organization, commented, “Velocity believes that clinical trials should be a safe space for the entire LGBT community. Right now, there seems to be a heightened focus on the ‘T’ part, and I am concerned about how an emphasis on ‘transgender’ could potentially create a negative perception. It’s important that all members of the LGBT community feel welcome, regardless of their specific identities. If participants feel excluded or unwelcome, it could affect the broader community’s trust. We should ensure that Velocity remains a place where everyone, from all parts of the LGBT spectrum, feels respected and included.”

It appears that, for patients, pharmaceutical companies’ efforts to publicly share information about employing diverse staff are important. A 2023 study by the Center for Information and Study on Clinical Research Participation (CISCRP) found that diverse staffing in pharma companies boosts trust among Black and Hispanic patients.16 Similarly, patients trust providers who “look like me.”17

As sponsors and health systems scale back their diversity hiring practices, they should be mindful that transparency and representativeness mean a great deal to patients.

Healthcare providers: What is the outlook for future researchers and the opportunity to reach patients for trial participation?

After declaring racism a public health emergency in 2020, health systems announced DEI and health equity initiatives to improve access to care and employ a workforce that better reflects community demographics.18,19 However, backlash against these initiatives is growing. For instance, Cleveland Clinic faced a civil rights complaint in 2024 over services tailored for Black and Latino patients.20

Health systems and medical colleges, published reports note, appear cautious and have refrained from commenting on their future DEI plans.19

DEI programs, such as those at Mount Sinai, support many populations often underrepresented in the healthcare system, including veterans, disabled patients, and those who identify as SGD.21 The ramifications of the executive orders on these programs are unclear.22

The cuts by the National Institutes of Health (NIH) to indirect expenses provided to research grants will impact academic medical centers that rely on these funds to support their research infrastructure.23

According to the Associated Press, federal funds supply the largest source of research dollars for Duke University, and these cuts could hinder research as well as the pipeline for the next generation of researchers. Findings published in journals such as The American Journal of Bioethics and PLOS Global Public Health note that Black, Hispanic, and Indigenous physicians are already underrepresented not only in the healthcare workforce but also as researchers.24,25 Anti-DEI efforts could further widen this gap.25

Stymieing programs that promote access to care and a diversified pipeline of future researchers will directly impact clinical research.

Research sites: Can they uphold community engagement and staffing?

Effective inclusive recruitment practices reported by research sites include hiring staff who reflect the communities they partner with and who speak the same language.26 In fact, the diversity of patients enrolled at a site is proportionate to the diversity of the site personnel.27 Many research sites, such as academic medical centers, rely on research grants, and the reduction in funding may have a negative effect on staffing as well as community engagement programs that are critical for raising awareness and building trusting relationships.

The outlook may be more positive for private sites and private site organizations. At Velocity Clinical Research, Evans said it uses a three-pronged approach: recruiting diverse staff representative of their patient population; ensuring access (they even physically moved some sites) and actively engaging communities.

“Our staff reflects the population and we track metrics,” explained Evans. “Staff at Velocity are representative of the US Census and our sites are representative. We are going to keep carrying on doing that. The staff has to be representative of the population they are recruiting. I think everyone gets hung up on affirmative action.

“My grandmother always told me some things are just right and wrong,” he continued. “All we’ve ever tried to do is make sure we are a welcoming company and we recruit widely. That happens to be right. I’ve never set a quota in my life.”

Training research staff is an important component for cross-cultural recruitment.28 Study teams may lose important training on addressing the historical legacy of injustice and systemic racism, which contribute to institutional distrust among many underrepresented populations. Investigators and healthcare professionals have expressed a need for improved skills in cross-cultural communication across the many dimensions of diversity.28,29 Important components of such training include topics such as biases, stigmatization, and discrimination—themes that are now prohibited under the new policies.29

Payers: Coverage decisions remain the same, but barriers may persist

Managed care experts have indicated that payers are unlikely to change the way they review clinical trial data for their formulary decision-making. Payers need to see efficacy in populations that match the disease burden when formulary decisions are evaluated. If trial participants do not reflect a payer’s membership demographics, coverage may be restricted to the demographics represented in the study, potentially limiting access to effective treatments and disenfranchising individuals who could benefit from it.

Payers are scaling back their DEI initiatives in other areas. According to Modern Healthcare, 2024 annual reports from the largest health insurers show changes to their DEI strategies.30 While some organizations have stated they are not altering their DEI policies, references to DEI have been scaled back and replaced with more “neutral” language in their reports.30

While these changes seem to be limited to workplace DEI initiatives, they have a ripple effect. For example, as Modern Healthcare notes,Cigna removed the following statement from its report: “We also help to eliminate barriers to care and address other factors that contribute to health disparities.”30 If programs aimed at improving access to care are deprioritized, it could result in fewer visits to physicians, limiting opportunities for patients to learn about clinical trials—thus exacerbating existing barriers to clinical trial diversity.

Sponsors: Balancing compliance with diversity and inclusion in trial goals

Since the COVID-19 pandemic and the heightened focus on ensuring representation of historically underrepresented populations in COVID vaccine trials, pharmaceutical companies have expanded their efforts to ensure diverse representation in their clinical trials. Eli Lilly, for example, announced plans to shift toward predominantly conducting community-oriented research.31 Through another initiative called Equitable Breakthroughs in Medicine Development, which is funded by a grant from Pharmaceutical Research and Manufacturers of America, sponsor companies are working with community sites and partners to build an infrastructure of community-based sites.32

With the changing legislative requirements, many pharmaceutical companies are adjusting their policies and messaging regarding DEI initiatives. In its annual report, GSK stated, “Going forward, we will make changes in several areas related to inclusion and diversity to ensure continued compliance with the law and being respectful of our operating environment, including no longer setting aspirational targets for our leadership.”33 Pfizer updated its DEI webpage to reflect a commitment to merit-based hiring practices while ensuring it recruits the best talent from all backgrounds.34

When asked if clinical trial diversity efforts would be impacted by the regulatory changes, a VP of clinical operations for a biopharmaceutical company shared the following regarding its research that relies on public-private partnerships, “I do. Mainly because academic institutions will likely be impacted, which will increase our cost of doing business. Part of the reason is the proposed cut of indirect costs. High costs of specialty vendors that may specialize in recruitment of underrepresented people will have a downstream impact.”

Despite this, industry experts believe pharma companies are continuing their efforts to achieve more representative trials. “We are still writing the DAP, even though it’s not mandated right now. We are still staying the course,” the VP of clinical operations mentioned.

It is widely agreed that trial populations need to reflect the epidemiology of the disease being studied. “Variability of drug response is what we’re going after—assume there is a difference until proven otherwise,” noted the pharma executive. “The social issue is a construct people have made and have added complexity to this topic.”

Among pharma sponsors, the focus on ensuring representative trial populations is expected to continue despite the new regulations.

Healthcare system: Is there enough incentivization to continue public health initiatives?

A visible shift in research priorities is evident, with funding for health equity and diversity and inclusion programs cut across numerous federal agencies.5,14 The NIH All of Us Research Program plays a key role in advancing public health research and rebuilding trust in science and research. Yet, funding cuts necessitate the need to revise program priorities such as engaging rural populations.35 These cuts will reduce efforts to engage historically underrepresented communities and could hinder progress in research diversity.

One goal of Healthy People 2030 is to eliminate health disparities, achieve health equity, and attain health literacy to improve the health and well-being of all.36 Another goal is to improve the health, safety, and well-being of lesbian, gay, bisexual, and transgender people. However, a disclaimer now appears on the Healthy People webpage indicating that the content has been rejected.37 Given that language previously considered acceptable is now deemed “illegal,” it raises concerns about which public health initiatives will remain intact. These public health initiatives play a critical role in building trust in the healthcare system, which is essential for fostering participation in clinical research.

In 2018, racial and ethnic health disparities reportedly cost the US economy $451 billion.5 Findings from a recent study by the National Academies of Sciences, Engineering, and Medicine estimated that achieving even modest diversity in research could reduce health disparities, resulting in billions of dollars in savings to the US economy.38

If public health initiatives are scaled back, reducing health disparities may become more difficult, and clinical trials may become less representative, undermining progress toward equity.

Policy: Navigating conflicting messaging and understanding regulatory expectations

Clinical trial regulators must address conflicting mandates and clarify expectations.

The FDA’s stance on DEI in clinical trials remains supportive, as evidenced by the agency’s commissioner nominee Marty Makary’s confirmation hearing comments: “I believe in common sense, and I believe in clinical trial diversity,” he said. “If you’re going to make results extrapolated to the general population, you should have results in those populations that you’re making recommendations for.”39 However, Makary did not commit to restoring the FDA’s Office of Minority Health and Health Equity website.39

FDA recently removed and re-added DAP guidance with a disclaimer rejecting content on gender identity, citing “biological reality.” DAPs are a statutory requirement under FDORA, and final guidance is expected by June 2025. It remains unclear whether the EOs will impact clinical trial diversity mandates under FDORA.

The FDA’s draft guidance recommending that sponsors investigate potential sex-specific and gender-specific differences for the intended effect of medical devices, released in January 2025, was also removed.40 The guidance was set forth because of the lack of data on underrepresentation of nonbinary, transgender, and other gender identities as “gender was often conflated with sex or otherwise not properly reported in clinical studies.”41

The ambiguity surrounding these issues raises concerns in the clinical research community. The biopharma company VP of clinical operations I spoke with stated, “People are still in shock, not knowing what will happen next. DEI efforts taken in the course of clinical trials may have been taken out of context by the administration. There will likely be a course correction.”

Reframing the conversation

In the meantime, the industry should continue moving forward. Efforts to diversify clinical trials have grown significantly, even without FDA mandates. The clinical trial ecosystem has taken responsibility for improving representation, and made themselves accountable, but clear guidance is needed to align expectations. Based on the current regulatory landscape, here are suggestions for staying the course and reframing the conversation:

Use terms such as “comprehensive representation” or “representation in research” instead of DEI. Focus on population-reflective studies and set goals based on disease epidemiology.

Revise training programs to emphasize cross-cultural communication, cultural awareness, and the history of consent failures in clinical research, with careful attention to language.

Be transparent about research objectives and clearly communicate how each individual’s involvement contributes to the success and impact of the study, emphasizing the value of participation.

Develop new metrics to explore other dimensions of diversity. When appropriate, consider social determinants of health variables, which may have a more significant impact on outcomes than using race or ethnicity as proxies. 42

Leverage artificial intelligence (AI) for participant selection and the generation of representative trial data. The growing use of AI in clinical research can improve efficiency and accelerate drug development, offering novel approaches to ensure trials more accurately reflect the intended population.43

Advancing and adjusting

Despite the setbacks, stakeholders across the clinical trial ecosystem are dedicated to advancing diversity in drug development, and will adapt as necessary to continue making headway. By reframing the conversation as needed, clinical research professionals can maintain their momentum and perhaps achieve even more comprehensive representation than before. While the terminology may change, our collective resilience will ensure that the goal of accessible and inclusive clinical research remains within reach, no matter what we call it.

Rebecca Johnson, PhD, is a clinical research recruitment and inclusion executive and strategist. In her most recent role, Johnson led the clinical trial recruitment and diversity, equity, and inclusion functions at SPARC. Prior to that, she held several leadership positions within IQVIA’s patient recruitment team, including advising pharmaceutical sponsor teams with their efforts to achieve more diverse trial populations. Johnson has a Master’s in Public Health and a PhD in Health Sciences. Her doctoral dissertation research focused on increasing access to clinical trials for historically underrepresented populations.

References

1. Consolidated Appropriations Act, Food and Drug Omnibus Reform Act, Public Law No 117-328, H.R. 2617, Subtitle F, § 3601 et seq. (2022). https://www.congress.gov/117/plaws/publ328/PLAW-117publ328.pdf

2. Rottas, M., Thadeio, P., Simons, R., Houck, R., Gruben, D., Keller, D., Scholfield, D., Soma, K., Corrigan, B., Schettino, A., McCann, P.J., Hellio, M.P., Natarajan, K., Goodwin, R., Sewards, J., Honig, P., and MacKenzie, R. (2021). Demographic diversity of participants in Pfizer sponsored clinical trials in the United States. Contemporary Clinical Trials, 106, 106421. https://doi.org/10.1016/j.cct.2021.106421

3. Versavel, S., Subasinghe, A., Johnson, K., Golonski, N., Muhlhausen, J., Perry, P., and Sanchez, R. (2023). Diversity, equity and inclusion in clinical trials: A practical guide from the perspective of a trial sponsor. Contemporary Clinical Trials, 126, 107092. https://doi.org/10.1016/j.cct.2023.107092

4. Food and Drug Administration. (2024, June). Diversity Action Plans to improve enrollment of participants from underrepresented populations in clinical studies: Guidance for industry. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/diversity-action-plans-improve-enrollment-participants-underrepresented-populations-clinical-studies

5. Melville, D. (2025, January 29). Could cutting DEI in medicine cost lives? The high-stakes impact of Trump’s Executive Order. Forbes. https://www.forbes.com/sites/dougmelville/2025/01/29/could-cutting-dei-in-medicine-cost-lives-the-high-stakes-impact-of-trumps-executive-order/

6. Exec. Order No. 14173, 3 C.F.R. 8633-8636 (2025). https://www.govinfo.gov/content/pkg/FR-2025-01-31/pdf/2025-02097.pdf

7. Exec. Order No. 14151, 3 C.F.R. 8339-8341 (2025). https://www.govinfo.gov/content/pkg/FR-2025-01-29/pdf/2025-01953.pdf

8. Exec. Order No. 14168, 3 C.F.R. 8615-8618 (2025). https://www.govinfo.gov/content/pkg/FR-2025-01-30/pdf/2025-02090.pdf

9. Cohen, E., and Vigue, D. (2020, August 16). Covid-19 vaccine trials have been slow to recruit Black and Latino people – and that could delay a vaccine. CNNhealth. https://www.cnn.com/2020/08/16/health/covid-19-vaccine-trial-black-minority-recruitment/index.html

10. Food and Drug Administration. (2011, April). Guidance for industry: Postmarketing studies and clinical trials – implementation of Section 505(o)(3) of the Federal Food, Drug, and Cosmetic Act. https://www.fda.gov/media/131980/download

11. Alsan, M., Durvasula, M., Gupta, H., Schwartzstein, J., and Williams, H. (2024). Representation and extrapolation: Evidence from clinical trials. The Quarterly Journal of Economics, 139(1),575-635. https://doi.org/10.1093/qje/qjad036

12. Turnbull, A. (2025, February 19). Unpacking the Trump administration’s DEI orders and actions – FAQs and action plans. Morrison Foerster. https://www.mofo.com/resources/insights/250219-unpacking-the-trump-administration-s-dei-orders

13. McGee, L. (2025, February 15). The political & legal landscape of DEI: Navigating changes in 2025. Diversio. https://diversio.com/the-political-legal-landscape-of-dei-navigating-changes-in-2025/

14. Davidson, J. (2025, February 16). Explaining diversity, equity, inclusion, & accessibility (DEIA), the Trump administration’s recent actions on DEIA, and the impact on disabled Americans. AAPD. https://www.aapd.com/explaining-deia-recent-actions/

15. Bravo, L. (2024, December 4). More DEI bans will have dire side effects for public health – clinician diversity may be at risk with the new administration. MEDPAGETODAY. https://www.medpagetoday.com/opinion/second-opinions/113205

16. Center for Information & Study on Clinical Research Participation (CISCRP). (2023). 2023 perceptions and insights study. https://www.ciscrp.org/services/research-services/perceptions-and-insights-study/

17. Moore, C., Coates, E., Watson, A., de Heer, R., McLeod, A., and Prudhomme, A. (2023). Representation and extrapolation: Evidence from clinical trials. Journal of Racial and Ethnic Health Disparities, 10(5),2552-2564. 10.1007/s40615-022-01435-y

18. Hartnett, K. (2023, November 13). Anti-‘woke’ backlash forces health industry to adapt DEI efforts. Modern Healthcare. https://www.modernhealthcare.com/hospital-systems/healthcare-dei-anti-woke-law-ron-desantis-trinity-health

19. Desilva, H. (2025, January 30). CommonSpirit to expand DEI programs despite federal pushback. Modern Healthcare. https://www.modernhealthcare.com/providers/commonspirit-dei-program-extensions-morehouse

20. Devereaux, M. (2024, August 27). Healthcare leaders navigate pushback to health equity programs. Modern Healthcare. https://www.modernhealthcare.com/providers/health-equity-programs-cleveland-clinic-complaint

21. Mount Sinai. (2025). Office for diversity and inclusion. https://www.mountsinai.org/about/odi

22. McGrath, C. (2025, February 27). How DEI rollbacks are affecting healthcare. Healthcare Brew. https://www.healthcare-brew.com/stories/2025/02/27/how-dei-rollbacks-are-affecting-healthcare

23. Seminera, M. (2025, March 8). Universities are facing big cuts to research funding. At Duke, it’s a time for ‘damage control’. Associated Press. https://apnews.com/article/trump-cuts-research-funding-nih-duke-7f24b33bbad54490583520536ab40e0c

24. Sierra-Mercado, D., & Lázaro-Muñoz, G. (2018). Enhance diversity among researchers to promote participant trust in precision medicine research. The American Journal of Bioethics, 18(4), 44-46. 10.1080/15265161.2018.1431323

25. Blackstock O., Isom J., & Legha R. (2024) Health care is the new battlefront for anti-DEI attacks. PLOS Global Public Health 4(4): e0003131. https://doi.org/10.1371/journal.pgph.0003131

26. Johnson R., D’Abundo, M., Cahill, T., & DeLuca, D. (2023). Understanding organizational perspectives from clinical research stakeholders involved in recruitment for biopharmaceutical-sponsored clinical trials in the United States: Recommendations for organizational initiatives to improve access and inclusivity in clinical research. Contemporary Clinical Trials Communications. 18(33):e101148. https://doi.org/10.1016/j.conctc.2023.101148

27. Tufts Center for the Study of Drug Development. (2021). New study finds site personnel race and ethnicity highly correlated with diversity of patients enrolled [Impact Report]. Tufts University. 23(6) 1-4.

28. Boden-Albala, B., Carman, H., Southwick, L., Parikh, N., Roberts, E., Waddy, S., & Edwards, D. (2015). Examining barriers and practices to recruitment and retention in stroke clinical trials. Stroke, 46(8), 2232-2237.

29. Yu, H., Flores, D.D., Bonett, S., & Bauermeister, J. (2023). LGBTQ + cultural competency training for health professionals: a systematic review. BMC Medical Education, 23:558. https://doi.org/10.1186/s12909-023-04373-3

30. Berryman, L. (2025, March 3). Inside insurer annual reports: DEI is out, publicity worries are in. Modern Healthcare. https://www.modernhealthcare.com/insurance/unitedhealth-cigna-humana-elevance-dei?utm_source=modern-healthcare-

31. Masson, G. (2024, March 20). Are community-based trials the wave of the future? Eli Lilly thinks so. Fierce Biotech. https://www.fiercebiotech.com/cro/are-community-based-trials-wave-future-eli-lilly-thinks-so

32. Equitable Breakthroughs in Medicine Development (EQBMED). (2024, September 12). Equitable breakthroughs in medicine development (EQBMED) to work with inaugural sponsor companies to drive greater diversity in clinical trials. PR Newswire. https://www.prnewswire.com/news-releases/equitable-breakthroughs-in-medicine-development-eqbmed-to-work-with-inaugural-sponsor-companies-to-drive-greater-diversity-in-clinical-trials-302246462.html

33. GSK. (2024). GSK Responsible Business Performance Report 2024. https://www.gsk.com/media/11863/responsible-business-performance-report-2024.pdf

34. Chen, E., & Parker, J. (2025, February 27). Pfizer revises its DEI webpage to emphasize the importance of ‘merit’. STAT+. https://www.statnews.com/2025/02/27/pfizer-dei-diversity-equity-inclusion-webpage/

35. National Institutes of Health, All of Us research program. (2024, December 18). From the All of Us CEO: Program update. https://allofus.nih.gov/article/announcement-from-all-of-us-ceo-program-update

36. United States Department of Health and Human Services (USHHS), Office of Disease Prevention and Health Promotion, Healthy People 2030. (n.d.). Healthy People 2030 Framework. https://odphp.health.gov/healthypeople/about/healthy-people-2030-framework

37. United States Department of Health and Human Services (USHHS), Office of Disease Prevention and Health Promotion, Healthy People 2030. (n.d.). LGBT.

https://odphp.health.gov/healthypeople/objectives-and-data/browse-objectives/lgbt

38. National Academies of Sciences, Engineering, and Medicine. (2022). Improving representation in clinical trials and research: Building research equity for women and underrepresented groups. Washington, DC: The National Academies Press. https://doi.org/10.17226/26479

39. Lawrence, L., Owermohle, S., Chen, E., Herper, M., & Todd, S. (2025, March 6). FDA nominee Marty Makary questioned on vaccines, measles, layoffs at confirmation hearing. STAT+. https://www.statnews.com/2025/03/06/marty-makary-confirmation-hearing-fda-commissioner-live-coverage/

40. Crowell & Moring LLP. (2025, January 27). After Trump executive orders, FDA removes diversity guidance from website. Crowell & Moring LLP: Client Alert. https://www.crowell.com/en/insights/client-alerts/after-trump-executive-orders-fda-removes-diversity-guidance-from-website

41. United States Department of Health and Human Services (USHHS), Food and Drug Administration. Evaluation of sex-specific and gender-specific data in medical device clinical studies; Draft guidance for industry and Food and Drug administration staff. Federal Register. 90(4) 1161-1162. https://www.govinfo.gov/content/pkg/FR-2025-01-07/pdf/2024-31526.pdf

42. National Academies of Sciences, Engineering, and Medicine. (2024). Rethinking race and ethnicity in biomedical research. Washington, DC: The National Academies Press. https://doi.org/10.17226/27913

43. Beigi, M., Shafquat, A., Chen, J., Aptekar, J., & Itzkovich, Y. (2024, December 10). Accelerating breakthroughs with synthetic clinical trial data. Applied Clinical Trials. 33(12). https://www.appliedclinicaltrialsonline.com/view/accelerating-breakthroughs-with-synthetic-clinical-trial-data

Generative artificial intelligence (AI) has recently attracted widespread attention across various fields, including the GPT1,2 and LLaMA3,4 series for text generation, DALL-E5 for image generation, as well as Sora6 for video generation. In health care systems, generative AI holds promise for applications in consulting, diagnosis, treatment, management, and education7,8. Additionally, the utilization of generative AI could enhance the quality of health services for patients while alleviating the workload for clinicians8,9,10.

Despite this, it is crucial to acknowledge the inherent limitations of generative AI models, which include susceptibility to biases from pre-training data11, lack of transparency, the potential to generate incorrect content, and difficulty in maintaining up-to-date knowledge, among others7. For instance, large language models (LLMs) were shown to generate biased responses by adopting outdated race-based equations to estimate renal function12. In the process of image generation, biases related to gender, skin tone, and geo-cultural factors have been observed13. Similarly, for downstream tasks such as question answering, LLM-generated content is often factually inconsistent and lacks evidence for verification14. Moreover, due to their static knowledge and inability to access external data, generative AI models are unable to provide up-to-date clinical advice for physicians or effective personalized health management for patients15.

In tackling these challenges, retrieval-augmented generation (RAG) has been explored as a potential solution16,17. By providing models with externally retrieved data, RAG can enhance the reliability of generated content. A typical RAG framework consists of three parts (Fig. 1): indexing, retrieval, and generation. In the indexing stage, external data is split into chunks, encoded into vectors, and stored in a vector database. In the retrieval stage, the user’s query is encoded into a vector representation, and then the most relevant information is retrieved through similarity calculations between the query and the information in the vector database. The retrieval techniques are not limited to dense retrieval but also include sparse and hybrid retrieval approaches, and advanced reranking methods can be employed to improve the relevance of retrieved content. In the generation stage, both the user’s query and the retrieved relevant information are prompted by the model to generate content. Compared to model fine-tuning for a specific task, RAG is generally more computationally efficient and has been shown to improve accuracy for knowledge-intensive tasks18, offering a more flexible paradigm for model updates and integration with other AI techniques.

Fig. 1: A typical retrieval-augmented generation framework. External data is first encoded into vectors and stored in the vector database (where vectors are mathematical representations of various types of data in a high-dimensional space). In the retrieval stage, when receiving a user query, the retriever searches for the most relevant information from the vector database. In the generation stage, both the user’s query and the retrieved information are used to prompt the model to generate content. Full size image

In this perspective, we discuss the role of RAG in the context of generative AI, particularly its possible applications within health care. We examine RAG’s possible contributions from three aspects: equity, reliability, and personalization (Fig. 2). Additionally, we explore the limitations of RAG in medical application scenarios, emphasizing the need for further research to understand the impact of its implementation within health systems19.

The AI buzz is alive and well, with the technology expected to remain integral to health systems’ IT plans in 2025. As health systems strategize about how best to identify and integrate AI tools, they must ensure that their clinical staff are on board. Without support from these critical stakeholders, AI implementation efforts could be dead in the water.

A recent AMA survey revealed that physician enthusiasm about health AI is on the rise overall. Among 1,183 physicians polled, the share of physicians whose enthusiasm exceeded their concerns with AI increased from 30% in 2023 to 35% in 2024. However, most physicians noted that a designated feedback channel (88%) and data privacy assurances (87%) are critical to facilitating AI adoption among clinical teams.

Provider trust in AI is also critical because provider support is closely tied to patient trust. A study published in 2023 showed that though patients were almost evenly split when asked whether they would prefer a human clinician or an AI tool, they were significantly more likely to accept the use of the AI tool if they were told their provider supported the tool’s use and felt it was helpful.

AI adoption and utilization begin with clinical teams. Leaders from some of the country’s top health systems told Healthtech Analytics about their clinicians’ most pressing concerns with AI and how they mitigated them to ensure successful AI implementations.

Health systems grow AI efforts, but clinician concerns linger Though clinician concerns regarding AI use in healthcare vary, one common concern is AI model accuracy. When Vanderbilt University Medical Center began implementing AI tools, clinicians had a host of questions for leadership, said Adam Wright, PhD, professor of biomedical informatics and medicine at VUMC and director of the Vanderbilt Clinical Informatics Center. The health system is using AI, including deep learning and neural network-based systems, to enhance clinical decision support and ease administrative burdens. AI use cases in the provider organization include sepsis management, predicting patient deterioration and capacity management. The system is also exploring generative AI tools to help draft messages to patients or appointment summaries. But, before AI tools were implemented for the above use cases, clinicians wanted information about the algorithms and models involved. “How accurate are they? Were they trained on Vanderbilt data?” said Wright. “Do they represent the kinds of patients that we see? People were also really interested in transparency of the models. So, can I understand who developed the model, how it was developed, how up-to-date it is?” At Duke Health, clinician concerns also centered on AI safety, efficacy and equity, said Eric Gon-Chee Poon, MD, chief health information officer for Duke Medicine and primary care internal medicine provider at the Durham Medical Center. Duke Health has integrated AI into various clinical areas, including sepsis management. The health system has developed a sepsis algorithm that alerts clinicians when a patient is at risk of developing sepsis. “Today, all the patients in our emergency room benefit from this algorithm in the background,” Poon explained. “You can think of that as a guardian angel that is watching over every single patient.” The input needs to be at the right time at the right place, and it’s going to actually influence decision making. Eric Gon-Chee Poon, MDChief health information officer, Duke Medicine, and primary care internal medicine provider, Durham Medical Center The algorithm determines if a patient is at high risk of developing sepsis and sends an alert to a rapid response team, which then intervenes, he said. In addition, the health system is using AI to enhance operations, such as operating room scheduling, clinical documentation and revenue cycle management. With AI spreading across the health system, Duke Health leaders had to ensure clinicians understood the technology, including its potential limitations. In addition to AI accuracy, clinicians raised concerns about workflow integration. Wright underscored that clinician buy-in rests on setting up workflows where the clinician gets the right suggestion at the right time. “If we show an accurate suggestion at the wrong time in the workflow, it’s totally worthless,” he said. “And so figuring out who needs to see the result of an AI tool, when do they need to see it, what actions might they take on it, how do we make those actions as easy as possible? That is, in my opinion, even more important than getting the model right. It’s getting the workflow right.” Workflow integration requires a nuanced approach. Mark Larson, MD, a practicing gastroenterologist and medical director for the Mayo Clinic Platform — a Mayo Clinic division focused on using technology and data to enhance healthcare — noted that physicians working in different specialties and with varying levels of expertise will have varied experiences with AI. “Someone in primary care might have a very different experience with an AI tool than someone with subspecialty expertise because the limitations of AI are what data set it has trained off of,” he said. “And there may be experts or specialists that have a database of knowledge that an AI tool just can’t compete with. On the other hand, there may be entry-level providers, residents, students who have limited experience with patient decision-making, where the AI tool might be extremely helpful.” Mayo Clinic is “carefully and cautiously” dipping its toes into AI, Larson added. The health system is examining how AI tools could ease repetitive administrative tasks in addition to enhancing clinical decision-making. Alleviating clinician concerns is critical for health systems that aim to grow AI use within their facilities. However, a one-size-fits-all approach is likely not the answer. Health system leaders must develop diverse approaches to gain and maintain clinician trust in AI.

3. Distributed energy resources integration: Utilities look at sum-of-parts solutions

In 2024, some states and electric utilities integrated behind-the-meter distributed energy resources (DERs) and flexible loads through compensations, rate designs, and other models. This occurred against the backdrop of rising electricity demand, on one hand, with DERs providing reliability to the grid and challenges such as permitting, and interconnection with building utility-scale resources increasing, on the other. Additionally, extreme weather events, which have been increasing due to climate change, are also impacting the electricity system and causing power failures. Between 2000 and 2023, 80% of all major power outages were due to severe storms, wildfires, and extreme heat.59

As utilities address these challenges, DERs can provide a variety of capabilities, including energy efficiency, demand response, power generation, and energy storage to the grid. By combining these capabilities, utilities can create smart systems such as non-wire alternatives, microgrids, and virtual power plants (VPPs), optimizing grid operations and enhancing resilience. For example:

DERs used as non-wire alternatives, regardless of ownership, have the potential to reduce system operating costs and delay the need for system upgrades. 60 For example, Xcel Energy recently proposed to state regulators the construction of a network of strategically located solar-powered energy storage hubs. These hubs would be linked with technology to operate in concert, designed to enhance grid efficiency and reliability. 61

For example, Xcel Energy recently proposed to state regulators the construction of a network of strategically located solar-powered energy storage hubs. These hubs would be linked with technology to operate in concert, designed to enhance grid efficiency and reliability. The increasing availability of battery electric storage systems has strengthened the case for microgrids to enhance grid resilience and reliability. In February 2024, SDG&E introduced four advanced microgrids capable of operating independently or in conjunction with the larger regional grid, which provide a combined storage capacity of 180 MWh across four substations. 62

By combining high-capacity, low-deployment DERs like EV storage with vehicle-to-grid technology and low-capacity, ubiquitous DERs like smart thermostats, utilities can create VPPs to optimize peak load management. Deloitte’s recent analysis points out that residential electrification creates a load that could potentially serve itself by 2035.63

Combination of such capabilities can help not only strengthen the grid at its most vulnerable points supporting reliability, particularly in areas experiencing high energy demand, but also provide resiliency, mitigating the risk of disruptions.

Additionally, utilities may still have an opportunity to unlock potential from commercial and industrial customers.64 The immense energy demand of data centers, often seen as a challenge, can become an asset for the future grid. Their ability to rapidly adjust power consumption levels and location can make them candidates for participation in VPPs, enhancing grid stability and supporting the integration of renewable energy sources.65 The introduction of FERC 2222, which allows DERs to participate in the energy market, would expand the services that VPPs can provide.66 Though there have been delays in the implementation across Independent System Operators,67 in April 2024, New York launched the nation’s first program to integrate aggregations of DERs into wholesale markets.68

While these solutions may seem isolated, they are expected to continue to converge and could continue to create synergies for a more reliable and sustainable electricity infrastructure. This integration can deliver economic benefits to both the grid and customers. As the industry prepares for rising demand from data centers, VPP platforms leveraging AI and ML algorithms can aid in managing power generation assets, understanding customer behavior, and adjusting output levels based on demand and forecast consumption.

However, currently, there are hurdles that need to be overcome to facilitate greater investment in VPP infrastructure. According to Deloitte survey respondents, technology integration, cyber, and operation complexities are the top three challenges in scaling VPPs (figure 4).69

The US CTSA network contains over 60 hubs. We sent email invitations to 64 CTSA leaders, each responding on behalf of a unique CTSA site, with 42 confirming participation. Ultimately, we received 36 complete responses, yielding an 85.7% completion rate. Only fully completed responses were included in the analysis, as the six unfinished responses had 0–65% progress and were excluded. The survey questions are available in Supplementary Material A. Of the 36 completed responses, 15 (41.7%) represented only a CTSA, and 21 (58.3%) represented a CTSA and its affiliated hospital.

Stakeholder identification and roles

Figure 1a shows that senior leaders were the most involved in GenAI decision-making (94.4%), followed by information technology (IT) staff, researchers, and physicians. Cochran’s Q test revealed significant differences in stakeholder involvement (Q = 165.9, p < 0.0001). Post-hoc McNemar tests (see Methods) with Bonferroni correction showed senior and departmental leaders were significantly more involved than business unit leaders, nurses, patients, and community representatives (all corrected p < 0.0001; Supplementary Table 1). Nurses were also less engaged than researchers and IT staff (corrected p < 0.0001).

Fig. 1: Results on stakeholder identification and roles. a Which stakeholder groups are involved in your organization’s decision-making and implementation of GenAI? b Who leads the decision-making process for implementing GenAI applications in your organization? c How are decisions regarding adopting GenAI made in your healthcare institution? Full size image

We further split our analysis based on whether institutions have formal committees or task forces overseeing GenAI governance to provide insights into how governance models may impact GenAI adoption. 77.8% (28/36) respondents reported having formal committees or task forces overseeing GenAI governance, 19.4% (7/36) did not, and 2.8% (1/36) were unsure. We grouped those without formal committees for analysis to simplify the comparison and focus on clear distinctions between institutions with and without established governance structures. Institutions without formal committees did not involve patients and community representatives as stakeholders in the decision-making and implementation of GenAI (Fig. 1a).

Further, the decision-making process for implementing GenAI (Fig. 1b) was primarily led by cross-functional committees (80.6%), with clinical leadership also playing a key role (50.0%). Institutions without formal committees were led more by clinical leadership. Specific mentions include the dean, CTSA and innovation teams, researchers, and health AI governance committees. Cochran’s Q test revealed significant differences in leadership involvement (Q = 46.8, p < 0.0001), especially between cross-functional committees and both regulatory bodies and other stakeholders (corrected p < 0.0001; Supplementary Table 2).

Decision-making and governance structure

The decision-making process for adopting GenAI in healthcare institutions varied (Fig. 1c). A centralized (top-down) approach was used by 61.1% (22/36) of respondents, while 8.3% (3/36) mentioned alternative methods, such as decisions based on the tool’s nature or a mix of centralized and decentralized approaches.

Thematic analysis of statements about governance structures in organizations with formal committees identified two major themes (Fig. 2). “AI Governance and Policy” reflects institutions’ structured approaches to ensure responsible GenAI implementation. Institutions often establish multidisciplinary committees to integrate GenAI policies with existing frameworks, aligning AI deployment with organizational goals and regulatory requirements and focusing on legal and ethical compliance. “Strategic Leadership and Decision Making” highlights the crucial role of leadership in GenAI initiatives. High-level leaders drive GenAI integration through strategic planning and resource allocation, with integrated teams from IT, research, and clinical care fostering a culture of innovation and collaboration. Excerpts on these governance practices are detailed in Supplementary Table 3.

Fig. 2: Thematic analysis of governance and leadership structures in GenAI deployment across CTSA institutions with featured responses. This figure illustrates two primary domains of governance and leadership structures: AI Governance and Policy (blue) and Strategic Leadership and Decision-Making (orange), divided into seven subcategories. Segment sizes reflect the prevalence of each approach. Annotated quotes provide qualitative insights into governance strategies, showcasing the diversity of institutional practices in GenAI deployment. Full size image

Regulatory and ethical considerations

Regulatory body involvement in GenAI deployment varied widely across institutions (Fig. 3a). Federal agencies were engaged in 33.3% (12/36) of organizations. A significant portion (55.6%) identified other bodies, including institutional review boards (IRBs), ethics committees, community advocates, and state agencies. Internal governance committees and university task forces were also explicitly mentioned.

Fig. 3: Results on regulatory, ethical, and budget considerations. a Which regulatory bodies are involved in overseeing the deployment of GenAI in your organization? b Do you have an ethicist or an ethics committee involved in the decision-making process for implementing GenAI technologies in your organization? c Please rank the following ethical considerations from most important (1) to least important (6) when decision-makers are deciding to implement GenAI technologies. d What is the stage of GenAI adoption in your organization? e How well do GenAI applications integrate with your existing systems and workflows? f How familiar are members of the workforce with the use of LLMs in your organization? g How desirable is it for the workforce to receive further LLM training? h Have funds been allocated for GenAI projects? i Compared to 2021, how does the budget allocated to GenAI projects in your organization change? Full size image

Regarding ethical oversight (Fig. 3b), 36.1% (13/36) of respondents reported an ethicist’s involvement in GenAI decision-making; 27.8% (10/36) mentioned an ethics committee, while 19.4% (7/36) reported neither, and 16.7% (6/36) were unsure. Ethical considerations were ranked based on importance (Fig. 3c), with “Bias and fairness” (mean rank 2.31) and “Patient Privacy” (mean rank 2.36) being the top priorities.

Stage of adoption

Institutions were at varying stages of GenAI adoption (Fig. 3d), with 75.0% (27/36) in the experimentation phase, focusing on exploring AI’s potential, building skills, and identifying areas for value addition. Integrating existing systems and workflows was met with mixed responses (Fig. 3e), with 50.0% (18/36) rating it as neutral.

Workforce familiarity with LLMs also varied (Fig. 3f), with 36.1% (13/36) of respondents reporting slight familiarity and 25.0% (9/36) reporting moderate familiarity. Workforce training on LLMs was uneven, with only 36.1% (13/36) having received training, while 44.4% (16/36) considered but did not receive training, and 19.4% (7/36) neither received nor considered training. The demand for further training was evident, with 83.3% (30/36) finding it desirable or even more (Fig. 3g). The respondents who indicated receiving further LLM training for their workforce was undesirable were from institutions without a formal committee.

Vendor collaboration was crucial, with 69.4% (25/36) of institutions partnering with multiple vendors, ranging from one to twelve, to implement GenAI solutions. Notable vendors included major service providers, established EHR vendors, and various startups. Some respondents noted that discussions are often confidential or lack comprehensive information on enterprise-wide vendor engagements. Additionally, 25.0% (9/36) have considered vendor collaboration but have not engaged, while only 5.6% (2/36) have neither considered nor pursued such partnerships.

Budget trends

Regarding funds allocation for GenAI projects, 50.0% (18/36) of respondents reported that ad-hoc funding was allocated mostly from institutions with formal committees (Fig. 3h). Most institutions without formal committees reported that no funds had been allocated for GenAI projects (62.5%; 5/8). Since 2021, 36.1% (13/36) of respondents were unsure about budget changes, 19.4% (7/36) noted the budget remained roughly the same, and 44.5% reported budget increases ranging from 10% to over 300% (Fig. 3i).

Current LLM usage

Institutions were adopting LLMs with varied strategies (Fig. 4a), with 61.1% (22/36) using a combination of both open and proprietary LLMs, 11.1% (4/36) using open LLMs only, and 25.0% (9/36) using proprietary LLMs only. Only 2.8% (1/36) reported not using any LLMs. Significant differences exist (Q = 28.7, p < 0.0001) between the types of LLMs used. Post-hoc tests revealed significant differences (Supplementary Table 4) between using open and proprietary LLMs versus open LLMs only (corrected p = 0.0032), indicating a notable preference for combining different LLM types in some institutions. No significant differences were found among specific open or proprietary LLM types (Q = 2.4, p = 0.4936), suggesting that institutions did not exhibit strong preferences between particular open or proprietary LLM models. Institutions developing open LLMs prioritized technical architecture and deployment (61.1%), followed by customization and integration features (50.0%, Fig. 4c). Some institutions focused on research and experimentation, comparing open to proprietary LLMs, with interests in medical education and cost-effectiveness. Technical architecture and deployment are prioritized over clinician or patient buy-in (corrected p = 0.0024; Supplementary Table 5).

Fig. 4: Results on LLMs usage. a Which of the LLMs are you currently using? b What AI deployment options does your organization currently use? c You indicated that your organization is using open LLMs (blue) or proprietary LLMs (red). What factors influenced your decision to develop internally/to go with commercial solutions? d Which of the following use cases are you currently using LLMs for? e On a scale from 1 to 5, please rate the importance of each of the following criteria when evaluating LLMs. 1 means “Not at all Important,” and 5 means “Extremely Important”. f On a scale of 1 to 5, please rate how significant the following potential limitations or roadblocks are to your roadmap for current generative AI technology, with 1 being not important and 5 being very important. Full size image

Regarding GenAI deployment (Fig. 4b), private cloud and on-premises self-hosting were the most common approaches (both 63.9%), suggesting that most institutions have both approaches but do not take a hybrid approach. Some institutions specified using local supercomputing resources or statewide high-performance computing infrastructure. Statistical analysis (Q = 42.6, p < 0.0001) indicated a preference for more controlled environments, with private cloud and on-premises self-hosting significantly more favored than public cloud (corrected p = 0.0022 and p = 0.0060, respectively; Supplementary Table 6).

For institutions adopting proprietary LLMs, the critical factors for decision-making include technical architecture and deployment (61.1%), and scalability and performance (Fig. 4c). Respondents noted the importance of ease of deployment, especially in partnerships with established EHR vendors, and the advantage of existing Health Insurance Portability and Accountability Act (HIPAA) Business Associate Agreements with major cloud service providers. Statistical analysis (Q = 57.4, p < 0.0001) revealed significant differences, particularly between technical architecture and deployment and monitoring and reporting and AI workforce development (both corrected p = 0.0113; Supplementary Table 7). Scalability and performance were significantly more prioritized than LLM output compliance and AI monitoring and reporting (corrected p values = 0.0405).

Finally, LLMs were applied across diverse domains, with common uses in biomedical research (66.7%), medical text summarization (66.67%), and data abstraction (63.9%, Fig. 4d). Co-occurrence analysis showed frequent overlaps in these areas (Supplementary Table 8). Medical imaging analysis was the most common use case for institutions without formal committees overseeing GenAI governance. Significant differences (Supplementary Table 9) were observed in using LLMs for data abstraction compared to drug development, machine translation, and scheduling and between biomedical research and drug development, machine translation, and scheduling (corrected p values < 0.05).

LLM evaluation

Respondents prioritized accuracy and reproducible and consistent answers when evaluating LLMs for healthcare (Fig. 4e; Supplementary Table 10), each receiving the highest mean rating of 4.5. Healthcare-specific models and security and privacy risks were also deemed important, though responses varied. An analysis of variance (ANOVA) test revealed significant differences among the importance ratings (F = 3.4, p = 0.0031). Post-hoc Tukey’s honestly significant difference (HSD) tests showed a significant difference between accuracy, explainability, and transparency (p = 0.0299).

Regarding potential roadblocks to adopting GenAI in healthcare, regulatory compliance issues were rated as the most significant concern, with a mean rating of 4.2 (Fig. 4f; Supplementary Table 11). While ‘Too expensive’ and ‘Not built for healthcare and life science’ were less of a concern, they still posed challenges for some respondents, though there are no significant differences among these ratings (F = 2.0, p = 0.0606).

Projected impact

Participants rated the anticipated impact of LLMs on various use cases over the next 2–3 years (Fig. 5a; Supplementary Table 12), with the highest mean ratings for natural language query interface, information extraction, and medical text summarization (4.5 each), followed by transcribing medical encounters (4.3). Data abstraction (4.3) and medical image analysis (4.2) were also highly rated, while synthetic data generation, scheduling (3.5 each), and drug development (3.4) received lower ratings. Additional use cases, such as medical education and decentralized clinical trials, suggest an expanding scope for LLM applications.

Fig. 5: Results on projected impact and enhancement strategies. a On a scale of 1 to 5, please rate how much you think LLMs will impact each use case over the next 2–3 years. 1 means very negative, and 5 means very positive. b What improvements, if any, have you observed since implementing Generative AI (GenAI) solutions in your healthcare institution? (c) What drawbacks or negative impacts, if any, have you observed since implementing GenAI solutions? d Which steps do you take to test and improve your LLM models? e What type(s) of evaluations have your deployed LLM solutions undergone? f What challenges, if any, have you faced in integrating GenAI with existing systems? Full size image

Further, respondents reported increased operational efficiency (44.4%) as the most commonly observed improvement, with faster decision-making processes noted by 13.9% (Fig. 5b). However, none reported improved patient outcomes. Other reported improvements included increased patient satisfaction and enhanced research capacity, although some noted it was too early to prove such benefits. Significant differences among these improvements were observed (Q = 38.9, p < 0.0001; Supplementary Table 13), particularly between better patient engagement and improved patient outcomes (corrected p = 0.0026).

Regarding GenAI implementation concerns (Fig. 5c), data security was identified as a major issue by 52.78% of respondents, followed by a lack of clinician trust (50.0%) and AI bias (44.44%). Cochran’s Q Test confirmed variability in these concerns (Q = 33.3, p < 0.001). Other challenges included the time required to train models, lack of validation tools, inadequate provider training, and concerns about organizational trust. Some respondents also noted that their observations were based on internal experiences, with no implementations yet in production.

Enhancement strategies

Respondents identified several strategies for testing and improving LLMs in healthcare, with human-in-the-loop being the most common (83.3%, Fig. 5d). Significant differences (Supplementary Table 14) were noted between human-in-the-loop and methods like quantization and pruning and Reinforcement Learning with human feedback22 (corrected p < 0.0001). Significant differences were found between adversarial testing23 and human-in-the-loop and guardrails and human-in-the-loop (corrected p = 0.0067).

In evaluating deployed LLMs (Fig. 5e), the most common assessments focused on hallucinations or disinformation (50.0%) and robustness (38.9%). However, 19.4% (7/36) of respondents indicated no evaluations had been conducted. Cochran’s Q Test revealed significant variation in the importance of these evaluations (Q = 77.1, p < 0.0001), with post-hoc analysis (Supplementary Table 15) showing significant differences between explainability and prompt injection (i.e., a technique where specific prompts or questions are used to trick the GenAI into bypassing its specified restrictions, revealing weaknesses in how it understands and responds to information), and between fairness versus ideological leaning and prompt injection (corrected p = 0.0040).

Integrating GenAI into healthcare presents several challenges (Fig. 5f), with technical architecture and deployment cited most frequently (72.2%). Interestingly, AI workforce development is the most common challenge for institutions without a formal committee. Data lifecycle management was noted as a critical limitation by 52.8% (19/36) of respondents. Challenges often overlap, with technical architecture and deployment closely linked to security, scalability, and regulatory compliance issues. Additional gaps were also highlighted, such as the absence of a training plan and a limited workforce. Significant variability was observed (Q = 45.4, p < 0.0001), with post-hoc analysis indicating that technical architecture and deployment were more prevalent than LLM output compliance (i.e., the trustworthiness of the LLM output) and scalability and performance (corrected p = 0.0269; Supplementary Table 16).

Additional insights into GenAI integration

Nine respondents provided additional insights into the complexities of integrating GenAI into healthcare. They emphasized the challenges posed by the rapid pace of technological change, which complicates long-term investment and integration decisions. Organizational approaches to GenAI vary; some institutions aggressively pursue it, while others have yet to implement it on a broader scale despite individual use. The integration of GenAI has improved collaboration between researchers, physicians, and administrators, but slow decision-making and a significant gap in AI workforce skills remain critical issues. The evolving nature of AI initiatives makes it difficult to fully capture current practices, highlighting the need for a comprehensive approach that addresses technological, organizational, and workforce challenges.