example of research problem about covid 19

Early COVID-19 research is riddled with poor methods and low-quality results − a problem for science the pandemic worsened but didn’t create

Professor of Epidemiology and Biostatistics, Texas A&M University

Disclosure statement

Dennis M. Gorman does not work for, consult, own shares in or receive funding from any company or organization that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.

Texas A&M University provides funding as a founding partner of The Conversation US.

View all partners

Early in the COVID-19 pandemic, researchers flooded journals with studies about the then-novel coronavirus. Many publications streamlined the peer-review process for COVID-19 papers while keeping acceptance rates relatively high. The assumption was that policymakers and the public would be able to identify valid and useful research among a very large volume of rapidly disseminated information.

However, in my review of 74 COVID-19 papers published in 2020 in the top 15 generalist public health journals listed in Google Scholar, I found that many of these studies used poor quality methods . Several other reviews of studies published in medical journals have also shown that much early COVID-19 research used poor research methods.

Some of these papers have been cited many times. For example, the most highly cited public health publication listed on Google Scholar used data from a sample of 1,120 people, primarily well-educated young women, mostly recruited from social media over three days. Findings based on a small, self-selected convenience sample cannot be generalized to a broader population. And since the researchers ran more than 500 analyses of the data, many of the statistically significant results are likely chance occurrences. However, this study has been cited over 11,000 times .

A highly cited paper means a lot of people have mentioned it in their own work. But a high number of citations is not strongly linked to research quality , since researchers and journals can game and manipulate these metrics. High citation of low-quality research increases the chance that poor evidence is being used to inform policies, further eroding public confidence in science.

Methodology matters

I am a public health researcher with a long-standing interest in research quality and integrity. This interest lies in a belief that science has helped solve important social and public health problems. Unlike the anti-science movement spreading misinformation about such successful public health measures as vaccines, I believe rational criticism is fundamental to science.

The quality and integrity of research depends to a considerable extent on its methods. Each type of study design needs to have certain features in order for it to provide valid and useful information.

For example, researchers have known for decades that for studies evaluating the effectiveness of an intervention, a control group is needed to know whether any observed effects can be attributed to the intervention.

Systematic reviews pulling together data from existing studies should describe how the researchers identified which studies to include, assessed their quality, extracted the data and preregistered their protocols. These features are necessary to ensure the review will cover all the available evidence and tell a reader which is worth attending to and which is not.

Certain types of studies, such as one-time surveys of convenience samples that aren’t representative of the target population, collect and analyze data in a way that does not allow researchers to determine whether one variable caused a particular outcome .

All study designs have standards that researchers can consult. But adhering to standards slows research down. Having a control group doubles the amount of data that needs to be collected, and identifying and thoroughly reviewing every study on a topic takes more time than superficially reviewing some. Representative samples are harder to generate than convenience samples, and collecting data at two points in time is more work than collecting them all at the same time.

Studies comparing COVID-19 papers with non-COVID-19 papers published in the same journals found that COVID-19 papers tended to have lower quality methods and were less likely to adhere to reporting standards than non-COVID-19 papers. COVID-19 papers rarely had predetermined hypotheses and plans for how they would report their findings or analyze their data. This meant there were no safeguards against dredging the data to find “statistically significant” results that could be selectively reported.

Such methodological problems were likely overlooked in the considerably shortened peer-review process for COVID-19 papers. One study estimated the average time from submission to acceptance of 686 papers on COVID-19 to be 13 days, compared with 110 days in 539 pre-pandemic papers from the same journals. In my study, I found that two online journals that published a very high volume of methodologically weak COVID-19 papers had a peer-review process of about three weeks .

Publish-or-perish culture

These quality control issues were present before the COVID-19 pandemic. The pandemic simply pushed them into overdrive.

Journals tend to favor positive, “novel” findings : that is, results that show a statistical association between variables and supposedly identify something previously unknown. Since the pandemic was in many ways novel, it provided an opportunity for some researchers to make bold claims about how COVID-19 would spread, what its effects on mental health would be, how it could be prevented and how it might be treated.

Academics have worked in a publish-or-perish incentive system for decades, where the number of papers they publish is part of the metrics used to evaluate employment, promotion and tenure. The flood of mixed-quality COVID-19 information afforded an opportunity to increase their publication counts and boost citation metrics as journals sought and rapidly reviewed COVID-19 papers, which were more likely to be cited than non-COVID papers.

Online publishing has also contributed to the deterioration in research quality. Traditional academic publishing was limited in the quantity of articles it could generate because journals were packaged in a printed, physical document usually produced only once a month. In contrast, some of today’s online mega-journals publish thousands of papers a month. Low-quality studies rejected by reputable journals can still find an outlet happy to publish it for a fee.

Healthy criticism

Criticizing the quality of published research is fraught with risk. It can be misinterpreted as throwing fuel on the raging fire of anti-science. My response is that a critical and rational approach to the production of knowledge is, in fact, fundamental to the very practice of science and to the functioning of an open society capable of solving complex problems such as a worldwide pandemic.

Publishing a large volume of misinformation disguised as science during a pandemic obscures true and useful knowledge . At worst, this can lead to bad public health practice and policy.

Science done properly produces information that allows researchers and policymakers to better understand the world and test ideas about how to improve it. This involves critically examining the quality of a study’s designs, statistical methods, reproducibility and transparency, not the number of times it has been cited or tweeted about.

Science depends on a slow, thoughtful and meticulous approach to data collection, analysis and presentation, especially if it intends to provide information to enact effective public health policies. Likewise, thoughtful and meticulous peer review is unlikely with papers that appear in print only three weeks after they were first submitted for review. Disciplines that reward quantity of research over quality are also less likely to protect scientific integrity during crises.

Public health heavily draws upon disciplines that are experiencing replication crises , such as psychology, biomedical science and biology. It is similar to these disciplines in terms of its incentive structure, study designs and analytic methods, and its inattention to transparent methods and replication. Much public health research on COVID-19 shows that it suffers from similar poor-quality methods.

Reexamining how the discipline rewards its scholars and assesses their scholarship can help it better prepare for the next public health crisis.

• Public health
• Academic journals
• Research integrity
• Scientific research
• Public health crisis
• Research quality
• Science and society
• Science journals
• Public health research
• Study design
• COVID-19 research

Want to write?

Write an article and join a growing community of more than 195,500 academics and researchers from 5,104 institutions.

Register now

An official website of the United States government

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock Locked padlock icon ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

• Publications
• Account settings
• Advanced Search
• Journal List

Challenges and solutions in clinical research during the COVID‐19 pandemic: A narrative review

Mahin nomali, neda mehrdad, mohammad eghbal heidari, aryan ayati, amirhossein yadegar, moloud payab, alireza olyaeemanesh, bagher larijani.

• Author information
• Article notes
• Copyright and License information

Correspondence Bagher Larijani, Endocrinology and Metabolism Research Institute, Tehran University of Medical Sciences, Jalal Al‐e‐ Ahmad highway, Postal code: 1411713139, Tehran, Iran. Email: [email protected] and [email protected]

Corresponding author.

Revised 2023 Jul 15; Received 2023 Mar 13; Accepted 2023 Jul 25; Collection date 2023 Aug.

This is an open access article under the terms of the http://creativecommons.org/licenses/by/4.0/ License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

Background and Aims

The COVID‐19 pandemic has presented significant challenges to clinical research, necessitating the adoption of innovative and remote methods to conduct studies. This study aimed to investigate these challenges and propose solutions for conducting clinical research during the pandemic.

A narrative review was conducted (approval ID: IR.AMS.REC.1401.029), utilizing keyword searches in PubMed and Web of Science (WOS) citation index expanded (SCI‐EXPANDED) from January 2020 to January 2023. Keywords included COVID‐19, clinical research, barriers, obstacles, facilitators and enablers.

Out of 2508 records retrieved, 43 studies were reviewed, providing valuable insights into the challenges and corresponding solutions for conducting clinical research during the COVID‐19 pandemic. The identified challenges were categorized into four main groups: issues related to researchers or investigators, issues related to participants and ethical concerns, administrative issues, and issues related to research implementation. To address these challenges, multiple strategies were proposed, including remote monitoring through phone or video visits, online data collection and interviews to minimize in‐person contact, development of virtual platforms for participant interaction and questionnaire completion, consideration of financial incentives, adherence to essential criteria such as inclusion and exclusion parameters, participant compensation, and risk assessment for vulnerable patients.

The COVID‐19 pandemic has significantly impacted clinical research, requiring the adaptation and enhancement of existing research structures. Although remote methods and electronic equipment have limitations, they hold promise as effective solutions during this challenging period.

Keywords: challenges, clinical research, COVID‐19, narrative review, pandemics

1. INTRODUCTION

The COVID‐19 disease has caused millions of deaths around the world. The most common symptoms of the infection were respiratory problems, which caused the disease to spread rapidly. 1 At the beginning of the pandemic, measures such as quarantine, social distancing, rapid tracking of patients and restriction of presence in closed spaces, and crowding were carried out to control the disease. 2 In the absence of definitive treatment and effective vaccines, these measures were effective to some extent but negatively affected society. 3 The COVID‐19 pandemic changed many aspects of human life worldwide, causing adverse social and economic consequences that will persist for years. 4 , 5

Research is a critical aspect of responding to public health emergencies. Research efforts from various groups were focused on the origin of the COVID‐19 disease and management strategies, including drugs and vaccines through numerous clinical trials. 6 Health scientists were confronting COVID‐19 and solving its complications with investigations, research, and clinical trials. Many researchers were faced with significant challenges due to the spread of COVID‐19. These issues include the unwillingness of volunteers and research participants, reductions in research funds due to shifting toward the treatment and hospitalization of the affected, and emerging difficulties in traveling and field investigations. Furthermore, stress and concerns about COVID‐19 were shared among the participants, volunteers, research team, and scientists. 7

Field investigations are necessary for many studies, requiring the presence of scientists in the clinical environment. Furthermore, the presence of participants and volunteers is essential for many researchers. Ethical problems such as obtaining consent to participate in the research, explaining the purpose of the study, follow‐ups, and referring vulnerable patients during the study were negatively affected by the pandemic. 8 Due to the rapid spread of the COVID‐19 disease and its high mortality, there was an urgent need for research on medications, vaccines, improving diagnostic tools, and medical management, which was a challenge for scientists. 9 , 10

The COVID‐19 pandemic has significantly impacted various clinical research methods, particularly clinical trials, which are crucial in evaluating the safety and efficacy of new medical treatments. Many aspects of clinical trials, including patient recruitment, obtaining informed consent, and implementing interventions, were traditionally conducted in person by the research team. However, due to the pandemic, these processes have been disrupted, and alternative methods have had to be developed to ensure the continuity of clinical research while prioritizing the safety of participants and researchers. 11 The COVID‐19 pandemic also affected clinical research about infectious diseases as well as other medical fields, including cancer, chronic diseases, obstetrics, and gynecology. 12 , 13

The COVID‐19 pandemic led to the discontinuation of several significant health studies. This was primarily due to a lack of effective preparation for such a crisis and inadequate guidance for clinical researchers on addressing research challenges by utilizing various strategies rather than halting research in clinical settings.

As a result, many researchers were not equipped to adapt to the new circumstances and continue their studies safely and effectively. However, it is crucial to note that innovative solutions have emerged in response to these challenges, such as remote data collection and telemedicine, which have enabled researchers to continue their work while ensuring the safety of study participants and staff. Going forward, it is essential to prioritize preparedness for potential crises and provide clear guidance to researchers to ensure the continuity of critical health studies during challenging times.

Previous reviews have examined the challenges of conducting research during the COVID‐19 pandemic. However, these reviews are limited to clinical trials, research sponsors, or particular diseases. 7 , 14 A comprehensive review that examines all aspects of conducting clinical research during the pandemic is currently lacking, yet it is crucial for future crises.

Therefore, we aimed to comprehensively review the challenges faced worldwide in conducting clinical research during the COVID‐19 pandemic. We will also identify solutions to these challenges to aid researchers and policymakers in facilitating clinical research during future crises. By conducting this review, we hope to provide a comprehensive understanding of the impact of the pandemic on clinical research and contribute to the development of strategies to mitigate its effects on research activities.

2.1. Study design

This study was a narrative review approved by the ethical committee of the Iranian Academy of Medical Sciences (IAMS) (REC approval ID: IR.AMS.REC.1401.029).

2.2. Search strategy and information sources

We searched PubMed and Web of Science (WOS) citation index expanded (SCI‐EXPANDED) on January 4, 2023, to identify related articles published between January 2020 to January 2023. We developed the search strategy for the PubMed database and modified it for the WOS database (Table  1 ). The search strategy was applied without any limitation on data and languages. Manual searching in key journals for relevant articles was conducted after the initial search of databases, and the reference list of included articles was checked for possible related studies (Figure  S1 ).

Search strategies used to retrieve related documents.

2.3. Study selection process

Two authors reviewed and screened all retrieved documents independently (MN, MH) based on titles and abstracts according to the eligibility criteria. Afterward, the full texts of the selected studies were assessed. Any disagreements were resolved by the third author (NM).

The inclusion criteria consisted of studies conducted during the COVID‐19 pandemic examining challenges related to conducting research regardless of their study design. There is no limitation on the type of studies. Articles without abstract, full‐text, or sufficient relevant data were excluded.

2.4. Data extraction

The list of data extraction included the first author's name, publication year, study design, and study country or setting. Information about the challenges and related solutions was extracted and reviewed from the included articles narratively. To categorize challenges, we used expert opinion.

A total of 2508 records were retrieved through electronic databases. After removing duplicates and screening based on title and abstract, the full texts of 156 studies were assessed, and 43 studies were included in the qualitative synthesis (Figure  S1 ).

All the studies were conducted during the COVID‐19 pandemic and evaluated the challenges of conducting research in this era. The characteristics of the included studies are provided in Table  2 .

Characteristics of included studies.

The challenges included “issues related to researchers or investigators, issues related to participants and ethical concerns, administrative issues (i.e., research ethics committee [REC] or institutional review board [IRB] approval), and issues related to research conduction,” which are reviewed in the following sections.

3.1. Issues related to researchers or investigators

The increasing pressure of the pandemic on healthcare systems caused an extensive change in the employee workflow, increasing the duties of research personnel. 53 Research nurses had to work as clinical nurses in labor, delivery, and postpartum units. The pressure of researching clinical staff was significantly reduced, allowing them to respond to the patient's needs. 39 The research staffs were concerned about the risk of COVID‐19. Exposure to the disease during face‐to‐face meetings increased the chance of infection. Reducing the number of in‐person meetings and planning them through video conferences were necessary to reduce the risk of disease transmission. Also, appropriate personal protective equipments were mandatory to address these concerns. 7 , 12 Although the COVID‐19 limited medical students' classes and their presence in the hospital, many remained at their workplaces and engaged in their clinical and scientific activities. 54 At some institutions, medical students and residents played a more prominent role in screening, consenting, and enrolling clinical research participants. Tasks that were generally performed by research personnel. For example, residents performed research activities for high‐priority research regarding public health concerns such as COVID‐19 instead of research staff. 9 The participation of medical students in research varied in different countries and was related to the policies of each country and university. 55 The activity of medical students may be necessary and temporary in some critical times, such as the COVID‐19 pandemic. 54

International research and collaboration among researchers can enhance global knowledge and awareness. However, differences in goals, research priorities, and pandemic conditions can hinder cooperation. Despite these challenges, international researchers can collaborate on shared topics such as preventing disease spread, treatments, and vaccines, including different phases of clinical studies 40 (Table  1 ).

3.2. Issues related to participants and ethical concerns

During the COVID‐19 pandemic, study participant presence was a significant research limitation due to quarantine, social distancing, travel restrictions, and participant concerns. Many participants withdrew from studies due to infection fears, while high‐risk populations, such as infants, the elderly, and pregnant women, were still needed for research purposes. 41

Pandemic circumstances caused additional burdens on the health system, including the psychological pressure on researchers to provide a solution for the pandemic. Therefore, The process of project approvals should be revised in terms of speed, prioritization, and the presence of experts in the ethical approach. The research hypotheses may not be investigated in time if they are not approved and started at the right time. Also, prioritizing critical issues related to health in the ethics committees should be considered due to the crisis conditions. 6 During the Ebola epidemic, for example, clinical trials proceeded non‐stop. Research design and conduction should differ from traditional approaches during infectious disease outbreaks. 7

The main ethical challenges that organizations should investigate were obtaining informed consent and addressing ethical issues according to the study design and human interventions. 7 Another ethical issue that organizations should consider was the participants' interest in participating in research. For any research to be considered ethical, its benefits should be higher than its risks. Moreover, COVID‐19 has psychological effects on individuals. Thus, studies on mental health, depression, suicide, and self‐harm had to be carefully considered. In high‐risk projects, the purpose of the research, the stakeholders, and how it can be implemented should be apparent. Ethics committees worldwide must consider these fundamental issues and examine them seriously. 17 , 56

The expansion of online methods created favorable flexibility against COVID‐19 restrictions, such as obtaining consent electronically, responding online to ethical issues, and creating a platform for employees to handle research files remotely and outside work and office hours. 7 Some of the remote qualitative methods that were utilized included online or phone‐based interviews and focus group discussions, audio‐diary forms, photovoice (use of photography to capture lived experiences), video documenting, documentary analysis of social media (e.g., Facebook and WhatsApp groups, YouTube comments or podcasts) and auto‐ethnography. Remote quantitative methods included mobile phone surveys implemented using: interactive voice response (IVR), short messaging service (SMS), or computer‐assisted telephone interviews (CATI) and self‐completed online questionnaires shared via email or social media platforms. These methods were not new, with telephone and postal surveys used in higher‐income countries, yet their use became essential during the COVID‐19 era to support data collection directly from individuals and populations. Using technology in conducting studies was different in each country, and it depended on the national policies regarding the use of technology in health‐related research. 37 A study by Megana found that remote e‐consent‐based recruitment was crucial for trial continuity during the COVID‐19 pandemic. This method adheres to ethical and regulatory guidelines for informed consent while minimizing face‐to‐face interactions that increase COVID‐19 transmission risk. Patients provided positive feedback on using these platforms. 49

Ensuring participant safety and privacy are critical ethical considerations in clinical studies. 15 , 28 Accountability, tracking, and follow‐up before and after interventions must be prioritized to continue trials. Confidentiality of patient data and the secure delivery of investigational treatments from trial sites are essential. Participants must also be provided with instructions for properly storing and using investigational drugs. 16 , 29

Result by Shields et al. 30 showed that fear of COVID‐19 was a major barrier to follow‐ups. This fear included patients who felt unsafe exposing themselves or their family members or a patient's family member feeling unsafe exposing the patient. The next most commonly reported barriers were long waiting times and financial costs.

Informed consent is a common and fundamental part of any clinical research. It is usually provided by paper forms that explain the purpose of the study, the procedures, and possible adverse effects and are signed by the participants. 28 , 57 A virtual electronic consent form is an alternative approach to traditional written forms. 16 Considering the risk of infection transmission during pandemics, consent can be acquired electronically. Verbal consent for quarantined patients can be obtained first in the presence of a witness, followed by written consent when participants are released from quarantine. Thus, institutions should allocate the necessary resources to develop an appropriate consent form. Facilitating communication between participants, researchers, and institutions can help better collaboration between participants. 57

Many studies are conducted on healthy community populations, and some projects are carried out on sensitive populations and high‐risk patients. Various studies are conducted in the suburbs and villages. These populations are essential in many ways, including dangerous risk factors such as obesity, unemployment, health considerations, and community health. However, the COVID‐19 pandemic prevented these individuals from participating in studies, and effective incentives were needed to encourage participation. 58 , 59 On the other hand, research on sensitive populations was considered dangerous. Addicts, sex workers, and the homeless did not follow many health protocols. Many lived in the same room with several people and did not practice social distancing. These cases could cause the spread of the coronavirus to the researchers and others, endangering the health of the participants and study operators. 18 , 58

Another effective way to attract participants is through financial incentives. Allocating the necessary funding for these incentives is a task that health organizations should notice. Providing essential funding, creating financial incentives, and paying attention to the participants' health can facilitate active participation in the research. 58 A study by Basel showed that statistically significant increases were seen in participants' consent rates and responses when offered even small monetary value incentives. These findings suggest that incentives may be used to reduce the rate of recruitment failure and subsequent study termination 60 (Table  3 ).

Challenges and solutions in clinical research during the COVID‐19 pandemic.

3.3. Administrative issues (REC or IRB approval)

3.3.1. rapid review of research.

Thousands of clinical trials were registered in the first few months of the pandemic, facing ethics committees with a high load of studies. A thorough review was necessary to prevent high‐risk and low‐benefit treatments on patients. IRBs had to prioritize specific issues such as inclusion or exclusion criteria, participant compensation, and risk assessments for vulnerable patients to facilitate rapid research review and management of time (Table  3 ).

3.3.2. Ethical issues after IRBs

Despite ethical review board approval, many studies deviated from their protocols due to circumstances during the study. To ensure transparency and efficiency, modifications to pre‐study documents, consent forms, study entry reports, conflict of interest, sponsorship, and side effects had to be reported to the ethics committee. This allowed for transparency and ensures that changes are made appropriately. 15 , 39

3.3.3. Structure and process of IRB

Ethics committees faced the challenge of requiring a multidisciplinary team of experts in virology, infectious diseases, pharmaceuticals, and public health for quick and accurate document review. 28 To address this, committees should prioritize investigator‐initiated trials from a public health perspective and expedite the review of academic trials that address important questions. Regulatory approval processes should be streamlined, redundancies in research design approval processes eliminated, and urgent public health trials facilitated. Experts in different fields can review these indicators. 39 To expedite the approval of interventional studies, having only one national ethics committee review and approve studies is recommended. This approval should be accepted throughout the country without needing re‐approval by another hospital or city's ethics committee 20 (Table  3 ).

3.4. Issues related to research conduction

3.4.1. clinical trials.

Limited access to healthcare facilities and resources significantly impacted research during the COVID‐19 pandemic. Quarantine restrictions affected adherence to clinical study protocols, making it challenging to conduct studies, document procedures, and report adverse events and safety evaluations. This prevented the implementation of numerous clinical studies. Risk assessment was necessary to consider current risks and disadvantages when starting a new study or recruiting trial participants. 12 , 42

The pandemic significantly impacted clinical trials, particularly in cases where patient follow‐ups and randomization were halted, leading to economic losses. Many unnecessary experiments were stopped to prioritize the research with a greater benefit‐to‐harm ratio. 21

Ethically speaking, exposing trial participants to risk is unacceptable if the study is not designed to provide valid results. Therefore, rigorous methodology should be implemented, including randomization, blinding, and placebo use, to enhance scientific validity and societal value. However, in severe epidemics, insisting on randomization can create a conflict between individual health and societal interests, precluding patients' autonomy in choosing their therapy. 66 In a clinical trial conducted during the Ebola epidemic, Perez et al. recommended prioritizing individual patient interests over the reliability of trial methodology when faced with a high risk of death. In a pandemic scenario, a high number of seriously ill patients presenting simultaneously with a high mortality rate make it ethically unacceptable to randomly allocate patients from the same family or location to receive or not receive an experimental drug. Additionally, critically ill patients may find the randomization procedure difficult to understand. 67 It would be unethical and impractical to conduct a randomized controlled trial (RCT) that asks patients or family members to consent to standard care when a potentially beneficial therapy is available. In the LOTUS China, an open‐label RCT, 31 patients' families (8.6%) did not provide consent. For the Ebola trial, investigators conducted one group open‐label non‐randomized trial, where all patients received Favipiravir with standardized care. The investigators used historical mortality data to define efficacy endpoints and a target mortality threshold a priori, which was valuable in deciding whether to stop or continue the trial and guide data analysis and interpretation. This approach could improve the utility of efficacy information from non‐randomized trials. The World Health Organization (WHO) has planned SOLIDARITY, a large global trial of four drugs—Remdesivir, Chloroquine and Hydroxychloroquine, Lopinavir‐Ritonavir, and lopinavir‐ritonavir plus interferon‐beta. Its simple design allowed physicians to recruit confirmed COVID‐19 cases after obtaining informed consent and administer any of the four available drugs as per randomization by the WHO. 68 , 69 , 70

Patient enrolment

One of the problems in conducting research during the COVID‐19 pandemic was patient enrolment. VACCELERATE Volunteer Registry was one of the systems that facilitated the enrolment of patients into studies. VACCELERATE is a comprehensive and coordinated database for conducting and enrolling volunteers for Phase II and Phase III clinical trials. Moreover, this registry can also be expanded to test vaccines on humans in future health emergencies. 43

The pandemic limitations urged new measures for retaining study participants and registering new participants. Strong communication and commitment to participants, creating technological capabilities for teleworking, visits, and delivery of study medication are essential in effectively retaining study participants and recruiting new participants during the COVID‐19 pandemic. 34 , 35 , 52 Facilitating remote patient visits, motivation to perform procedures at the patient's home, permission to use healthcare facilities, direct distribution of the medicine to the patient's home by site personnel or sponsors, and extension of reimbursement to patients and caregivers are solutions that can facilitate the process of clinical studies in pandemic crisis. 20 , 36 , 45 , 63 Online platforms and social media were among the most practical strategies to reduce the imposed limitations. 29 Simmons et al. 46 replaced all in‐person parts of their clinical study using two key technology platforms: Study Pages (Yuzu Labs Public Benefit Corporation, 2022) and Pattern Health (Durham, NC). Recruitment and screening, consent, enrollment, randomization, data collection, blinding, adherence, and retention were performed with these platforms.

While recruiting study subjects can be difficult in typical circumstances, the COVID‐19 pandemic posed additional obstacles for individuals and children seeking to participate in pediatric nursing research. Skeens' study found that using social media to recruit a sample of parent‐child dyads during the COVID‐19 pandemic was an innovative technique. 51 In addition, an original web‐based survey determined that social media was a successful and efficient technique for gathering data on COVID‐19 in a short period of time. 27

Faster research dissemination

In response to COVID‐19, the research community has rapidly adopted a new way of disseminating research. However, unfortunately, the way in which research is being conducted has not changed. There has been an unprecedented surge of COVID‐19‐related preprints and peer‐reviewed publications. While preprint servers and faster peer review processes have clear merits, such as quicker dissemination of results, informing policies, and speeding up the R&D process for COVID‐19 therapeutics and vaccines, the quality of COVID‐19 research has been largely subpar. Many preprints, which are not peer‐reviewed, were rushed to dissemination without sufficient oversight, leading to potential inaccuracies and false claims. 31

Employing virtual platforms

Limited face‐to‐face interactions during the pandemic significantly reduced the number of research visits, and study evaluations, requiring most research visits to be conducted remotely or via phone or video calls. For example, in drug effectiveness studies, by editing the protocols, the study medications could be mailed to the participants instead of in‐person deliveries. 7 , 15

The lack of experience regarding virtual platforms to implement clinical studies also affected the results. Lack of face‐to‐face communication, the reduction of interpersonal interaction between the researcher and the participants, and the accuracy of the acquired information were among the limitations that could cause bias in clinical studies. 39

Each remote data collection method has its advantages and disadvantages that determine its feasibility and acceptability in certain settings. For example, when considering a mobile phone survey, IVR and SMS surveys are more affordable than CATI, but require participants to have high literacy levels. CATI, on the other hand, allows for the inclusion of individuals regardless of their literacy level and provides opportunities for researchers to encourage participation and clarify questions. In low‐ and middle‐income countries, where mobile phone ownership is widespread but access to smartphones and the internet is limited, mobile phone methods are more commonly used and are the focus of this commentary. However, few experts interviewed had implemented or planned online strategies due to their limited reach in certain low‐ and middle‐income countries. Some exceptions include online surveys designed for specific target groups, such as members of established professional associations and university students. 37

3.4.2. Epidemiologic studies

Epidemiological studies, like other studies, have been affected by COVID‐19. During recruitment and longitudinal assessments, epidemiologic studies are susceptible to refusals and losses of follow‐up. In face‐to‐face data collection, researchers adopt strategies such as changing the interviewer or contacting the participant on different days/times to mitigate this issue. However, researchers cannot pinpoint the number of people reached by internet‐based approaches. While some social media platforms, such as Instagram, allow publishers to see how many people were reached by posted advertisements, others, like WhatsApp, do not. Thus, it is not possible to calculate refusal/loss rates. However, sample size calculations should consider a certain percentage of losses and refusals. Therefore, sample size calculations should be conducted before data collection begins, and researchers should devise a recruitment strategy that allows them to reach the previously defined sample. 32

3.4.3. Data analysis

One of the essential component of clinical studies is statistical models methods. 22 Statistical methods are necessary to prevent or minimize the risk of bias, a common threat in clinical and epidemiological studies. Obtaining appropriate clinical information from  patients with COVID‐19 in the city of Wuhan was only possible by the epidemiological data. Data Integration and cleaning from large multicenter hospitals are critical and require complex data management. Artificial intelligence (AI) and deep learning algorithms can be crucial in dealing with these challenges. AI and machine‐learning solutions could have a significant impact on fighting the disease. For instance, machine learning techniques have been used intensively in studying different conditions regarding protein analysis, forecasting, prediction, and paving the way towards vaccines and antivirals. An example of such a disease is the seasonal Flu. From this perspective, many AI approaches (including disease forecasting, surveillance, expected peak, and spread models) have been proposed and developed for several diseases, including the seasonal flu, which is relatively similar in its symptoms to COVID‐19. 64

There was also a need for an international committee of statistical experts to decide on statistical methods during the COVID‐19 pandemic. 33 , 71 Additional measures were needed besides the usual strategies for conducting a clinical trial to deal with the mentioned challenges in a pandemic. The conditions of participation, measures needed to prevent infection, and the possibility of withdrawing from the study should be available before making decisions for participants at increased risk of infection. 7 , 24

3.4.4. Research protocols and guidelines during a pandemic

During a pandemic, data security, patient satisfaction, and ethical statements, which are necessary in non‐pandemic situations, can be considered bureaucratic obstacles. However, rapid access to clinical data during epidemic circumstances requires special handling of these matters, which should be discussed nationally. 23 Another statistical challenge during the COVID‐19 pandemic was that many clinical studies were not implemented according to written protocols due to the inability to blind, obtain a high sample size, and randomize. Therefore, statistical methods must be adapted to pandemic conditions. Data should be collected and analyzed in a standardized way, and statisticians are encouraged to develop appropriate analytical strategies for data collected from standardized protocols such as ISARIC and LEOSS. Rapid and valid information flow and reporting are crucial during a pandemic, and long‐lasting reporting guidelines may do more harm than good. Specific reporting guidelines are needed for pandemic settings. 23

Another challenge was related to studies started before the pandemic that were affected by COVID‐19. Challenges included discontinuation of medication, withdrawal of a significant number of participants, deaths due to COVID‐19, and changes in study arms, which were not foreseen and affect study designs. Changing and updating the study protocol, continuing the investigation, and performing sensitivity analysis for missing data can be suitable solutions. 25

3.4.5. High‐risk populations

Research on the elderly population with chronic diseases posed another challenge. To prevent disruptions in research implementation for this population, patient registry systems, improved interactions with other institutions associated with the elderly, and improved study participation conditions such as transportation, health, and safety are necessary. 43 COVID‐19 has also posed one of the biggest challenges for non‐COVID‐19 research on older people. The pandemic has made research challenging to conduct in practice and diverted the time and resources of investigators, funders, regulators, and delivery teams away from non‐COVID‐19 research. Survey data from the British Association of Stroke Physicians showed that most UK stroke research projects had been halted, and all responding sites had seen a substantial decrease in stroke research activity. The economic shock delivered by the pandemic is likely to lead to significant cuts to public and charity budgets worldwide, and it is unclear to what extent this will affect medical research. Even if medical research budgets are preserved, COVID‐19‐related research will likely compete with non‐COVID‐19 research for funding. 26

Funding and financial sponsorship were other prominent issues during the COVID‐19 pandemic. Most funds were devoted to the treatment of patients and protective measures, leading to financial challenges that need to be resolved by organizations and institutions during crises and pandemics. To address this issue, a top‐down decision‐making mechanism was established in the European Union, where adequate funding was quickly provided through the Horizon Europe and ERA4Health budgets. 48

4. DISCUSSION

4.1. main findings.

In this study, we reviewed fundamental challenges in conducting clinical research in the era of COVID‐19 pandemic. Individuals, communities, and societies are facing severe social, physical, and emotional challenges during the COVID‐19 pandemic. Decisions about conduct research using remote methods should consider the research burden and the risks associated with COVID‐19 to study participants.

4.2. Comaprison with previous studies

Remote data collection requires much effort from the study participants, who may need to use their own resources, such as a phone, internet access, and identifying a private space to participate in the study. On the other hand, remote methods may be preferable for study participants and eliminate the time and opportunity costs associated with travel to study sites. As with any research, the potential risks must be weighed against the benefits and the ethical imperative to continue the research to produce evidence useful for public health. 37 Original studies also showed that remote data collection was an effective way to deal with the restrictions created by COVID‐19. Also, original studies determined that using technology like social media was an effective strategy for conducting research. 27 , 51

Key challenges in remote data collection encompass gathering diverse experiences in qualitative research, obtaining a representative sampling frame of the target population in quantitative research, and reaching out to more accessible populations. 63 , 72 While some of these challenges also exist in face‐to‐face research, the limited ability to personally recruit participants, whether at home, in a clinic, or any other locations, along with the reliance on mobile phones for recruitment, poses a specific challenge. This necessitates the exploration of alternative sampling methods for qualitative research, including purposive, snowball, and convenience sampling.

Purposive sampling aims to ensure diversity by considering key factors that are theorized to influence the experience. Recruitment can be facilitated through community‐based organizations, influential community leaders, neighborhood health committees, or established networks. Snowball sampling can be an effective approach for qualitative research; however, it is crucial to involve several initial participants who can then recruit others from within their own networks to achieve the desired diversity. 73 , 74 These sampling methods can also be used in quantitative research. Snowball sampling may be useful for online surveys shared via email or social media platforms, 75 and a convenience sample can be employed through online social networking platforms.

Verbal consent (via phone or voice note) or written consent (via email, WhatsApp, or SMS) is accepted by some ethics committees because written informed consent becomes challenging or impossible during a pandemic. For mobile phone‐based research with adolescents, which requires parental consent, additional challenges arise in verifying the participant's age to determine the adolescent consent. Parents' satisfaction should be examined in line with adolescent satisfaction. For these reasons, verbal consent may be preferred over written consent, which can be recorded or performed in conjunction with written consent. Concise and simple language is required to convey complete information remotely while maintaining the strict ethical standards of face‐to‐face research. Consent should always be documented appropriately while protecting patient information and confidentiality. Documentation can take the form of a list of participants, stored on a password‐protected computer, who have consented to participate in various study components, which can also serve as a record for audit purposes. 37

The privacy and safety of participants are crucial considerations when conducting research. In face‐to‐face studies, it is the responsibility of the researcher to establish and ensure privacy, and data collection must be halted if privacy is compromised. However, the onus is placed on the study participant in remote research to maintain their privacy. Nonetheless, establishing privacy can be challenging when participants share living spaces and have limited access to private areas and time. This becomes particularly significant in studies that examine sensitive topics like gender‐based violence, where compromised privacy can have harmful consequences. 76

To address this issue, it is essential to inform participants about the potentially sensitive nature of the study at the beginning of data collection and encourage them to seek a private space. Strategies such as incorporating “passwords” or “exit buttons” can be implemented to mitigate risks. These mechanisms allow participants to verbally state or click on an option to indicate a breach of privacy. 76 IVR and online surveys allow participants to complete surveys at a time and place of their choice, enabling them to establish privacy more effectively. Furthermore, these surveys can include a question asking respondents whether they completed the survey in private or in the presence of someone else, such as their child, parent/guardian, or friend. 37

Data protection, including end‐to‐end encryption of phone calls and the security of platforms used to deliver online surveys and interview transcripts, is an additional privacy and confidentiality issue that needs to be addressed. 77 In addition, researchers have the duty of care and should carefully consider safeguarding issues, particularly where COVID‐19 has affected the availability of support services. Information about online or telephone services must be available during the consent process. Specific referral protocols should be established, interviewers should be notified if certain responses may trigger automatic referrals, and follow‐up should be provided if safeguarding issues arise. As part of this protocol, researchers must establish a system to regularly check that these services remain operational. 42

4.3. Strengths and limitations

This is the first review to study both the challenges and solutions of conducting clinical research during the COVID‐19 pandemic, providing a practical guide for researchers and policymakers in future similar pandemic conditions. However, this study had some limitations. We had to rely on primary studies, as there was not enough information about the challenges of conducting studies in all types of research. Additionally, the majority of the studies discussed in this article were in the form of editorials, highlighting the need for more rigorous studies to investigate the subject matter further.

Nevertheless, this study has proposed effective solutions that policymakers can consider for implementation in the context of decision‐making for addressing the ongoing pandemic and future crises. Although WHO has declared the end of the COVID‐19 pandemic, 65 this review can still provide valuable information to achieve structured guidelines for researchers in future crises.

5. CONCLUSION

The study findings revealed significant challenges associated with conducting research during the COVID‐19 era. These challenges span various stages, ranging from research inception and study approval to patient enrollment and data analysis. Existing solutions must be adapted to the prevailing circumstances, highlighting the importance of enhancing the underlying research infrastructure to ensure continuity during times of crisis and pandemics. Numerous studies have proposed remote methods and electronic equipment as viable approaches to conduct research. However, the successful implementation of these methods relies on the availability of adequate infrastructure and adherence to country‐specific national and university policies.

AUTHOR CONTRIBUTIONS

Mahin Nomali : Data curation; Formal analysis; Investigation; Validation; Writing—original draft. Neda Mehrdad : Conceptualization; Investigation; Supervision; Validation. Mohammad Eghbal Heidari : Data curation; Investigation; Writing—original draft. Aryan Ayati : Writing—original draft; Writing—review & editing. Amirhossein Yadegar : Writing—review & editing. Moloud Payab : Supervision; Validation. Alireza Olyaeemanesh : Data curation; Investigation. Bagher Larijani : Conceptualization; Project administration; Supervision; Validation.

CONFLICT OF INTEREST STATEMENT

The authors declare no conflicts of interest.

ETHICAL STATEMENT

All the authors declare that the present work has been carried out per the Journal's Practice Guidelines on Publishing Ethics and has been performed ethically and responsibly, with no research misconduct. The article has not been previously published and is not currently submitted elsewhere. The study proposal was passed by the ethical committee of the Iranian Academy of Medical Sciences (IAMS) (ID: IR.AMS.REC.1401.029).

TRANSPARENCY STATEMENT

The lead author Bagher Larijani affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.

Supporting information

Supporting information.

ACKNOWLEDGMENTS

We would like to thank the Iranian Academy of Medical Sciences for approving the study protocol and for the financial support, which make this review possible. This study was supported financially by the Iranian Academy of Medical Sciences (IAMS).

Nomali M, Mehrdad N, Heidari ME, et al. Challenges and solutions in clinical research during the COVID‐19 pandemic: a narrative review. Health Sci Rep. 2023;6:e1482. 10.1002/hsr2.1482

Mahin Nomali and Neda Mehrdad equally contributed as co‐first authors.

DATA AVAILABILITY STATEMENT

All data associated with the article is available upon request.

• 1. Ali I, Alharbi OML. COVID‐19: disease, management, treatment, and social impact. Sci Total Environ. 2020;728:138861. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 2. Mishra BK, Keshri AK, Rao YS, et al. COVID‐19 created chaos across the globe: three novel quarantine epidemic models. Chaos Solitons Fractals. 2020;138:109928. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 3. Güner R, Hasanoğlu İ, Aktaş F. COVID‐19: prevention and control measures in community. Turk J Med Sci. 2020;50(9):571‐577. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 4. Wang C, Horby PW, Hayden FG, Gao GF. A novel coronavirus outbreak of global health concern. The lancet. 2020;395(10223):470‐473. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 5. Zhang H. Ethical reflections for conducting 2019‐nCoV related‐research during the public health emergency. Chin Med Ethics . 2020. 10.14055/j.cnki.33-1056/f.20200210.001 [ DOI ]
• 6. Burgess T, Rennie S, Moodley K. Key ethical issues encountered during COVID‐19 research: a thematic analysis of perspectives from South African research ethics committees. BMC Med Ethics. 2023;24(1):11. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 7. Mourad M, Bousleiman S, Wapner R, Gyamfi‐Bannerman C, eds. Conducting research during the COVID‐19 pandemic. Seminars in perinatology. Elsevier; 2020. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 8. Huang C, Wang Y, Li X, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. The lancet. 2020;395(10223):497‐506. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 9. Munblit D, Nicholson TR, Needham DM, et al. Studying the post‐COVID‐19 condition: research challenges, strategies, and importance of Core Outcome Set development. BMC Med. 2022;20(1):50. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 10. Samimisedeh P, Jafari Afshar E, Shafiabadi Hassani N, Rastad H. Cardiac MRI findings in COVID‐19 vaccine‐related myocarditis: a pooled analysis of 468 patients. J Magn Reson Imaging. 2022;56(4):971‐982. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 11. Bierer BE, White SA, Barnes JM, Gelinas L. Ethical challenges in clinical research during the COVID‐19 pandemic. J Bioeth Inq. 2020;17(4):717‐722. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 12. Loucks TL, Tyson C, Dorr D, et al. Clinical research during the COVID‐19 pandemic: the role of virtual visits and digital approaches. J Clin Transl Sci. 2021;5(1):e102. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 13. Tayebi A, Rezaei M, Bakhtiyari M, Mohamadi Afrakoti M. Outcomes, risk factors, and incidence of acute kidney injury in hospitalized COVID‐19 patients. Iran Red Crescent Med J . 2023;25(2). 10.32592/ircmj.2023.25.2.2125 [ DOI ] [ Google Scholar ]
• 14. Ma X, Wang Y, Gao T, et al. Challenges and strategies to research ethics in conducting COVID‐19 research. J Evid Based Med. 2020;13(2):173‐177. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 15. Collier EK, Hsiao JL, Shi VY. Conducting clinical trials during the COVID‐19 pandemic. J Dermatolog Treat. 2020;31(4):330‐332. [ DOI ] [ PubMed ] [ Google Scholar ]
• 16. Miguel M, Doger B, Boni V, et al. Increased vulnerability of clinical research units during the COVID‐19 crisis and their protection. Cancer. 2020;126(17):3907‐3911. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 17. Townsend E, Nielsen E, Allister R, Cassidy SA. Key ethical questions for research during the COVID‐19 pandemic. The Lancet Psychiatry. 2020;7(5):381‐383. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 18. Kantamneni N. The impact of the COVID‐19 pandemic on marginalized populations in the United States: a research agenda. J Vocat Behav. 2020;119:103439. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 19. Cardel MI, Manasse S, Krukowski RA, et al. COVID‐19 impacts mental health outcomes and ability/desire to participate in research among current research participants. Obesity. 2020;28(12):2272‐2281. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 20. Cagnazzo C, Besse M‐G, Manfellotto D, et al. Lessons learned from COVID‐19 for clinical research operations in Italy: what have we learned and what can we apply in the future?: SAGE Publications Sage UK: London. England. 2016;107:11‐16. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 21. Palayew A, Norgaard O, Safreed‐Harmon K, Andersen TH, Rasmussen LN, Lazarus JV. Pandemic publishing poses a new COVID‐19 challenge. Nat Hum Behav. 2020;4(7):666‐669. [ DOI ] [ PubMed ] [ Google Scholar ]
• 22. Hamasaki T, Bretz F, Cooner F, LaVange LM, Posch M. Statistical challenges in the conduct and Management of Ongoing Clinical Trials during the COVID‐19 pandemic. Taylor & Francis; 2020:397‐398. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 23. Wolkewitz M, Puljak L. Methodological challenges of analysing COVID‐19 data during the pandemic. BioMed Central. 2020;20:1‐4. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 24. Padala PR, Jendro AM, Padala KP. Conducting clinical research during the COVID‐19 pandemic: investigator and participant perspectives. JMIR Public Health Surveill. 2020;6(2):e18887. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 25. Akacha M, Branson J, Bretz F, et al. Challenges in assessing the impact of the COVID‐19 pandemic on the integrity and interpretability of clinical trials. statistics in biopharmaceutical. Research. 2020;12(4):419‐426. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 26. Richardson SJ, Carroll CB, Close J, et al. Research with older people in a world with COVID‐19: identification of current and future priorities, challenges and opportunities. Age Ageing. 2020;49(6):901‐906. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 27. Ali SH, Foreman J, Capasso A, Jones AM, Tozan Y, DiClemente RJ. Social media as a recruitment platform for a nationwide online survey of COVID‐19 knowledge, beliefs, and practices in the United States: methodology and feasibility analysis. BMC Med Res Methodol. 2020;20(1):116. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 28. Tamariz L, Hendler FJ, Wells JM, Anderson A, Bartlett S. A call for better, not faster, research ethics committee reviews in the Covid‐19 era. Ethics Hum Res. 2021;43(5):42‐44. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 29. Mohan S. Challenges of clinical research administration during the COVID‐19 pandemic. Narrat Inq Bioeth. 2021;11(1):101‐105. [ DOI ] [ PubMed ] [ Google Scholar ]
• 30. Shields CN, Cherkas EG, Mokhashi N, et al. Barriers to follow‐up retinal care during the COVID‐19 pandemic: a survey study. Ophthalmic Surg Lasers Imaging Retina. 2021;52(10):526‐533. [ DOI ] [ PubMed ] [ Google Scholar ]
• 31. Park JJ, Mogg R, Smith GE, et al. How COVID‐19 has fundamentally changed clinical research in global health. Lancet Glob Health. 2021;9(5):e711‐e720. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 32. Caputo EL, Feter N, Rombaldi AJ, Silva MC, Reichert FF. What are the challenges of epidemiological research during the COVID‐19 pandemic? Motriz: Revista de Educação Física. 2021;27:27. [ Google Scholar ]
• 33. Röhr S, Arai H, Mangialasche F, et al. Impact of the COVID‐19 pandemic on statistical design and analysis plans for multidomain intervention clinical trials: experience from World‐Wide FINGERS. Alzheimer's & dementia (New York, N. Y.). 2021;7(1):12143. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 34. Tom M, Cohen EJ, LopezJimenez C, Hochman JS, Troxel AB, Jeng BH. Meeting the challenges of retention and enrollment of study participants in clinical trials during the COVID‐19 pandemic from the study leadership perspective: experience from the zoster eye disease study (ZEDS). Contemp Clin Trials Commun. 2021;24:100875. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 35. Anderson M. How the COVID‐19 pandemic is changing clinical trial conduct and driving innovation in bioanalysis. Bioanalysis. 2021;13(15):1195‐1203. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 36. Janiaud P, Axfors C, Ioannidis JPA, Hemkens LG. Recruitment and results reporting of COVID‐19 randomized clinical trials registered in the first 100 days of the pandemic. JAMA Network Open. 2021;4(3):e210330. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 37. Hensen B, Mackworth‐Young CRS, Simwinga M, et al. Remote data collection for public health research in a COVID‐19 era: ethical implications, challenges and opportunities. Health Policy Plan. 2021;36(3):360‐368. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 38. Tremblay S, Castiglione S, Audet L‐A, Desmarais M, Horace M, Peláez S. Conducting qualitative research to respond to COVID‐19 challenges: reflections for the present and beyond. Int J Qual Methods. 2021;20:160940692110096. [ Google Scholar ]
• 39. Bassi A, Arfin S, Joshi R, et al. Challenges in operationalising clinical trials in India during the COVID‐19 pandemic. the lancet. Lancet Global Health. 2022;10(3):e317‐e319. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 40. Mobaraka J, Elkazzaz L, Rizkalla N. Challenges of researchers in conducting international study during the eruption of COVID‐19: student and mentor perspectives. Int J Environ Res Public Health. 2022;19(2):844. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 41. Sewell CA, Sheehan SM, Gill MS, et al. Scientific, ethical, and legal considerations for the inclusion of pregnant people in clinical trials. Am J Obstet Gynecol. 2022;227(6):805‐811. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 42. Walshe C, Garner I, Dunleavy L, et al. Prohibit, protect, or adapt? The changing role of volunteers in palliative and hospice care services during the COVID‐19 pandemic A multinational survey (CovPall). Int J Health Policy Manag. 2022;11(10):2146. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 43. Salmanton‐García J, Stewart FA, Heringer S, et al. VACCELERATE volunteer registry: a European study participant database to facilitate clinical trial enrolment. Vaccine. 2022;40(31):4090‐4097. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 44. Santillan DA, Brandt DS, Sinkey R, et al. Barriers and solutions to developing and maintaining research networks during a pandemic: an example from the iELEVATE perinatal network. J Clin Transl Sci. 2022;6(1):e56. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 45. Haynes RM, Sirintrapun SJ, Gao J, McKenzie AJ. Using technology to enhance cancer clinical trial participation. Am Soc Clin Oncol Educ Book. 2022;42:39‐45. [ DOI ] [ PubMed ] [ Google Scholar ]
• 46. Simmons LA, Phipps JE, Whipps M, et al. From hybrid to fully remote clinical trial amidst the COVID‐19 pandemic: strategies to promote recruitment, retention, and engagement in a randomized mHealth trial. Digital Health. 2022;8:205520762211290. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 47. Sharma RK, Teng A, Asirot MG, Taylor JO, Borson S, Turner AM. Challenges and opportunities in conducting research with older adults with dementia during COVID‐19 and beyond. J Am Geriatr Soc. 2022;70(5):1306‐1313. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 48. Diallo A, Trøseid M, Simensen VC, et al. Accelerating clinical trial implementation in the context of the COVID‐19 pandemic: challenges, lessons learned and recommendations from DisCoVeRy and the EU‐SolidAct EU response group. Clin Microbiol Infect. 2022;28(1):1‐5. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 49. Almeida‐Magana R, Maroof H, Grierson J, et al. E‐Consent‐a guide to implement an innovative solution to maintain recruitment in clinical trials during the COVID‐19 pandemic. 2022. [ DOI ] [ PMC free article ] [ PubMed ]
• 50. McMillan G, Mayer C, Tang R, et al. Planning for the next pandemic: ethics and innovation today for improved clinical trials tomorrow. statistics in biopharmaceutical. Research. 2022;14(1):22‐27. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 51. Skeens MA, Sutherland‐Foggio M, Damman C, Gerhardt CA, Akard TF. Facebook recruitment for research of children and parents during the COVID‐19 pandemic. Appl Nurs Res. 2022;65:151574. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 52. Pertl K, Petluri R, Wiest K, et al. Recruitment challenges for a prospective telehealth cohort study. Contemp Clin Trials Commun. 2023;31:101043. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 53. Abadi SAH, Atbaei R, Forouhi M, et al. Preventive and therapeutic approaches in medical departments during the COVID‐19 pandemic. Kindle. 2022;2(1):1‐216. [ Google Scholar ]
• 54. Ferrel MN, Ryan JJ. The impact of COVID‐19 on medical education. Cureus. 2020;12(3):e7492. 10.7759/cureus.7492 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 55. Jhurree V. Technology integration in education in developing countries: guidelines to policy makers. International Education Journal. 2005;6(4):467‐483. [ Google Scholar ]
• 56. Robert R, Kentish‐Barnes N, Boyer A, Laurent A, Azoulay E, Reignier J. Ethical dilemmas due to the Covid‐19 pandemic. Ann Intensive Care . 2020;10(84). 10.1186/s13613-020-00702-7 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 57. Rothwell E, Brassil D, Barton‐Baxter M, et al. Informed consent: old and new challenges in the context of the COVID‐19 pandemic. Journal of Clinical and Translational Science. 2021;5(1):e105. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 58. Sevelius JM, Gutierrez‐Mock L, Zamudio‐Haas S, et al. Research with marginalized communities: challenges to continuity during the COVID‐19 pandemic. AIDS Behav. 2020;24(7):2009‐2012. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 59. Bailey C, Black JRM, Swanton C. Cancer research: the lessons to learn from COVID‐19. Cancer Discov. 2020;10(9):1263‐1266. [ DOI ] [ PubMed ] [ Google Scholar ]
• 60. Abdelazeem B, Abbas KS, Amin MA, et al. The effectiveness of incentives for research participation: a systematic review and meta‐analysis of randomized controlled trials. PLoS One. 2022;17(4):e0267534. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 61. Hashem H, Abufaraj M, Tbakhi A, Sultan I. Obstacles and considerations related to clinical trial research during the COVID‐19 pandemic. Front Med. 2020;7:598038. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 62. Panda PK, Stockler MR, Gulia A. Clinical research during coronavirus disease pandemic: challenges and way forward. Indian J Med Sci. 2020;72(2):101‐106. [ Google Scholar ]
• 63. Janiaud P, Hemkens LG, Ioannidis JPA. Challenges and lessons learned from COVID‐19 trials: should we be doing clinical trials differently? Can J Cardiol. 2021;37(9):1353‐1364. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 64. Hasan MM, Islam MU, Sadeq MJ, Fung W‐K, Uddin J. Review on the evaluation and development of artificial intelligence for COVID‐19 containment. Sensors. 2023;23(1):527. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 65. Wise J. Covid‐19: WHO declares end of global health emergency. Br Med J . 2023. [ DOI ] [ PubMed ]
• 66. Organization WH Guidance for managing ethical issues in infectious disease outbreaks. 2016.
• 67. Carazo Perez S, Folkesson E, Anglaret X, et al. Challenges in preparing and implementing a clinical trial at field level in an Ebola emergency: A case study in Guinea, West Africa. PLoS Neglected Trop Dis. 2017;11(6):e0005545. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 68. Guan W, Ni Z, Hu Y, et al. Clinical characteristics of coronavirus disease 2019 in China. N Engl J Med. 2020;382(18):1708‐1720. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 69. Sissoko D, Laouenan C, Folkesson E, et al. Experimental treatment with favipiravir for Ebola virus disease (the JIKI Trial): a historically controlled, single‐arm proof‐of‐concept trial in Guinea. PLoS Med. 2016;13(3):e1001967. 10.1371/journal.pmed.1001967 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 70. Kupferschmidt K, Cohen J. WHO launches global megatrial of the four most promising coronavirus treatments. Science . 2020;22(3). [ Google Scholar ]
• 71. Khoza‐Shangase K, Monroe N, Sebothoma B. Conducting clinical research in the era of the COVID‐19 pandemic: challenges and lessons for speech‐language pathology and audiology research. S Afr J Commun Disord. 2022;69(2):898. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 72. Tran MC, Labrique AB, Mehra S, et al. Analyzing the mobile “digital divide”: changing determinants of household phone ownership over time in rural Bangladesh. JMIR Mhealth Uhealth. 2015;3(1):e24. 10.2196/mhealth.3663 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 73. Greenleaf AR, Gibson DG, Khattar C, Labrique AB, Pariyo GW. Building the evidence base for remote data collection in low‐and middle‐income countries: comparing reliability and accuracy across survey modalities. J Med Internet Res. 2017;19(5):e140. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 74. Heckathorn DD. Respondent‐driven sampling: a new approach to the study of hidden populations. Soc Probl. 1997;44(2):174‐199. [ Google Scholar ]
• 75. Johnston LG, Sabin K. Sampling hard‐to‐reach populations with respondent driven sampling. Methodol Innov. 2010;5(2):38‐48. [ Google Scholar ]
• 76. Fielding NG, Blank G, Lee RM. The SAGE handbook of online research methods. SAGE Publications Ltd; 2016;684. [ Google Scholar ]
• 77. Dabalen A, Etang A, Hoogeveen J, Mushi E, Schipper Y, von Engelhardt J. Mobile phone panel surveys in developing countries: a practical guide for microdata collection. World Bank Publications; 2016. [ Google Scholar ]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Data availability statement.

• View on publisher site
• PDF (1.3 MB)
• Collections

Similar articles

Cited by other articles, links to ncbi databases.

• Download .nbib .nbib
• Format: AMA APA MLA NLM

Add to Collections

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Published: 16 June 2020

COVID-19 impact on research, lessons learned from COVID-19 research, implications for pediatric research

• Debra L. Weiner 1 , 2 ,
• Vivek Balasubramaniam 3 ,
• Shetal I. Shah 4 &
• Joyce R. Javier 5 , 6

on behalf of the Pediatric Policy Council

Pediatric Research volume  88 ,  pages 148–150 ( 2020 ) Cite this article

148k Accesses

87 Citations

19 Altmetric

Metrics details

The COVID-19 pandemic has resulted in unprecedented research worldwide. The impact on research in progress at the time of the pandemic, the importance and challenges of real-time pandemic research, and the importance of a pediatrician-scientist workforce are all highlighted by this epic pandemic. As we navigate through and beyond this pandemic, which will have a long-lasting impact on our world, including research and the biomedical research enterprise, it is important to recognize and address opportunities and strategies for, and challenges of research and strengthening the pediatrician-scientist workforce.

The first cases of what is now recognized as SARS-CoV-2 infection, termed COVID-19, were reported in Wuhan, China in December 2019 as cases of fatal pneumonia. By February 26, 2020, COVID-19 had been reported on all continents except Antarctica. As of May 4, 2020, 3.53 million cases and 248,169 deaths have been reported from 210 countries. 1

Impact of COVID-19 on ongoing research

The impact on research in progress prior to COVID-19 was rapid, dramatic, and no doubt will be long term. The pandemic curtailed most academic, industry, and government basic science and clinical research, or redirected research to COVID-19. Most clinical trials, except those testing life-saving therapies, have been paused, and most continuing trials are now closed to new enrollment. Ongoing clinical trials have been modified to enable home administration of treatment and virtual monitoring to minimize participant risk of COVID-19 infection, and to avoid diverting healthcare resources from pandemic response. In addition to short- and long-term patient impact, these research disruptions threaten the careers of physician-scientists, many of whom have had to shift efforts from research to patient care. To protect research in progress, as well as physician-scientist careers and the research workforce, ongoing support is critical. NIH ( https://grants.nih.gov/policy/natural-disasters/corona-virus.htm ), PCORI ( https://www.pcori.org/funding-opportunities/applicant-and-awardee-faqs-related-covid-19 ), and other funders acted swiftly to provide guidance on proposal submission and award management, and implement allowances that enable grant personnel to be paid and time lines to be relaxed. Research institutions have also implemented strategies to mitigate the long-term impact of research disruptions. Support throughout and beyond the pandemic to retain currently well-trained research personnel and research support teams, and to accommodate loss of research assets, including laboratory supplies and study participants, will be required to complete disrupted research and ultimately enable new research.

In the long term, it is likely that the pandemic will force reallocation of research dollars at the expense of research areas funded prior to the pandemic. It will be more important than ever for the pediatric research community to engage in discussion and decisions regarding prioritization of funding goals for dedicated pediatric research and meaningful inclusion of children in studies. The recently released 2020 National Institute of Child Health and Development (NICHD) strategic plan that engaged stakeholders, including scientists and patients, to shape the goals of the Institute, will require modification to best chart a path toward restoring normalcy within pediatric science.

COVID-19 research

This global pandemic once again highlights the importance of research, stable research infrastructure, and funding for public health emergency (PHE)/disaster preparedness, response, and resiliency. The stakes in this worldwide pandemic have never been higher as lives are lost, economies falter, and life has radically changed. Ultimate COVID-19 mitigation and crisis resolution is dependent on high-quality research aligned with top priority societal goals that yields trustworthy data and actionable information. While the highest priority goals are treatment and prevention, biomedical research also provides data critical to manage and restore economic and social welfare.

Scientific and technological knowledge and resources have never been greater and have been leveraged globally to perform COVID-19 research at warp speed. The number of studies related to COVID-19 increases daily, the scope and magnitude of engagement is stunning, and the extent of global collaboration unprecedented. On January 5, 2020, just weeks after the first cases of illness were reported, the genetic sequence, which identified the pathogen as a novel coronavirus, SARS-CoV-2, was released, providing information essential for identifying and developing treatments, vaccines, and diagnostics. As of May 3, 2020 1133 COVID-19 studies, including 148 related to hydroxychloroquine, 13 to remdesivir, 50 to vaccines, and 100 to diagnostic testing, were registered on ClinicalTrials.gov, and 980 different studies on the World Health Organization’s International Clinical Trials Registry Platform (WHO ICTRP), made possible, at least in part, by use of data libraries to inform development of antivirals, immunomodulators, antibody-based biologics, and vaccines. On April 7, 2020, the FDA launched the Coronavirus Treatment Acceleration Program (CTAP) ( https://www.fda.gov/drugs/coronavirus-covid-19-drugs/coronavirus-treatment-acceleration-program-ctap ). On April 17, 2020, NIH announced a partnership with industry to expedite vaccine development ( https://www.nih.gov/news-events/news-releases/nih-launch-public-private-partnership-speed-covid-19-vaccine-treatment-options ). As of May 1, 2020, remdesivir (Gilead), granted FDA emergency use authorization, is the only approved therapeutic for COVID-19. 2

The pandemic has intensified research challenges. In a rush for data already thousands of manuscripts, news reports, and blogs have been published, but to date, there is limited scientifically robust data. Some studies do not meet published clinical trial standards, which now include FDA’s COVID-19-specific standards, 3 , 4 , 5 and/or are published without peer review. Misinformation from studies diverts resources from development and testing of more promising therapeutic candidates and has endangered lives. Ibuprofen, initially reported as unsafe for patients with COVID-19, resulted in a shortage of acetaminophen, endangering individuals for whom ibuprofen is contraindicated. Hydroxychloroquine initially reported as potentially effective for treatment of COVID-19 resulted in shortages for patients with autoimmune diseases. Remdesivir, in rigorous trials, showed decrease in duration of COVID-19, with greater effect given early. 6 Given the limited availability and safety data, the use outside clinical trials is currently approved only for severe disease. Vaccines typically take 10–15 years to develop. As of May 3, 2020, of nearly 100 vaccines in development, 8 are in trial. Several vaccines are projected to have emergency approval within 12–18 months, possibly as early as the end of the year, 7 still an eternity for this pandemic, yet too soon for long-term effectiveness and safety data. Antibody testing, necessary for diagnosis, therapeutics, and vaccine testing, has presented some of the greatest research challenges, including validation, timing, availability and prioritization of testing, interpretation of test results, and appropriate patient and societal actions based on results. 8 Relaxing physical distancing without data regarding test validity, duration, and strength of immunity to different strains of COVID-19 could have catastrophic results. Understanding population differences and disparities, which have been further exposed during this pandemic, is critical for response and long-term pandemic recovery. The “Equitable Data Collection and Disclosure on COVID-19 Act” calls for the CDC (Centers for Disease Control and Prevention) and other HHS (United States Department of Health & Human Services) agencies to publicly release racial and demographic information ( https://bass.house.gov/sites/bass.house.gov/files/Equitable%20Data%20Collection%20and%20Dislosure%20on%20COVID19%20Act_FINAL.pdf )

Trusted sources of up-to-date, easily accessible information must be identified (e.g., WHO https://www.who.int/emergencies/diseases/novel-coronavirus-2019/global-research-on-novel-coronavirus-2019-ncov , CDC https://www.cdc.gov/coronavirus/2019-nCoV/hcp/index.html , and for children AAP (American Academy of Pediatrics) https://www.aappublications.org/cc/covid-19 ) and should comment on quality of data and provide strategies and crisis standards to guide clinical practice.

Long-term, lessons learned from research during this pandemic could benefit the research enterprise worldwide beyond the pandemic and during other PHE/disasters with strategies for balancing multiple novel approaches and high-quality, time-efficient, cost-effective research. This challenge, at least in part, can be met by appropriate study design, collaboration, patient registries, automated data collection, artificial intelligence, data sharing, and ongoing consideration of appropriate regulatory approval processes. In addition, research to develop and evaluate innovative strategies and technologies to improve access to care, management of health and disease, and quality, safety, and cost effectiveness of care could revolutionize healthcare and healthcare systems. During PHE/disasters, crisis standards for research should be considered along with ongoing and just-in-time PHE/disaster training for researchers willing to share information that could be leveraged at time of crisis. A dedicated funded core workforce of PHE/disaster researchers and funded infrastructure should be considered, potentially as a consortium of networks, that includes physician-scientists, basic scientists, social scientists, mental health providers, global health experts, epidemiologists, public health experts, engineers, information technology experts, economists and educators to strategize, consult, review, monitor, interpret studies, guide appropriate clinical use of data, and inform decisions regarding effective use of resources for PHE/disaster research.

Differences between adult and pediatric COVID-19, the need for pediatric research

As reported by the CDC, from February 12 to April 2, 2020, of 149,760 cases of confirmed COVID-19 in the United States, 2572 (1.7%) were children aged <18 years, similar to published rates in China. 9 Severe illness has been rare. Of 749 children for whom hospitalization data is available, 147 (20%) required hospitalization (5.7% of total children), and 15 of 147 required ICU care (2.0%, 0.58% of total). Of the 95 children aged <1 year, 59 (62%) were hospitalized, and 5 (5.3%) required ICU admission. Among children there were three deaths. Despite children being relatively spared by COVID-19, spread of disease by children, and consequences for their health and pediatric healthcare are potentially profound with immediate and long-term impact on all of society.

We have long been aware of the importance and value of pediatric research on children, and society. COVID-19 is no exception and highlights the imperative need for a pediatrician-scientist workforce. Understanding differences in epidemiology, susceptibility, manifestations, and treatment of COVID-19 in children can provide insights into this pathogen, pathogen–host interactions, pathophysiology, and host response for the entire population. Pediatric clinical registries of COVID-infected, COVID-exposed children can provide data and specimens for immediate and long-term research. Of the 1133 COVID-19 studies on ClinicalTrials.gov, 202 include children aged ≤17 years. Sixty-one of the 681 interventional trials include children. With less diagnostic testing and less pediatric research, we not only endanger children, but also adults by not identifying infected children and limiting spread by children.

Pediatric considerations and challenges related to treatment and vaccine research for COVID-19 include appropriate dosing, pediatric formulation, and pediatric specific short- and long-term effectiveness and safety. Typically, initial clinical trials exclude children until safety has been established in adults. But with time of the essence, deferring pediatric research risks the health of children, particularly those with special needs. Considerations specific to pregnant women, fetuses, and neonates must also be addressed. Childhood mental health in this demographic, already struggling with a mental health pandemic prior to COVID-19, is now further challenged by social disruption, food and housing insecurity, loss of loved ones, isolation from friends and family, and exposure to an infodemic of pandemic-related information. Interestingly, at present mental health visits along with all visits to pediatric emergency departments across the United States are dramatically decreased. Understanding factors that mitigate and worsen psychiatric symptoms should be a focus of research, and ideally will result in strategies for prevention and management in the long term, including beyond this pandemic. Social well-being of children must also be studied. Experts note that the pandemic is a perfect storm for child maltreatment given that vulnerable families are now socially isolated, facing unemployment, and stressed, and that children are not under the watch of mandated reporters in schools, daycare, and primary care. 10 Many states have observed a decrease in child abuse reports and an increase in severity of emergency department abuse cases. In the short term and long term, it will be important to study the impact of access to care, missed care, and disrupted education during COVID-19 on physical and cognitive development.

Training and supporting pediatrician-scientists, such as through NIH physician-scientist research training and career development programs ( https://researchtraining.nih.gov/infographics/physician-scientist ) at all stages of career, as well as fostering research for fellows, residents, and medical students willing to dedicate their research career to, or at least understand implications of their research for, PHE/disasters is important for having an ongoing, as well as a just-in-time surge pediatric-focused PHE/disaster workforce. In addition to including pediatric experts in collaborations and consortiums with broader population focus, consideration should be given to pediatric-focused multi-institutional, academic, industry, and/or government consortiums with infrastructure and ongoing funding for virtual training programs, research teams, and multidisciplinary oversight.

The impact of the COVID-19 pandemic on research and research in response to the pandemic once again highlights the importance of research, challenges of research particularly during PHE/disasters, and opportunities and resources for making research more efficient and cost effective. New paradigms and models for research will hopefully emerge from this pandemic. The importance of building sustained PHE/disaster research infrastructure and a research workforce that includes training and funding for pediatrician-scientists and integrates the pediatrician research workforce into high-quality research across demographics, supports the pediatrician-scientist workforce and pipeline, and benefits society.

Johns Hopkins Coronavirus Resource Center. Covid-19 Case Tracker. Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (JHU). https://coronavirus.jhu.edu/map.html (2020).

US Food and Drug Administration. Coronavirus (COVID-19) update: FDA issues emergency use authorization for potential COVID-19 treatment. FDA News Release . https://www.fda.gov/news-events/press-announcements/coronavirus-covid-19-update-fda-issues-emergency-use-authorization-potential-covid-19-treatment (2020).

Evans, S. R. Fundamentals of clinical trial design. J. Exp. Stroke Transl. Med. 3 , 19–27 (2010).

Article   Google Scholar  

Antman, E. M. & Bierer, B. E. Standards for clinical research: keeping pace with the technology of the future. Circulation 133 , 823–825 (2016).

Food and Drug Administration. FDA guidance on conduct of clinical trials of medical products during COVID-19 public health emergency. Guidance for Industry, Investigators and Institutional Review Boards . https://www.fda.gov/regulatory-information/search-fda-guidance-documents/fda-guidance-conduct-clinical-trials-medical-products-during-covid-19-public-health-emergency (2020).

National Institutes of Health. NIH clinical trials shows remdesivir accelerates recovery from advanced COVID-19. NIH New Releases . https://www.nih.gov/news-events/news-releases/nih-clinical-trial-shows-remdesivir-accelerates-recovery-advanced-covid-19#.XrIX75ZmQeQ.email (2020).

Radcliffe, S. Here’s exactly where we are with vaccines and treatments for COVID-19. Health News . https://www.healthline.com/health-news/heres-exactly-where-were-at-with-vaccines-and-treatments-for-covid-19 (2020).

Abbasi, J. The promise and peril of antibody testing for COVID-19. JAMA . https://doi.org/10.1001/jama.2020.6170 (2020).

CDC COVID-19 Response Team. Coronavirus disease 2019 in children—United States, February 12–April 2, 2020. Morb. Mortal Wkly Rep . 69 , 422–426 (2020).

Agarwal, N. Opinion: the coronavirus could cause a child abuse epidemic. The New York Times . https://www.nytimes.com/2020/04/07/opinion/coronavirus-child-abuse.html (2020).

Download references

Author information

Authors and affiliations.

Department of Pediatrics, Division of Emergency Medicine, Boston Children’s Hospital, Boston, MA, USA

Debra L. Weiner

Harvard Medical School, Boston, MA, USA

Department of Pediatrics, University of Wisconsin School of Medicine and Public Health, Madison, WI, USA

Vivek Balasubramaniam

Department of Pediatrics and Division of Neonatology, Maria Fareri Children’s Hospital at Westchester Medical Center, New York Medical College, Valhalla, NY, USA

Shetal I. Shah

Division of General Pediatrics, Children’s Hospital Los Angeles, Los Angeles, CA, USA

Joyce R. Javier

Keck School of Medicine, University of Southern California, Los Angeles, CA, USA

You can also search for this author in PubMed   Google Scholar

Contributions

All authors made substantial contributions to conception and design, data acquisition and interpretation, drafting the manuscript, and providing critical revisions. All authors approve this final version of the manuscript.

Pediatric Policy Council

Scott C. Denne, MD, Chair, Pediatric Policy Council; Mona Patel, MD, Representative to the PPC from the Academic Pediatric Association; Jean L. Raphael, MD, MPH, Representative to the PPC from the Academic Pediatric Association; Jonathan Davis, MD, Representative to the PPC from the American Pediatric Society; DeWayne Pursley, MD, MPH, Representative to the PPC from the American Pediatric Society; Tina Cheng, MD, MPH, Representative to the PPC from the Association of Medical School Pediatric Department Chairs; Michael Artman, MD, Representative to the PPC from the Association of Medical School Pediatric Department Chairs; Shetal Shah, MD, Representative to the PPC from the Society for Pediatric Research; Joyce Javier, MD, MPH, MS, Representative to the PPC from the Society for Pediatric Research.

Corresponding author

Correspondence to Debra L. Weiner .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Members of the Pediatric Policy Council are listed below Author contributions.

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Weiner, D.L., Balasubramaniam, V., Shah, S.I. et al. COVID-19 impact on research, lessons learned from COVID-19 research, implications for pediatric research. Pediatr Res 88 , 148–150 (2020). https://doi.org/10.1038/s41390-020-1006-3

Download citation

Received : 07 May 2020

Accepted : 21 May 2020

Published : 16 June 2020

Issue Date : August 2020

DOI : https://doi.org/10.1038/s41390-020-1006-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Federated learning as a smart tool for research on infectious diseases.

• Laura C. Zwiers
• Diederick E. Grobbee
• David S. Y. Ong

BMC Infectious Diseases (2024)

Catalysing global surgery: a meta-research study on factors affecting surgical research collaborations with Africa

• Thomas O. Kirengo
• Hussein Dossajee
• Nchafatso G. Obonyo

Systematic Reviews (2024)

Lessons learnt while designing and conducting a longitudinal study from the first Italian COVID-19 pandemic wave up to 3 years

• Alvisa Palese
• Stefania Chiappinotto
• Carlo Tascini

Health Research Policy and Systems (2023)

Pediatric Research and COVID-19: the changed landscape

• E. J. Molloy
• C. B. Bearer

Pediatric Research (2022)

Cancer gene therapy 2020: highlights from a challenging year

• Georgios Giamas
• Teresa Gagliano

Cancer Gene Therapy (2022)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

The impact of the COVID-19 pandemic on scientific research in the life sciences

Roles Conceptualization, Formal analysis, Methodology, Writing – original draft, Writing – review & editing

Affiliation AXES, IMT School for Advanced Studies Lucca, Lucca, Italy

Roles Conceptualization, Data curation, Formal analysis, Methodology, Software, Visualization, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliation Chair of Systems Design D-MTEC, ETH Zürich, Zurich, Switzerland

• Massimo Riccaboni, 
• Luca Verginer

• Published: February 9, 2022
• https://doi.org/10.1371/journal.pone.0263001
• Reader Comments

The COVID-19 outbreak has posed an unprecedented challenge to humanity and science. On the one side, public and private incentives have been put in place to promptly allocate resources toward research areas strictly related to the COVID-19 emergency. However, research in many fields not directly related to the pandemic has been displaced. In this paper, we assess the impact of COVID-19 on world scientific production in the life sciences and find indications that the usage of medical subject headings (MeSH) has changed following the outbreak. We estimate through a difference-in-differences approach the impact of the start of the COVID-19 pandemic on scientific production using the PubMed database (3.6 Million research papers). We find that COVID-19-related MeSH terms have experienced a 6.5 fold increase in output on average, while publications on unrelated MeSH terms dropped by 10 to 12%. The publication weighted impact has an even more pronounced negative effect (-16% to -19%). Moreover, COVID-19 has displaced clinical trial publications (-24%) and diverted grants from research areas not closely related to COVID-19. Note that since COVID-19 publications may have been fast-tracked, the sudden surge in COVID-19 publications might be driven by editorial policy.

Citation: Riccaboni M, Verginer L (2022) The impact of the COVID-19 pandemic on scientific research in the life sciences. PLoS ONE 17(2): e0263001. https://doi.org/10.1371/journal.pone.0263001

Editor: Florian Naudet, University of Rennes 1, FRANCE

Received: April 28, 2021; Accepted: January 10, 2022; Published: February 9, 2022

Copyright: © 2022 Riccaboni, Verginer. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: The processed data, instructions on how to process the raw PubMed dataset as well as all code are available via Zenodo at https://doi.org/10.5281/zenodo.5121216 .

Funding: The author(s) received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

Introduction

The COVID-19 pandemic has mobilized the world scientific community in 2020, especially in the life sciences [ 1 , 2 ]. In the first three months after the pandemic, the number of scientific papers about COVID-19 was fivefold the number of articles on H1N1 swine influenza [ 3 ]. Similarly, the number of clinical trials related to COVID-19 prophylaxis and treatments skyrocketed [ 4 ]. Thanks to the rapid mobilization of the world scientific community, COVID-19 vaccines have been developed in record time. Despite this undeniable success, there is a rising concern about the negative consequences of COVID-19 on clinical trial research, with many projects being postponed [ 5 – 7 ]. According to Evaluate Pharma, clinical trials were one of the pandemic’s first casualties, with a record number of 160 studies suspended for reasons related to COVID-19 in April 2020 [ 8 , 9 ] reporting a total of 1,200 trials suspended as of July 2020. As a consequence, clinical researchers have been impaired by reduced access to healthcare research infrastructures. Particularly, the COVID-19 outbreak took a tall on women and early-career scientists [ 10 – 13 ]. On a different ground, Shan and colleagues found that non-COVID-19-related articles decreased as COVID-19-related articles increased in top clinical research journals [ 14 ]. Fraser and coworker found that COVID-19 preprints received more attention and citations than non-COVID-19 preprints [ 1 ]. More recently, Hook and Porter have found some early evidence of ‘covidisation’ of academic research, with research grants and output diverted to COVID-19 research in 2020 [ 15 ]. How much should scientists switch their efforts toward SARS-CoV-2 prevention, treatment, or mitigation? There is a growing consensus that the current level of ‘covidisation’ of research can be wasteful [ 4 , 5 , 16 ].

Against this background, in this paper, we investigate if the COVID-19 pandemic has induced a shift in biomedical publications toward COVID-19-related scientific production. The objective of the study is to show that scientific articles listing covid-related Medical Subject Headings (MeSH) when compared against covid-unrelated MeSH have been partially displaced. Specifically, we look at several indicators of scientific production in the life sciences before and after the start of the COVID-19 pandemic: (1) number of papers published, (2) impact factor weighted number of papers, (3) opens access, (4) number of publications related to clinical trials, (5) number of papers listing grants, (6) number of papers listing grants existing before the pandemic. Through a natural experiment approach, we analyze the impact of the pandemic on scientific production in the life sciences. We consider COVID-19 an unexpected and unprecedented exogenous source of variation with heterogeneous effects across biomedical research fields (i.e., MeSH terms).

Based on the difference in difference results, we document the displacement effect that the pandemic has had on several aspects of scientific publishing. The overall picture that emerges from this analysis is that there has been a profound realignment of priorities and research efforts. This shift has displaced biomedical research in fields not related to COVID-19.

The rest of the paper is structured as follows. First, we describe the data and our measure of relatedness to COVID-19. Next, we illustrate the difference-in-differences specification we rely on to identify the impact of the pandemic on scientific output. In the results section, we present the results of the difference-in-differences and network analyses. We document the sudden shift in publications, grants and trials towards COVID-19-related MeSH terms. Finally, we discuss the findings and highlight several policy implications.

Materials and methods

The present analysis is based primarily on PubMed and the Medical Subject Headings (MeSH) terminology. This data is used to estimate the effect of the start of the COVID 19 pandemic via a difference in difference approach. This section is structured as follows. We first introduce the data and then the econometric methodology. This analysis is not based on a pre-registered protocol.

Selection of biomedical publications.

We rely on PubMed, a repository with more than 34 million biomedical citations, for the analysis. Specifically, we analyze the daily updated files up to 31/06/2021, extracting all publications of type ‘Journal Article’. For the principal analysis, we consider 3,638,584 papers published from January 2019 to December 2020. We also analyze 11,122,017 papers published from 2010 onwards to identify the earliest usage of a grant and infer if it was new in 2020. We use the SCImago journal ranking statistics to compute the impact factor weighted number (IFWN) of papers in a given field of research. To assign the publication date, we use the ‘electronically published’ dates and, if missing, the ‘print published’ dates.

Medical subject headings.

We rely on the Medical Subject Headings (MeSH) terminology to approximate narrowly defined biomedical research fields. This terminology is a curated medical vocabulary, which is manually added to papers in the PubMed corpus. The fact that MeSH terms are manually annotated makes this terminology ideal for classification purposes. However, there is a delay between publication and annotation, on the order of several months. To address this delay and have the most recent classification, we search for all 28 425 MeSH terms using PubMed’s ESearch utility and classify paper by the results. The specific API endpoint is https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi , the relevant scripts are available with the code. For example, we assign the term ‘Ageusia’ (MeSH ID D000370) to all papers listed in the results of the ESearch API. We apply this method to the whole period (January 2019—December 2020) and obtain a mapping from papers to the MeSH terms. For every MeSH term, we keep track of the year they have been established. For instance, COVID-19 terms were established in 2020 (see Table 1 ): in January 2020, the WHO recommended 2019-nCoV and 2019-nCoV acute respiratory disease as provisional names for the virus and disease. The WHO issued the official terms COVID-19 and SARS-CoV-2 at the beginning of February 2020. By manually annotating publications, all publications referring to COVID-19 and SARS-CoV-2 since January 2020 have been labelled with the related MeSH terms. Other MeSH terms related to COVID-19, such as coronavirus, for instance, have been established years before the pandemic (see Table 2 ). We proxy MeSH term usage via search terms using the PubMed EUtilities API; this means that we are not using the hand-labelled MeSH terms but rather the PubMed search results. This means that the accuracy of the MeSH term we assign to a given paper is not perfect. In practice, this means that we have assigned more MeSH terms to a given term than a human annotator would have.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0263001.t001

The list contains only terms with at least 100 publications in 2020.

https://doi.org/10.1371/journal.pone.0263001.t002

Clinical trials and publication types.

We classify publications using PubMed’s ‘PublicationType’ field in the XML baseline files (There are 187 publication types, see https://www.nlm.nih.gov/mesh/pubtypes.html ). We consider a publication to be related to a clinical trial if it lists any of the following descriptors:

• D016430: Clinical Trial
• D017426: Clinical Trial, Phase I
• D017427: Clinical Trial, Phase II
• D017428: Clinical Trial, Phase III
• D017429: Clinical Trial, Phase IV
• D018848: Controlled Clinical Trial
• D065007: Pragmatic Clinical Trial
• D000076362: Adaptive Clinical Trial
• D000077522: Clinical Trial, Veterinary

In our analysis of the impact of COVID-19 on publications related to clinical trials, we only consider MeSH terms that are associated at least once with a clinical trial publication over the two years. We apply this restriction to filter out MeSH terms that are very unlikely to be relevant for clinical trial types of research.

Open access.

We proxy the availability of a journal article to the public, i.e., open access, if it is available from PubMed Central. PubMed Central archives full-text journal articles and provides free access to the public. Note that the copyright license may vary across participating publishers. However, the text of the paper is for all effects and purposes freely available without requiring subscriptions or special affiliation.

We infer if a publication has been funded by checking if it lists any grants. We classify grants as either ‘old’, i.e. existed before 2019, or ‘new’, i.e. first observed afterwards. To do so, we collect all grant IDs for 11,122,017 papers from 2010 on-wards and record their first appearance. This procedure is an indirect inference of the year the grant has been granted. The basic assumption is that if a grant number has not been listed in any publication since 2010, it is very likely a new grant. Specifically, an old grant is a grant listed since 2019 observed at least once from 2010 to 2018.

Note that this procedure is only approximate and has a few shortcomings. Mistyped grant numbers (e.g. ‘1234-M JPN’ and ‘1234-M-JPN’) could appear as new grants, even though they existed before, or new grants might be classified as old grants if they have a common ID (e.g. ‘Grant 1’). Unfortunately, there is no central repository of grant numbers and the associated metadata; however, there are plans to assign DOI numbers to grants to alleviate this problem (See https://gitlab.com/crossref/open_funder_registry for the project).

Impact factor weighted publication numbers (IFWN).

In our analysis, we consider two measures of scientific output. First, we simply count the number of publications by MeSH term. However, since journals vary considerably in terms of impact factor, we also weigh the number of publications by the impact factor of the venue (e.g., journal) where it was published. Specifically, we use the SCImago journal ranking statistics to weigh a paper by the impact factor of the journal it appears in. We use the ‘citation per document in the past two years’ for 45,230 ISSNs. Note that a journal may and often has more than one ISSN, i.e., one for the printed edition and one for the online edition. SCImago applies the same score for a venue across linked ISSNs.

For the impact factor weighted number (IFWN) of publication per MeSH terms, this means that all publications are replaced by the impact score of the journal they appear in and summed up.

COVID-19-relatedness.

To measure how closely related to COVID-19 is a MeSH term, we introduce an index of relatedness to COVID-19. First, we identify the focal COVID-19 terms, which appeared in the literature in 2020 (see Table 1 ). Next, for all other pre-existing MeSH terms, we measure how closely related to COVID-19 they end up being.

Our aim is to show that MeSH terms that existed before and are related have experienced a sudden increase in the number of (impact factor weighted) papers.

Intuitively we can read this measure as: what is the probability in 2020 that a COVID-19 MeSH term is present given that we chose a paper with MeSH term i ? For example, given that in 2020 we choose a paper dealing with “Ageusia” (i.e., Complete or severe loss of the subjective sense of taste), there is a 96% probability that this paper also lists COVID-19, see Table 1 .

Note that a paper listing a related MeSH term does not imply that that paper is doing COVID-19 research, but it implies that one of the MeSH terms listed is often used in COVID-19 research.

In sum, in our analysis, we use the following variables:

• Papers: Number of papers by MeSH term;
• Impact: Impact factor weighted number of papers by MeSH term;
• PMC: Papers listed in PubMed central by MeSH term, as a measure of Open Access publications;
• Trials: number of publications of type “Clinical Trial” by MeSH term;
• Grants: number of papers with at least one grant by MeSH term;
• Old Grants: number of papers listing a grant that has been observed between 2010 and 2018, by MeSH term;

Difference-in-differences

The difference-in-differences (DiD) method is an econometric technique to imitate an experimental research design from observation data, sometimes referred to as a quasi-experimental setup. In a randomized controlled trial, subjects are randomly assigned either to the treated or the control group. Analogously, in this natural experiment, we assume that medical subject headings (MeSH) have been randomly assigned to be either treated (related) or not treated (unrelated) by the pandemic crisis.

Before the COVID, for a future health crisis, the set of potentially impacted medical knowledge was not predictable since it depended on the specifics of the emergency. For instance, ageusia (loss of taste), a medical concept existing since 1991, became known to be a specific symptom of COVID-19 only after the pandemic.

Specifically, we exploit the COVID-19 as an unpredictable and exogenous shock that has deeply affected the publication priorities for biomedical scientific production, as compared to the situation before the pandemic. In this setting, COVID-19 is the treatment, and the identification of this new human coronavirus is the event. We claim that treated MeSH terms, i.e., MeSH terms related to COVID-19, have experienced a sudden increase in terms of scientific production and attention. In contrast, research on untreated MeSH terms, i.e., MeSH terms not related to COVID-19, has been displaced by COVID-19. Our analysis compares the scientific output of COVID-19 related and unrelated MeSH terms before and after January 2020.

In our case, some of the terms turn out to be related to COVID-19 in 2020, whereas most of the MeSH terms are not closely related to COVID-19.

Thus β 1 identifies the overall effect on the control group after the event, β 2 the difference across treated and control groups before the event (i.e. the first difference in DiD) and finally the effect on the treated group after the event, net of the first difference, β 3 . This last parameter identifies the treatment effect on the treated group netting out the pre-treatment difference.

For the DiD to have a causal interpretation, it must be noted that pre-event, the trends of the two groups should be parallel, i.e., the common trend assumption (CTA) must be satisfied. We will show that the CTA holds in the results section.

To specify the DiD model, we need to define a period before and after the event and assign a treatment status or level of exposure to each term.

Before and after.

The pre-treatment period is defined as January 2019 to December 2019. The post-treatment period is defined as the months from January 2020 to December 2020. We argue that the state of biomedical research was similar in those two years, apart from the effect of the pandemic.

Treatment status and exposure.

The treatment is determined by the COVID-19 relatedness index σ i introduced earlier. Specifically, this number indicates the likelihood that COVID-19 will be a listed MeSH term, given that we observe the focal MeSH term i . To show that the effect becomes even stronger the closer related the subject is, and for ease of interpretation, we also discretize the relatedness value into three levels of treatment. Namely, we group MeSH terms with a σ between, 0% to 20%, 20% to 80% and 80% to 100%. The choice of alternative grouping strategies does not significantly affect our results. Results for alternative thresholds of relatedness can be computed using the available source code. We complement the dichotomized analysis by using the treatment intensity (relatedness measure σ ) to show that the result persists.

Panel regression.

In this work, we estimate a random effects panel regression where the units of analysis are 28 318 biomedical research fields (i.e. MeSH terms) observed over time before and after the COVID-19 pandemic. The time resolution is at the monthly level, meaning that for each MeSH term, we have 24 observations from January 2019 to December 2020.

The outcome variable Y it identifies the outcome at time t (i.e., month), for MeSH term i . As before, P t identifies the period with P t = 0 if the month is before January 2020 and P t = 1 if it is on or after this date. In (3) , the treatment level is measure by the relatedness to COVID-19 ( σ i ), where again the γ 1 identifies pre-trend (constant) differences and δ 1 the overall effect.

In total, we estimate six coefficients. As before, the δ l coefficient identifies the DiD effect.

Verifying the Common Trend Assumption (CTA).

We show that the CTA holds for this model by comparing the pre-event trends of the control group to the treated groups (COVID-19 related MeSH terms). Namely, we show that the pre-event trends of the control group are the same as the pre-event trends of the treated group.

Co-occurrence analysis

To investigate if the pandemic has caused a reconfiguration of research priorities, we look at the MeSH term co-occurrence network. Precisely, we extract the co-occurrence network of all 28,318 MeSH terms as they appear in the 3.3 million papers. We considered the co-occurrence networks of 2018, 2019 and 2020. Each node represents a MeSH term in these networks, and a link between them indicates that they have been observed at least once together. The weight of the edge between the MeSH terms is given by the number of times those terms have been jointly observed in the same publications.

Medical language is hugely complicated, and this simple representation does not capture the intricacies, subtle nuances and, in fact, meaning of the terms. Therefore, we do not claim that we can identify how the actual usage of MeSH terms has changed from this object, but rather that it has. Nevertheless, the co-occurrence graph captures rudimentary relations between concepts. We argue that absent a shock to the system, their basic usage patterns, change in importance (within the network) would essentially be the same from year to year. However, if we find that the importance of terms changes more than expected in 2020, it stands to reason that there have been some significant changes.

To show that that MeSH usage has been affected, we compute for each term in the years 2018, 2019 and 2020 their PageRank centrality [ 17 ]. The PageRank centrality tells us how likely a random walker traversing a network would be found at a given node if she follows the weights of the empirical edges (i.e., co-usage probability). Specifically, for the case of the MeSH co-occurrence network, this number represents how often an annotator at the National Library of Medicine would assign that MeSH term following the observed general usage patterns. It is a simplistic measure to capture the complexities of biomedical research. Nevertheless, it captures far-reaching interdependence across MeSH terms as the measure uses the whole network to determine the centrality of every MeSH term. A sudden change in the rankings and thus the position of MeSH terms in this network suggests that a given research subject has risen as it is used more often with other important MeSH terms (or vice versa).

We then compare the growth for each MeSH i term in g i (2019), i.e. before the the COVID-19 pandemic, with the growth after the event ( g i (2020)).

Publication growth

Changes in output and COVID-19 relatedness

Before we show the regression results, we provide descriptive evidence that publications from 2019 to 2020 have drastically increased. By showing that this growth correlates strongly with a MeSH term’s COVID-19 relatedness ( σ ), we demonstrate that (1) σ captures an essential aspect of the growth dynamics and (2) highlight the meteoric rise of highly related terms.

We look at the year over year growth in the number of the impact weighted number of publications per MeSH term from 2018 to 2019 and 2019 to 2020 as defined in the methods section.

Fig 1 shows the yearly growth of the impact weighted number of publications per MeSH term. By comparing the growth of the number of publications from the years 2018, 2019 and 2020, we find that the impact factor weighted number of publications has increased by up to a factor of 100 compared to the previous year for Betacoronavirus, one of the most closely related to COVID-19 MeSH term.

Each dot represents, a MeSH term. The y axis (growth) is in symmetric log scale. The x axis shows the COVID-19 relatedness, σ . Note that the position of the dots on the x-axis is the same in the two plots. Below: MeSH term importance gain (PageRank) and their COVID-19 relatedness.

https://doi.org/10.1371/journal.pone.0263001.g001

Fig 1 , first row, reveals how strongly correlated the growth in the IFWN of publication is to the term’s COVID-19 relatedness. For instance, we see that the term ‘Betacoronavirus’ skyrocketed from 2019 to 2020, which is expected given that SARS-CoV-2 is a species of the genus. Conversely, the term ‘Alphacoronavirus’ has not experienced any growth given that it is twin a genus of the Coronaviridae family, but SARS-CoV-2 is not one of its species. Note also the fast growth in the number of publications dealing with ‘Quarantine’. Moreover, MeSH terms that grew significantly from 2018 to 2019 and were not closely related to COVID-19, like ‘Vaping’, slowed down in 2020. From the graph, the picture emerges that publication growth is correlated with COVID-19 relatedness σ and that the growth for less related terms slowed down.

To show that the usage pattern of MeSH terms has changed following the pandemic, we compute the PageRank centrality using graph-tool [ 18 ] as discussed in the Methods section.

Fig 1 , second row, shows the change in the PageRank centrality of the MeSH terms after the pandemic (2019 to 2020, right plot) and before (2018 to 2019, left plot). If there were no change in the general usage pattern, we would expect the variance in PageRank changes to be narrow across the two periods, see (left plot). However, PageRank scores changed significantly more from 2019 to 2020 than from 2018 to 2019, suggesting that there has been a reconfiguration of the network.

To further support this argument, we carry out a DiD regression analysis.

Common trends assumption

As discussed in the Methods section, we need to show that the CTA assumption holds for the DiD to be defined appropriately. We do this by estimating for each month the number of publications and comparing it across treatment groups. This exercise also serves the purpose of a placebo test. By assuming that each month could have potentially been the event’s timing (i.e., the outbreak), we show that January 2020 is the most likely timing of the event. The regression table, as noted earlier, contains over 70 estimated coefficients, hence for ease of reading, we will only show the predicted outcome per month by group (see Fig 2 ). The full regression table with all coefficients is available in the S1 Table .

The y axis is in log scale. The dashed vertical line identifies January 2020. The dashed horizontal line shows the publications in January 2019 for the 0–20% group before the event. This line highlights that the drop happens after the event. The bands around the lines indicate the 95% confidence interval of the predicted values. The results are the output of the Stata margins command.

https://doi.org/10.1371/journal.pone.0263001.g002

Fig 2 shows the predicted number per outcome variable obtained from the panel regression model. These predictions correspond to the predicted value per relatedness group using the regression parameters estimated via the linear panel regression. The bands around the curves are the 95% confidence intervals.

All outcome measures depict a similar trend per month. Before the event (i.e., January 2020), there is a common trend across all groups. In contrast, after the event, we observe a sudden rise for the outcomes of the COVID-19 related treated groups (green and red lines) and a decline in the outcomes for the unrelated group (blue line). Therefore, we can conclude that the CTA assumption holds.

Regression results

Table 3 shows the DiD regression results (see Eq (3) ) for the selected outcome measures: number of publications (Papers), impact factor weighted number of publications (Impact), open access (OA) publications, clinical trial related publications, and publications with existing grants.

https://doi.org/10.1371/journal.pone.0263001.t003

Table 3 shows results for the discrete treatment level version of the DiD model (see Eq (4) ).

Note that the outcome variable is in natural log scale; hence to get the effect of the independent variable, we need to exponentiate the coefficient. For values close to 0, the effect is well approximated by the percentage change of that magnitude.

In both specifications we see that the least related group, drops in the number of publications between 10% and 13%, respectively (first row of Tables 3 and 4 , exp(−0.102) ≈ 0.87). In line with our expectations, the increase in the number of papers published by MeSH term is positively affected by the relatedness to COVID-19. In the discrete model (row 2), we note that the number of documents with MeSH terms with a COVID-19 relatedness between 20 and 80% grows by 18% and highly related terms by a factor of approximately 6.6 (exp(1.88)). The same general pattern can be observed for the impact weighted publication number, i.e., Model (2). Note, however, that the drop in the impact factor weighted output is more significant, reaching -19% for COVID-19 unrelated publications, and related publications growing by a factor of 8.7. This difference suggests that there might be a bias to publish papers on COVID-19 related subjects in high impact factor journals.

https://doi.org/10.1371/journal.pone.0263001.t004

By looking at the number of open access publications (PMC), we note that the least related group has not been affected negatively by the pandemic. However, the number of COVID-19 related publications has drastically increased for the most COVID-19 related group by a factor of 6.2. Note that the substantial increase in the number of papers available through open access is in large part due to journal and editorial policies to make preferentially COVID research immediately available to the public.

Regarding the number of clinical trial publications, we note that the least related group has been affected negatively, with the number of publications on clinical trials dropping by a staggering 24%. At the same time, publications on clinical trials for COVID-19-related MeSH have increased by a factor of 2.1. Note, however, that the effect on clinical trials is not significant in the continuous regression. The discrepancy across Tables 3 and 4 highlights that, especially for trials, the effect is not linear, where only the publications on clinical trials closely related to COVID-19 experiencing a boost.

It has been reported [ 19 ] that while the number of clinical trials registered to treat or prevent COVID-19 has surged with 179 new registrations in the second week of April 2020 alone. Only a few of these have led to publishable results in the 12 months since [ 20 ]. On the other hand, we find that clinical trial publications, considering related MeSH (but not COVID-19 directly), have had significant growth from the beginning of the pandemic. These results are not contradictory. Indeed counting the number of clinical trial publications listing the exact COVID-19 MeSH term (D000086382), we find 212 publications. While this might seem like a small number, consider that in 2020 only 8,485 publications were classified as clinical trials; thus, targeted trials still made up 2.5% of all clinical trials in 2020 . So while one might doubt the effectiveness of these research efforts, it is still the case that by sheer number, they represent a significant proportion of all publications on clinical trials in 2020. Moreover, COVID-19 specific Clinical trial publications in 2020, being a delayed signal of the actual trials, are a lower bound estimate on the true number of such clinical trials being conducted. This is because COVID-19 studies could only have commenced in 2020, whereas other studies had a head start. Thus our reported estimates are conservative, meaning that the true effect on actual clinical trials is likely larger, not smaller.

Research funding, as proxied by the number of publications with grants, follows a similar pattern, but notably, COVID-19-related MeSH terms list the same proportion of grants established before 2019 as other unrelated MeSH terms, suggesting that grants which were not designated for COVID-19 research have been used to support COVID-19 related research. Overall, the number of publications listing a grant has dropped. Note that this should be because the number of publications overall in the unrelated group has dropped. However, we note that the drop in publications is 10% while the decline in publications with at least one grant is 15%. This difference suggests that publications listing grants, which should have more funding, are disproportionately COVID-19 related papers. To further investigate this aspect, we look at whether the grant was old (pre-2019) or appeared for the first time in or after 2019. It stands to reason that an old grant (pre-2019) would not have been granted for a project dealing with the pandemic. Hence we would expect that COVID-19 related MeSH terms to have a lower proportion of old grants than the unrelated group. In models (6) in Table 4 we show that the number of old grants for the unrelated group drops by 13%. At the same time, the number of papers listing old grants (i.e., pre-2019) among the most related group increased by a factor of 3.1. Overall, these results suggest that COVID-19 related research has been funded largely by pre-existing grants, even though a specific mandate tied to the grants for this use is unlikely.

The scientific community has swiftly reallocated research efforts to cope with the COVID-19 pandemic, mobilizing knowledge across disciplines to find innovative solutions in record time. We document this both in terms of changing trends in the biomedical scientific output and the usage of MeSH terms by the scientific community. The flip side of this sudden and energetic prioritization of effort to fight COVID-19 has been a sudden contraction of scientific production in other relevant research areas. All in all, we find strong support to the hypotheses that the COVID-19 crisis has induced a sudden increase of research output in COVID-19 related areas of biomedical research. Conversely, research in areas not related to COVID-19 has experienced a significant drop in overall publishing rates and funding.

Our paper contributes to the literature on the impact of COVID-19 on scientific research: we corroborate previous findings about the surge of COVID-19 related publications [ 1 – 3 ], partially displacing research in COVID-19 unrelated fields of research [ 4 , 14 ], particularly research related to clinical trials [ 5 – 7 ]. The drop in trial research might have severe consequences for patients affected by life-threatening diseases since it will delay access to new and better treatments. We also confirm the impact of COVID-19 on open access publication output [ 1 ]; also, this is milder than traditional outlets. On top of this, we provide more robust evidence on the impact weighted effect of COVID-19 and grant financed research, highlighting the strong displacement effect of COVID-19 on the allocation of financial resources [ 15 ]. We document a substantial change in the usage patterns of MeSH terms, suggesting that there has been a reconfiguration in the way research terms are being combined. MeSH terms highly related to COVID-19 were peripheral in the MeSH usage networks before the pandemic but have become central since 2020. We conclude that the usage patterns have changed, with COVID-19 related MeSH terms occupying a much more prominent role in 2020 than they did in the previous years.

We also contribute to the literature by estimating the effect of COVID-19 on biomedical research in a natural experiment framework, isolating the specific effects of the COVID-19 pandemic on the biomedical scientific landscape. This is crucial to identify areas of public intervention to sustain areas of biomedical research which have been neglected during the COVID-19 crisis. Moreover, the exploratory analysis on the changes in usage patterns of MeSH terms, points to an increase in the importance of covid-related topics in the broader biomedical research landscape.

Our results provide compelling evidence that research related to COVID-19 has indeed displaced scientific production in other biomedical fields of research not related to COVID-19, with a significant drop in (impact weighted) scientific output related to non-COVID-19 and a marked reduction of financial support for publications not related to COVID-19 [ 4 , 5 , 16 ]. The displacement effect is persistent to the end of 2020. As vaccination progresses, we highlight the urgent need for science policy to re-balance support for research activity that was put on pause because of the COVID-19 pandemic.

We find that COVID-19 dramatically impacted clinical research. Reactivation of clinical trials activities that have been postponed or suspended for reasons related to COVID-19 is a priority that should be considered in the national vaccination plans. Moreover, since grants have been diverted and financial incentives have been targeted to sustain COVID-19 research leading to an excessive entry in COVID-19-related clinical trials and the ‘covidisation’ of research, there is a need to reorient incentives to basic research and otherwise neglected or temporally abandoned areas of biomedical research. Without dedicated support in the recovery plans for neglected research of the COVID-19 era, there is a risk that more medical needs will be unmet in the future, possibly exacerbating the shortage of scientific research for orphan and neglected diseases, which do not belong to COVID-19-related research areas.

Limitations

Our empirical approach has some limits. First, we proxy MeSH term usage via search terms using the PubMed EUtilities API. This means that the accuracy of the MeSH term we assign to a given paper is not fully validated. More time is needed for the completion of manually annotated MeSH terms. Second, the timing of publication is not the moment the research has been carried out. There is a lead time between inception, analysis, write-up, review, revision, and final publication. This delay varies across disciplines. Nevertheless, given that the surge in publications happens around the alleged event date, January 2020, we are confident that the publication date is a reasonable yet imperfect estimate of the timing of the research. Third, several journals have publicly declared to fast-track COVID-19 research. This discrepancy in the speed of publication of COVID-19 related research and other research could affect our results. Specifically, a surge or displacement could be overestimated due to a lag in the publication of COVID-19 unrelated research. We alleviate this bias by estimating the effect considering a considerable time after the event (January 2020 to December 2020). Forth, on the one hand, clinical Trials may lead to multiple publications. Therefore we might overestimate the impact of COVID-19 on the number of clinical trials. On the other hand, COVID-19 publications on clinical trials lag behind, so the number of papers related COVID-19 trials is likely underestimated. Therefore, we note that the focus of this paper is scientific publications on clinical trials rather than on actual clinical trials. Fifth, regarding grants, unfortunately, there is no unique centralized repository mapping grant numbers to years, so we have to proxy old grants with grants that appeared in publications from 2010 to 2018. Besides, grant numbers are free-form entries, meaning that PubMed has no validation step to disambiguate or verify that the grant number has been entered correctly. This has the effect of classifying a grant as new even though it has appeared under a different name. We mitigate this problem by using a long period to collect grant numbers and catch many spellings of the same grant, thereby reducing the likelihood of miss-identifying a grant as new when it existed before. Still, unless unique identifiers are widely used, there is no way to verify this.

So far, there is no conclusive evidence on whether entry into COVID-19 has been excessive. However, there is a growing consensus that COVID-19 has displaced, at least temporally, scientific research in COVID-19 unrelated biomedical research areas. Even though it is certainly expected that more attention will be devoted to the emergency during a pandemic, the displacement of biomedical research in other fields is concerning. Future research is needed to investigate the long-run structural consequences of the COVID-19 crisis on biomedical research.

Supporting information

S1 table. common trend assumption (cta) regression table..

Full regression table with all controls and interactions.

https://doi.org/10.1371/journal.pone.0263001.s001

• View Article
• Google Scholar
• PubMed/NCBI
• 8. Brown A, Edwin E, Fagg J. Evaluate Pharma 2021 Preview; 2020. https://www.evaluate.com/thought-leadership/vantage/evaluate-vantage-2021-preview .
• 15. Hook D, Porter S. The COVID Brain Drain; 2020. https://www.natureindex.com/news-blog/covid-brain-drain-scientific-research .
• 17. Page L, Brin S, Motwani R, Winograd T. The PageRank citation ranking: Bringing order to the web. Stanford InfoLab; 1999.

Challenges and solutions in clinical research during the COVID-19 pandemic: A narrative review

Affiliations.

• 1 Department of Epidemiology and Biostatistics, School of Public Health Tehran University of Medical Sciences Tehran Iran.
• 2 Diabetes Research Center, Endocrinology and Metabolism Clinical Sciences Institute Tehran University of Medical Sciences Tehran Iran.
• 3 Nursing and Midwifery Care Research Center, Health Management Research Institute Iran University of Medical Sciences Tehran Iran.
• 4 Students' Scientific Research Center, School of Nursing and Midwifery Tehran University of Medical Sciences Tehran Iran.
• 5 Tehran Heart Center, Cardiovascular Diseases Research Institute Tehran University of Medical Sciences Tehran Iran.
• 6 Endocrinology and Metabolism Research Center (EMRC), Vali-Asr Hospital Tehran University of Medical Sciences Tehran Iran.
• 7 Non-Communicable Diseases Research Center, Endocrinology and Metabolism Population Sciences Institute Tehran University of Medical Sciences Tehran Iran.
• 8 National Institute of Health Research Tehran University of Medical Sciences Tehran Iran.
• 9 Health Equity Research Center (HERC) Tehran University of Medical Sciences (TUMS) Tehran Iran.
• 10 Endocrinology and Metabolism Research Center, Endocrinology and Metabolism Clinical Sciences Institute Tehran University of Medical Sciences Tehran Iran.
• PMID: 37554954
• PMCID: PMC10404843
• DOI: 10.1002/hsr2.1482

Background and aims: The COVID-19 pandemic has presented significant challenges to clinical research, necessitating the adoption of innovative and remote methods to conduct studies. This study aimed to investigate these challenges and propose solutions for conducting clinical research during the pandemic.

Methods: A narrative review was conducted (approval ID: IR.AMS.REC.1401.029), utilizing keyword searches in PubMed and Web of Science (WOS) citation index expanded (SCI-EXPANDED) from January 2020 to January 2023. Keywords included COVID-19, clinical research, barriers, obstacles, facilitators and enablers.

Results: Out of 2508 records retrieved, 43 studies were reviewed, providing valuable insights into the challenges and corresponding solutions for conducting clinical research during the COVID-19 pandemic. The identified challenges were categorized into four main groups: issues related to researchers or investigators, issues related to participants and ethical concerns, administrative issues, and issues related to research implementation. To address these challenges, multiple strategies were proposed, including remote monitoring through phone or video visits, online data collection and interviews to minimize in-person contact, development of virtual platforms for participant interaction and questionnaire completion, consideration of financial incentives, adherence to essential criteria such as inclusion and exclusion parameters, participant compensation, and risk assessment for vulnerable patients.

Conclusion: The COVID-19 pandemic has significantly impacted clinical research, requiring the adaptation and enhancement of existing research structures. Although remote methods and electronic equipment have limitations, they hold promise as effective solutions during this challenging period.

Keywords: COVID‐19; challenges; clinical research; narrative review; pandemics.

© 2023 The Authors. Health Science Reports published by Wiley Periodicals LLC.