An official website of the United States government
NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Joel S. Weissman, PhD, I. Glenn Cohen, JD, Eric Campbell, PhD, Holly Fernandez Lynch, JD, MBE, Emily A. Largent, JD, PhD, RN, Avni Gupta, BDS, MPH, Ronen Rozenblum, PhD, Melissa Abraham, PhD, Karen M. Spikes, PhD, Maureen Fagan, DNP, FNP-BC, FAAN, Luke Gelinas, PhD, Stuart Lipsitz, PhD, and Martie Carnie.
Author Information and AffiliationsInvestigators and IRBs face ethical and regulatory challenges in proposing, reviewing, and overseeing patient-centered outcomes research (PCOR). A substantial gap in evidence exists regarding what PCOR-related novel ethical challenges are relevant to IRB oversight and human subjects protections. Overcoming these challenges is important to promote meaningful patient involvement, adequate protection of patients involved in research, and reduction of unnecessary variation in oversight practices.
This research aimed to investigate ethical oversight challenges in PCOR from multiple perspectives, identify whether they are unique to PCOR, determine what support IRBs may need, and promote evidence-based policy development. The specific aims were the following:
This study used mixed methods, as follows:
Aim 1. Data collection involved a qualitative component (3 institutional case studies [n = 41], 13 individual interviews [n = 13], and 6 focus groups [n = 50] organized around the specific role of participants: 2 focus groups composed of IRB members and chairs, 2 with PCOR investigators, and 2 with patients) and a quantitative component (national survey of IRB chairs [n= 265]). Qualitative research participants were investigators, patients, IRB professionals, hospital executives, policymakers, ethicists, regulators, and other PCOR stakeholders. We conducted semistructured interviews that were transcribed and iteratively coded by team members to identify key themes that emerged during analysis. The survey sample included all IRB chairs of institutions in the United States that received NIH funding. Survey responses were tabulated using chi-square tests.
Aim 2. A modified Delphi methodology was used to develop recommendations to guide PCOR oversight. The Delphi technique is a method for consensus building that uses several rounds of quasi-anonymous feedback from individual members of an expert panel and statistical analysis techniques to interpret data. It has been used widely in health care research to set priorities and develop recommended practices when obtaining high-level scientific evidence is impractical. The modified Delphi technique differs somewhat from the original in that it allows for face-to-face interactions.1 Participants included 17 experts who are IRB chairs, directors of human research protection programs, PCOR principal investigators, patients and patient advocates, experts in bioethics, experts in law, and policymakers.
We found that the only unique aspect of PCOR from the perspective of ethical and regulatory oversight is patient engagement in nontraditional roles (ie, in roles other than as study participant); other oversight challenges associated with PCOR are more familiar to IRBs because they often occur in non-PCOR human subjects research, although they are encountered more frequently and on a larger scale in the PCOR context. Of the 459 chairpersons invited, 265 returned the questionnaire, generating a response rate of 61%. From our survey, we found that IRB chairs varied in their judgment of PCOR's overall value to the scientific enterprise and to research at their institution, and in the training and safeguards their IRBs require for patient partners. Furthermore, 27% of respondents considered patients serving in nontraditional roles to warrant the protections due to research subjects even when they do not meet the regulatory definition of a human subject of research. The modified Delphi panel issued 21 recommendations, including endorsement of a formal taxonomy of patient involvement in research and a proposal to ensure that investigators receive the input of a diverse set of patients.
The engagement of patients in nontraditional roles in PCOR raises ethical oversight challenges, and thus IRBs, investigators, and institutions need guidance. The evidence-based policy recommendations resulting from this project can set an agenda for immediate implementation and further study.
The qualitative component of the research study was based on convenience and snowball sampling of the sites and interviewees. The response rate of the survey was 61%, and the survey questions were limited in the level of detail they could provide to describe the particulars of a given protocol, owing to the need to maintain balance between the length of the survey, the complexity of the questions, and the information we wanted to capture. The conclusions from our Delphi panel of experts may differ from other expert panels.
This project resulted in 3 peer-reviewed published articles, and selected text from each of those articles is included in this report (with permission). The Background section of the report offers an overview of the full project. The Methods section is divided into 2 parts, 1 part for each study aim. The first part, aim 1, refers to the collection of empirical data and includes our results from the qualitative data collection (citation 1) and from the national survey of IRB chairs (citation 2). The second part, aim 2, refers to our development of recommendations using Delphi consensus methods (citation 3). The Results section is organized in the same way. The Discussion and Conclusions sections include newly written material as well as insights from the 3 articles. The 3 relevant citations are as follows:
Largent EA, Weissman JS, Gupta A, et al. Patient-centered outcomes research: stakeholder perspectives and ethical and regulatory oversight issues. IRB. 2018;40(1):7-17 [PMC free article: PMC6324926] [PubMed: 30631218].
Weissman JS, Campbell EG, Cohen IG, et al. IRB oversight of patient-centered outcomes research: a national survey of IRB chairpersons. J Empir Res Hum Res Ethics. 2018;13(4):421-431. doi:10.1177/1556264618779785 [PMC free article: PMC6146024] [PubMed: 29902953] [CrossRef]
Gelinas L, Weissman JS, Lynch H, et al; Delphi Working Group Participants. Oversight of patient-centered outcomes research: recommendations from a Delphi panel. Ann Intern Med. 2018;169(8):559-563. doi:10.7326/M18-1334 [PubMed: 30264127] [CrossRef]
In the United States, the growing emphasis on patient engagement in research is reflected by the founding of PCORI, as well as initiatives spearheaded by the Institute of Medicine and the FDA, to grant patients' perspectives a central role in determining research priorities and informing the design and conduct of clinical trials.2* Such initiatives, often captured under the rubric of patient-centered outcomes research (PCOR), are characterized by a commitment to incorporating patient perspectives at every stage of research and promoting patient involvement in key study roles.3* PCOR, therefore, refers to traditional research that engages researchers and patients such that the patients' contribution in research goes beyond their traditional roles as participants. PCOR departs from the traditional clinical research paradigm, in which investigators drive the conceptualization and implementation of research, and embraces an approach in which patients are involved throughout the research life cycle: “from proposal development to research design and dissemination of the study results.”4,5* The underlying assumption of these efforts is that “research meaningfully informed by the patient perspective is more likely to be used to inform patient decision-making and, in the end, to improve patient outcomes.”4*
PCOR, like other human subjects research, must satisfy a set of regulatory requirements and adhere to well-established ethical principles.6† However, primarily as a result of engaging patients in research roles other than study participants, PCOR may raise certain novel ethical and regulatory issues and exacerbate some familiar ones, such as challenges associated with emerging technologies, whose use in PCOR may be more prevalent than in traditional clinical research.3* A survey of PCORI-funded principal investigators (PIs) identified patient engagement to be the second most important ethical issue after informed consent.7† The authors of the survey surmised that factors contributing to these ethical issues could be traced to novel approaches to data capture, recruitment, privacy, and “overly restrictive interpretations of regulatory language.”7†
Despite notable efforts to promote and facilitate the conduct of PCOR, scant attention has been paid to oversight challenges.7-9* There is a gap in evidence regarding which PCOR-related novel ethical challenges are relevant to IRB oversight and human subjects protections. Such knowledge is important because oversight difficulties could impede PCORI's major initiatives to generate data to improve patient care.10 Further, there is no formal regulatory or ethics guidance on oversight of PCOR from the FDA, the Department of Health and Human Services Office for Human Research Protections (OHRP), or other regulatory bodies.3 Research was therefore warranted to identify the challenges to ethical oversight of PCOR and to promote evidence-based policy development to guide key stakeholders including investigators, IRBs, and funders.
Therefore, using a mixed-methods approach, we pursued the following aims:
Two patient advisors were recruited through a co-investigator on the project, the executive director of the Center for Patients and Families at Brigham and Women's Hospital (BWH): 1 advisor was a lung cancer survivor, and the other was a breast cancer survivor. Our patient advisors have experience in the development, interpretation, and dissemination of PCORI-funded research and are active patient advocates. In addition, we were joined by a patient co-investigator with a rare genetic disorder. Our patient co-investigator holds a doctoral degree, and her professional skills were deemed essential for this methods project, to understand the programmatic, ethical, and patient-centered aspects of the project. The patients included in our research team were part of a research study in the present or recent past. These patients were selected because, as a result of their research participation, they had some exposure to or some awareness about ethical conduct of research; this was considered important for our project, which focused on research ethics.
We recruited thought leaders and IRB members to our stakeholder advisory board. These are listed in Table 1.
Stakeholders.
Stakeholders provided extensive feedback and guidance for the development of the research proposal via meetings and email communications to make key grant development decisions.
The patients and other stakeholders provided suggestions on potential participants for qualitative data collection and provided data, as appropriate. All of the stakeholders were selected for their experience with PCOR and PCOR oversight, providing us an opportunity to incorporate their perspectives.
The patients and stakeholders actively reviewed and provided input on our interview and focus group guides, survey questionnaire, and Delphi panel recommendations and materials. Our patient advisors reviewed patient-facing documents for interviews and focus groups with patient advisors. They also reviewed deidentified interview notes to provide directions for future data-gathering processes.
The patients and stakeholders reviewed and provided input on the presentations and manuscripts resulting from our study. They also provided suggestions for structuring the symposium, and some of them were invited to speak as panelists.
We regularly engaged our stakeholders throughout the study period. In the first 2 years, when significant data collection and analysis was ongoing, we had project calls every other week. Acknowledging that patient advisors might not be as familiar with IRBs and the ethical oversight of research projects, in the beginning of the project we split the entire group into a “patient group” and “other investigators and stakeholders” for the purpose of these weekly calls/meetings to ensure we could address everyone's needs in terms of vocabulary, lived experience, and so on. For the patient group meetings, we usually met in person, and several sessions were devoted to orienting patients to the various terms and concepts associated with the project aims. We also invited an IRB member to one of these meetings, to train and provide information on the structure and working of IRBs. Individuals in either group were welcome to join both the meetings, and after every call/meeting, a summary was circulated to members of both groups. As the project progressed, the frequency of biweekly calls was reduced to monthly and then back to biweekly, to balance the time commitment by stakeholders and project requirements. Several individual calls/meetings were also set up with one or a few stakeholders to discuss specific topics as required during different phases of the project. For example, several multiple-hour meetings were organized close to the launch of the survey and Delphi panel to review specific instruments.
Throughout the project, all stakeholders and investigators were invited for the quarterly stakeholder calls; they received updates on study progress and were notified of upcoming activities. These meetings were also used as an opportunity to review several of our instruments and manuscripts. Stakeholders were also engaged via regular emails.
The contributions of our stakeholders significantly impacted the study development, design, and conduct, as well as recruitment. Based on discussions via emails, phone calls, and the stakeholder meeting in October 2014, we made several changes to the proposal during the development process, such as defining the role of the patient investigator and determining the need to include patient advisors on IRB submissions. Throughout the study, input from patient advisors educated us further about the role of patients as research partners, the challenges they face, and why they think engagement is important. They ensured that our instruments reflected the patients' perspective, and they emphasized the need to avoid exceptionalizing patients as members of the research team. They edited questions to gather appropriate information from patients in interviews and focus groups. For example, they helped discern the differences in several roles that patient research partners can play in projects and edited the recruitment sheets. In addition to patients, our other stakeholders were key to ensuring the technical validity of instruments, selecting correct terms, defining the role of patients as research partners, and finalizing the Delphi rating domains and scale. Besides these intellectual inputs, they helped identify and recruit participants, and suggested ways to disseminate our findings.
In addition to the 3 manuscripts (see the Organization of Report section), we made presentations at national meetings, wrote blogs, and tweeted (Table 2).
Presentations and Blogs.
We hosted a symposium on June 28, 2018, at the Petrie-Flom Center of Harvard Law School to disseminate the key findings and share the final Delphi-endorsed recommendations (http://petrieflom.law.harvard.edu/events/details/putting-patients-at-the-center-of-research). Approximately 115 people attended the symposium. It was organized in the form of 4 panel sessions whose details are available through the link provided here. Briefly, each session began with a presentation by one of the study team members and was followed by discussion and comments from the panel members and audience (Appendix 1: Symposium Agenda).
We conducted a mixed-methods study to address our research aims. The specific methods for each aim are described below, following O'Brien et al's Standards for Reporting Qualitative Research (SRQR),11 and the Consolidated Criteria for Reporting Qualitative Research (COREQ) guidelines,12 from the EQUATOR Network. As explained in Figure 1, the qualitative data from aim 1 informed our survey questionnaire, which was the quantitative part of aim 1. The survey responses informed the Delphi consensus method in aim 2, and the purpose of the symposium of aim 2 was to disseminate the findings from the Delphi process.
Organization of Study Aims.
The purpose of aim 1 was to empirically describe the human subject-related challenges posed by PCOR and learn how, if at all, IRBs in major research institutions are responding to those challenges. Specifically, we aimed to understand the challenges faced by IRBs in their oversight of PCOR, the concerns of stakeholders (IRB members, PCOR investigators, and patient partners) with the nature of this oversight, separation of responsibilities, and the approaches that IRBs take to address these challenges and concerns. The policy objective was to determine if IRBs needed specialized policies for PCOR or could benefit from novel insights into ethical and regulatory oversight of PCOR studies. The underlying premise for this aim was that the unique features of PCOR can give rise to challenges in its oversight.
This was a mixed-methods research project including qualitative research (focus groups, individual interviews, and institutional case studies) and quantitative research (national survey). We have organized methods under 4 headings: Aim 1: Qualitative; Aim 1: Survey; Aim 2: Modified Delphi; and Aim 2: Symposium.
Qualitative research is particularly well suited to exploratory studies of topics for which previous literature is limited. Such studies are useful for generating hypotheses that can later be tested with quantitative data.13 We used multiple qualitative methods including institutional case studies, individual interviews, and focus groups. They were implemented in diverse settings and contexts to capitalize on the strengths of each, to use the different perspectives of each to help triangulate our findings, and, ultimately, to generate stronger data.4,14,15 Our team consisted of health services researchers, qualitative experts, IRB members, lawyers, and ethicists. We describe the specific strengths of each study method here.
Case studies are an “investigation of a specific, unique system with patterned behavior, dynamic properties, and defined features.”‡ Case study research allowed us to select a limited number of units of IRBs of health care research institutions and examine their characteristics intensively.13 By comparing and contrasting cases, we could identify complex processes and relationships, and describe the perceptions, barriers, and facilitators to the review and oversight of PCOR. Our goal was to understand complex IRB behaviors in situ that would not be possible with a structured survey instrument. Also, compared with focus groups, case studies allowed us to dig deeper into the specific culture of the selected organizations to understand the context in which IRB review and oversight occurs.
“Focus groups are a form of group interview that capitalizes on communication between research participants in order to generate data. Although group interviews are often used simply as a quick and convenient way to collect data from several people simultaneously, focus groups explicitly use group interaction as part of the method.”16 Focus groups were used in combination with individual interviews because, whereas individual interviews offered depth, focus groups offered breadth.17
“Semistructured interview is a verbal interchange where one person, the interviewer, attempts to elicit information from another person by asking questions.”18 Individual interviews were particularly useful for gathering the perceptions of thought leaders and those struggling with the practical realities of these reviews, who will have opinions to lend, as well as others who were unwilling or unable to join group interviews.
We purposively sampled 3 organizations with experience in conducting PCOR: a research-intensive medical school, a school of public health, and an independent hospital system. The PI and co-investigators reached out to their contacts at some of the health care institutions and/or their IRBs. Selection of the sites for case study was then determined based on the interest of the site contacts in making connections with potential interviewees at the site and/or in identifying individuals to interview during the case study.
We conducted 6 focus groups between November 2015 and July 2016. Two were in person with IRB members and chairs (n = 19) recruited from among attendees at the 2015 Advancing Ethical Research Conference hosted by Public Responsibility in Medicine and Research (PRIM&R). They were recruited through an invitation that was sent out to all the attendees by our stakeholder member who is the executive director of PRIM&R. Two were online focus groups with PCOR-funded investigators (n = 17). They were recruited from among the list of PCORI-funded investigators obtained from PCORI's website and from authors of published PCOR articles that were searched using keywords “PCOR,” “patient-centered outcomes research,” and “patient-centered outcomes.” Two focus groups—1 online and 1 in person—were with patient and family advisors recruited via hospitals' patient and family advisory councils and PCORI-funded investigators (n = 14). The patient and family advisory council of BWH was contacted to recruit patients for 1 focus group. For the other focus group, we reached out to patient and family advisory councils in Massachusetts, and to the PCOR investigators who were identified for focus group with investigators. All focus group participants were chosen to ensure geographic, institutional, gender, and professional background diversity.
We conducted 13 semistructured interviews between November 2015 and March 2016. Interviewing began with members of our stakeholder advisory panel, who were chosen because they are nationally recognized leaders in the field of research ethics, patient advocates, and PCOR investigators. Additional recruitment was through snowball sampling, a technique that supplements purposive sampling by asking interviewees to identify additional information-rich informants. The purpose of snowball sampling was to add individuals who were not in our stakeholder advisory group but who have expertise and experience in our research topic.
A semistructured interview guide was developed based on expert opinion, input from the Patient-Centered Outcomes Research Oversight Study (PCOROS) stakeholder advisory panel (composed of IRB members, investigators, research ethicists, and patient and family advisors), and a review of the literature. Having a master guide, tailored in limited ways as appropriate to the research setting, ensured consistency in the questions asked across each of the respective data-gathering contexts. Because our focus was on policy rather than individual behaviors, the questions always covered the same 4 domains: (1) how to define PCOR; (2) what, if any, ethical and regulatory oversight challenges arise in relation to PCOR related to recruitment, data capture, and privacy, and to what extent these challenges are perceived as novel or problematic by IRB members, researchers, and patient partners; (3) how, if at all, oversight challenges are currently being addressed and how they should be addressed in the future for IRBs; and (4) general demographic information of the interviewees.
All the data (interviews and focus groups) were audio-recorded, and extensive notes were taken during discussions. “Saturation is an accepted methodological principle in qualitative research.”19 “It is defined as ‘data adequacy’ and operationalized as collecting data until no new information is obtained. There are no published guidelines or tests of adequacy for estimating the sample size required to reach saturation, and it needs to be operationalized in a way that is consistent with the research question(s) and the theoretical position and analytical framework adopted.”20 We determined saturation based on our assessment of the comprehensiveness of data, emergence of patterns, and themes that began to make sense. We were able to involve the diversity of opinions and elicit various views that were beginning to be reported repeatedly.
A rigorous approach was used to ensure reliable discovery of emergent themes from qualitative data, derived from case studies, individual interviews, and focus groups. With the use of the constant comparative method, which involves simultaneous coding and analyses, for comparing newly collected data with the data collected previously, we could form, enhance, confirm, or discount themes as a result of knowledge learned. This enabled us to refine questions, develop hypotheses, and follow up on emerging themes in further depth.
To ensure the quality of the data and its interpretation, our analysis generally followed the multistep iterative process described by Miles and Huberman.21 The first step was familiarization, in which we undertook a review of the material to immerse ourselves intellectually in the data and began to list key ideas, emphasizing the domains from our interview guides. Two members of the research team (Emily Largent and Avni Gupta, who have training in qualitative research methods) independently analyzed the notes and verbatim transcripts from each of the case studies, individual interviews, and focus groups to develop and summarize cross-cutting themes—that is, common responses mentioned by multiple respondents across data-collection modes—until no new themes emerged. Transcripts and codes were also reviewed with the PI, Joel Weissman. Because we were interested in a thematic and policy analysis more than individual behavior or the development of grounded theory, we were able to perform the coding manually rather than using software. PCOROS team meetings were held to reach consensus on main themes from the data; PCOROS team members who participated in the case studies, focus groups, and interviews were engaged at these meetings. Relevant quotations from respondents explaining or elaborating on a theme or subtheme were noted and keyed to a separate file of quotations.
Based on the interviews and focus groups, we conducted a national survey of IRB chairs that focused on how different IRBs address the challenges of oversight of PCOR, what approaches or solutions they have implemented, and what are their continuing needs for guidance. This information was aimed at informing the consensus panel, in order to develop recommendations and guidelines (aim 2).
We obtained lists of IRB chairpersons from OHRP for all US medical schools (n = 138), schools of public health (n = 61), and hospitals (n = 68) that had obtained NIH funding in 2015. We chose these institutions because their IRBs were most likely to oversee PCOR now or in the future. We invited all IRB chairpersons from these institutions who were registered with OHRP, for whom we could find contact information, and who were chairs of IRBs reviewing health care-related research protocols, to participate in the study.
Development of the survey domains was informed by previous surveys of IRB chairs and qualitative research performed as part of PCOROS, which included input from IRB members and chairs, investigators, ethicists, regulators, patients, and others. The major domains included (1) PCOR experience, (2) processes and procedures, (3) challenges in review and oversight, (4) handling of patients in nontraditional research roles, and (5) the need for guidance.3
Survey items were cognitively tested with 8 IRB chairpersons (5 from our sample of current IRB chairpersons and 3 former IRB chairpersons identified through our stakeholders). We did this to identify questions that were difficult to understand or answer and to ensure uniformity of comprehension. We also wanted to minimize the use of our IRB chairperson sample for survey preliminary work and save them for participating in the survey itself.
Because of cognitive testing, we made the following changes:
Cognitive testing was followed by a formal pretest mailed to 10 IRB chairs.
The final survey instrument (see Appendix 6) was 8 pages long, contained 35 items, and took approximately 15 minutes to complete. The data source for the analyses was the database of the responses to the survey, which was deidentified by Nielsen—a survey research firm approved by the IRB at Partners HealthCare—before it was sent to the study team.
Respondents reported on the volume of all protocols reviewed by all IRBs at their institution, the volume of new PCOR protocols in their most recent year as chair, and their own experience reviewing PCOR. Respondents who reported either the occurrence of new PCOR protocols or who personally reported “some,” “a little,” or “a lot” of PCOR experience were categorized as having “some” PCOR experience; otherwise they were categorized as having “none.” Respondents skipped or otherwise opted out of selected questions, for example, about their IRB practices, if they had no experience reviewing PCOR proposals.
A central goal of the study was to identify and quantify challenges encountered by IRBs reviewing PCOR protocols. The key feature of PCOR that sets it apart from other biomedical research in which patients traditionally participate exclusively as research subjects is the engagement of patients in nontraditional roles, defined in the survey as occurring “whenever patients are named as personnel in research projects as advisors, consultants, or investigators where they are involved in any aspect of research from topic development through study design, implementation, interpretation, and dissemination.” Thus, the survey contained a battery of questions about whether and how challenging for IRB review it was to have patients serve in nontraditional, nonsubject research (“very,” “somewhat,” “a little,” and “not at all challenging”). Next, we asked about challenges pertaining to study elements that are frequently (but not exclusively) encountered in PCOR, such as the creation of local registries, multisite studies, and investigators engaging in research activities for which they are not adequately trained.
The content of the survey included questions about the ethical and regulatory oversight issues associated with PCOR. We asked chairpersons for their opinions about how much responsibility their IRB had to (1) protect patients serving in nontraditional roles and (2) protect their institution from potential problems arising from patients serving in these roles. Next, we asked whether their IRB considers patients serving in nontraditional roles “to be research subjects even if they are not formally enrolled as subjects in the same study” (yes/no). Because actual policies and practices may offer insights into IRB behavior, for chairpersons whose IRB had reviewed a PCOR proposal in the last year, we asked how often their IRBs set requirements for patients serving in nontraditional roles to provide informed consent, undergo HIPAA training (training in privacy and security protections for health information) or Collaborative Institutional Training Initiative (CITI) training (foundational training in human subjects protections), or disclose conflicts of interest (COIs). We also asked whose responsibility it was (eg, PI, IRB, funder, patients, research institutions) to ensure that “patients serving in nontraditional roles were adequately trained in human subjects research ethics.” Finally, as a way to determine the usefulness of further developmental work in the area of PCOR research ethics, we asked respondents to indicate whether federal guidance would be helpful.
The Nielsen Company administered the survey by mail between November 2016 and April 2017. Subjects were sent the questionnaire, a postage-paid return envelope, a $20 cash incentive, and an introductory letter containing a web link for those who wished to respond online. To enhance the response rate, 3 rounds of mail, email, and telephone calls were made.
The average item response rate was 98%, although missing data varied from 0.0% to 17% depending on the specific questions. All missing data were considered missing at random, and the results were computed based on the available responses. No other methods were used to account for missing data. On comparing respondents and nonrespondents, we found that they were similar in terms of their institutional affiliation.
We used descriptive statistics including proportions and measures of central tendency and variability to report the sample characteristics as well as other survey responses. Pearson chi-square tests were used to evaluate bivariate associations with IRB volume of protocols and experience with PCOR. The volume of protocols at the institution was categorized as low, medium, and high based on terciles. All items were stratified by volume and PCOR experience, although we report stratified results only if more than 1 of the bivariate associations were statistically significant. Survey responses were based on the universe of eligible population; hence the results are not weighted. All statistical tests were 2-sided with an α of 0.05. Analyses were performed using STATA MP v.14 (StataCorp).
The purpose of aim 2 was to develop guidelines and recommendations for IRBs, investigators, sponsors, and policymaking bodies faced with navigating the ethical dimensions of PCOR. This aim was informed by the findings from aim 1, where 3 key domains emerged as meriting further guidance: (1) oversight of patients in nontraditional research roles, such as co-investigator, study personnel, or advisor; (2) the use of emerging technologies (eg, social media, mobile applications) in PCOR, where their use was thought to be more common and larger in scale than in traditional research; and (3) effective identification of patient research partners and mitigation of COIs.2
To generate recommendations for these areas of need, we used a modified Delphi process, a method that uses multiple rounds of evaluation and revision of a draft document to gauge and facilitate consensus among a group of expert stakeholders on a particular topic. The Delphi technique was chosen because it is recognized as an optimal method for consensus building by using a series of feedback from an expert panel, with the use of statistical analysis techniques to interpret the data. It has been used widely in health care research to set priorities and to develop a list of preferred practices when obtaining high-level scientific evidence is impractical. As such, it avoids some of the pitfalls of other methods such as the effects of dominant individuals or the tendency to conform to a particular viewpoint.
Our Delphi process consisted of 4 rounds. Before the first round, the PCOROS team drafted 21 policy-level recommendations responsive to the 3 domains identified earlier and targeted at different entities (IRBs, funding and oversight bodies, and investigators). In round 1, we asked panel members to provide qualitative feedback on the clarity, importance, and correctness of the recommendations and suggestions for additional recommendations, arriving at 23 recommendations for round 2 evaluation.
For round 2, we asked members of the Delphi panel to complete a survey evaluating the 23 recommendations along 3 axes: correctness, need, and feasibility. The choice of axes was motivated by our aim of selecting policy recommendations capable of responding to a real challenge (ie, need), providing appropriate guidance (ie, correctness), and being successfully implemented (ie, feasibility). The criteria for determining consensus were based on the UCLA/RAND consensus criteria adjusted for the size of our panel. Thirteen of the 23 recommendations met the overall criteria for consensus after round 2 voting and were thus deemed “accepted” without further discussion.
Round 3 consisted of a half-day webinar devoted to discussing and re-voting on the 10 remaining recommendations. Before the webinar, participants received information sheets summarizing the median round 2 score, distribution of votes, a reminder of their own vote, and anonymized round 2 feedback from all panelists. During the webinar, the panel re-voted on each recommendation following moderated discussion among the group. Our process allowed for and encouraged changes to the wording of the recommendations for clarity. We used a round 4 webinar to debrief the panel on the results.
To maximize response rates, we sent reminder emails for round 2. The number of respondents who did not rate 1 or more recommendations on 1 or more domains ranged from none to a maximum of 8. All missing data were considered missing at random, and the results were computed based on the available responses.
Members of our Delphi expert panel (n = 17) were selected with the aim of reflecting the diversity of stakeholders in PCOR oversight, without seeking representativeness given the small size of the group, as is typical. Members included (1) IRB chairs and directors of human research protection programs, (2) PCOR PIs, (3) patients and patient advocates, (4) experts in bioethics, (5) experts in law, and (6) policy makers. Appendix 7 provides a complete list of the participants, who were selected on the basis of their proven expertise in these areas, as exhibited by publication record and professional position and reputation. They were invited by the PI via email, and once an individual agreed to participate, suggestions for further well-qualified participants were solicited from them and used to inform subsequent choices about whom to invite.
In any Delphi process, decision rules are determined in advance to both define and determine consensus. We based our criteria on the European Union BIOMED Concerted Action on Appropriateness for surgical procedures as referenced in the RAND/UCLA Appropriateness Method User's Manual.1 We defined consensus (ie, agreement) as a clustering of scores in the high end of the scale (7-9), without “disagreement” (ie, scores in the low end of the scale, 1-3). A recommendation was selected if there was high (positive) consensus on correctness as long as there was no negative consensus on either need or feasibility.
As described earlier, on June 29, 2018, we held a symposium at the Petrie-Flom Center of Harvard Law School. Approximately 115 people attended. The purpose of the symposium was to share the recommendations developed through a modified Delphi process and to elicit feedback/comments from a broad audience.
In all, we spoke with more than 100 individuals engaged in various aspects of PCOR (Table 3).
Overview of Study Participants in Qualitative Data Collection.
Five main themes emerged from our work (Table 4). Hereafter, the term “respondents” is used to refer collectively to case study participants, individual interviewees, and focus group participants because distinctions are most helpfully drawn between their PCOR-associated roles rather than between the data-gathering contexts.
Main Themes Identified.
Respondents emphasized the patient perspective in their definitions of PCOR, but many remained uncertain about what exactly PCOR is and what methods are entailed. When asked to define PCOR, most respondents emphasized the importance of the patient perspective. For example, 1 investigator explained, “‘Patient-centered’ means actively incorporating the patient voice in everything that we do.” And an IRB member explained, “I think this is focused on primarily doing things that are of importance to patients and not rhetoric…having patients participate.” Others described PCOR in terms of patients but had a slightly narrower focus; for example, multiple respondents included “patient-reported outcomes” in their definitions.
As a group, investigators, particularly those with PCORI funding, had the easiest time defining PCOR and seemed most confident in their definitions. Investigators were also most likely to discuss PCOR methods, such as comparative effectiveness research (CER), and to know about PCORI-funded clinical data research networks. By comparison, IRB members were generally less confident when defining PCOR. Some IRB members, for instance, defined PCOR as studies that are “PCORI funded,” or, as 1 person put it, “To me, [PCOR] meant the same as PCORI.” Others admitted that they were not “familiar with [PCOR],” with one saying, “I don't know. I don't know.” Notably, few of the IRB members defined PCOR in terms of methods that might be used, such as CER techniques or the use of large data networks.
Respondents identified widespread engagement of patients in nontraditional roles as the 1 novel aspect of PCOR from an ethical and regulatory oversight perspective. Although respondents were given the opportunity to indicate that PCOR involves no new ethical or regulatory oversight challenges, they repeatedly identified the widespread involvement of patients in nontraditional roles (ie, in roles other than the traditional role of human subject) as a novel issue. Few believed there were any other novel ethical or regulatory oversight challenges.
Respondents described various nontraditional roles that can be assumed by patients engaged in PCOR. According to both patients and investigators, patients are most often engaged as “advisors” or “consultants.” In these roles, patient responsibilities may include providing insight into their lived experience, reviewing survey instruments, and commenting on recruitment materials. In some instances, patients are engaged to help with recruitment, community engagement, and dissemination of results. Only rarely were patient collaborators described as “co-investigators” with responsibilities such as data collection and data analysis.
Respondents had highly disparate views on whether and to what extent patients in nontraditional roles need to be protected by IRBs. “Some IRBs,” a patient advisor said, “see that someone has the ‘p-word’ [‘patient’] next to their name and [conclude that the person has] to be protected…I think that's ridiculous.” Some IRB members shared the opinion that traditional subject protections are inappropriate in this context because patients in nontraditional roles are not objects of study. Other IRB members, however, felt that despite occupying nontraditional roles, these individuals should be treated like traditional research participants because they are likely being asked to share personal experiences with the research team. These IRB members believed that patients in nontraditional roles should, for example, provide informed consent even when not technically serving as “subjects” under the federal regulations. In this instance, patients in nontraditional roles perceived the IRB as a barrier to conducting the research that was important to them, rather than as a source of protection. A third group of IRB members identified the problem but described feeling conflicted about how best to proceed.
A contentious dynamic between patients in nontraditional roles and IRBs was also reported by a variety of stakeholders. For example, 1 investigator described how delays perceived to be caused by IRB review “turn[ed] patients [in nontraditional roles] off” from participating in her research. Another investigator related how the patient organization with which she was working deemed the IRB to be “interfering” and found IRB review “too burdensome [and] complicated.” This contributed to the ultimate demise of the patient organization-investigator collaboration. One patient lamented, “From what I can tell, patients are willing to take on more risk than the IRB would approve…. Going through the traditional IRB process isn't going to gain anything but [will] dilute my efforts and energy.” Although some respondents suggested that more education is needed so patients in nontraditional roles will understand the importance of IRB oversight, a handful of others suggested that having patients embedded in the research team should lead to a reassessment of the necessity of some traditional elements of ethical and regulatory oversight.
During our interviews, some respondents suggested that IRBs had reason to be more concerned about protecting the institution than about protecting patients in nontraditional roles. For example, some interviewees identified concerns about potential violations of privacy that might occur if a patient engaged as a study advisor had not been sufficiently trained in HIPAA and research ethics. Others discussed concerns with distinguishing patients engaged in nontraditional roles from more traditional study staff and in determining where responsibility and liability would fall if there was a problem. Thus, some institutions required patients in nontraditional roles to be formally onboarded as employees of the study institution. Hence, respondents also reported highly variable practices around training and “onboarding” of patients in nontraditional roles.
Respondents perceived many barriers to the meaningful engagement of patients in nontraditional roles but did not widely view lack of meaningful engagement as a problem for IRBs to resolve. Although most patient and family advisors reported that they were meaningfully engaged in their own PCOR studies, several respondents—a group including patient and family advisors, patient engagement advocates, and investigators—perceived patient involvement in PCOR as no more than “checking a box” because “[the investigator] needed someone to get funded.” Further, only a handful of investigators reported that patient involvement had, in their personal experience, been a true partnership or resulted in substantive changes to their studies.
Respondents identified various barriers to meaningful engagement of patients in nontraditional roles. Most often, they cited what 1 respondent deemed “cultural barriers.” That is, patients were insufficiently steeped in research culture—lacking knowledge about the purpose of IRB review, research methods, or the medical jargon and research-specific acronyms investigators bandied about—to contribute meaningfully to a study. Many respondents asserted the importance of having training for patients in nontraditional roles, both in research methods and the ethics of human subjects research. Such training, they felt, would “level the playing field” between investigators and patients and enable patients in nontraditional roles to make more substantive contributions. Yet some expressed caution and worried that such training could turn patients into “insiders,” causing them to lose the unique (ie, nonresearcher) perspective that is a key feature of PCOR.
In a similar vein, some investigators and patient and family advisors raised concerns that patients chosen to act in nontraditional roles are more likely to be a vocal and well-educated minority, and therefore would not necessarily be representative of the larger community of patients. In several instances, researchers reported reaching out to advocacy organizations, some of which were supported by industry, to catalyze collaborations. Select respondents noted that this may lead to a kind of conflict of interest related to “capture”: Through association with and training by an advocacy organization, the patient may represent that organization's views, which can differ from those of the “typical” patient on whom the study is focused.
Patients and investigators also discussed the perceived inadequacies of compensation for patients assuming nontraditional research roles. Although some thought these patients should be compensated comparably with investigators to demonstrate that the groups' respective contributions to PCOR are equally valued, the norm regarding compensation appears to be small payments made on a per-meeting basis. However, compensation, when it occurred, usually varied with the individual role on the individual project. In the vast majority of cases, respondents reported that their institution's IRB did not review offers of payment made to patients in nontraditional roles, although a few said that such offers should be reviewed for the possibility of coercion and undue influence.
Generally, respondents believed it was the role of investigators and funders to ensure that patients were more meaningfully engaged in PCOR, and although extremely important, this was not a regulatory or ethical oversight issue within the purview of the IRB. Nevertheless, many respondents stated that additional guidance regarding patient engagement would be welcomed, not only to inform their own decisions but also to provide a standardized approach within and between institutions.
Respondents reported that, although the ethical and regulatory oversight issues associated with PCOR are largely familiar, PCOR oversight is nevertheless relatively challenging compared with traditional clinical research. Except for engagement of patients in nontraditional roles, the general consensus among respondents was that PCOR does not present novel ethical or regulatory oversight challenges. Even so, some IRB members characterized oversight of PCOR as relatively more burdensome than oversight of traditional clinical research while investigators described significant investments of time and energy in securing IRB approval for PCOR research. When pressed, respondents had 3 main explanations for this.
First, PCOR occurs on a scale that presents oversight challenges. For example, 1 respondent explained, “I don't think [PCOR is] a unique issue, but it potentially brings up a unique perspective” on issues like waiver of informed consent because PCOR is more likely than traditional clinical studies to involve large numbers of patients. As mentioned earlier, PCORI-funded clinical data research networks aim to include at least 1 million patients, and patient-powered research networks aim to include at least 50 000 patients. One investigator described spending many months “mud wrestling with the IRB” about appropriate recruitment strategies and consent procedures for a large study of this magnitude.
Second, PCOR studies are often conducted at multiple sites that can lead to variations in IRB review. An investigator described a study that was approved in 2 days at 1 site but took 3 months to be approved at another. Respondents recognized that variability and delays in IRB review are not in themselves unusual, but they described them as particularly frustrating for both IRBs and investigators in light of the deadline-driven schedule of PCORI-funded contracts.
Third, the growth in PCOR has corresponded with growth in new technologies, and ethical and regulatory oversight challenges related to these technologies have not yet been resolved satisfactorily. One respondent explained, “I think…the rise in PCOR coincides with a lot of other trends.” She pointed to the use of social media and mobile health (mHealth; ie, medical and public health practice supported by mobile devices, such as mobile phones, patient monitoring devices, personal digital assistants [PDAs], and other wireless devices). Although respondents noted that both social media and mHealth are being used in many different kinds of research, they identified these technologies as a particularly common source of oversight challenges in PCOR.
Investigators were excited about the opportunities presented by social media and mHealth for advertising to and recruiting research participants, gathering data, and disseminating research results. In fact, many felt that such technologies made PCOR possible—how else, for example, could you find 1 million patients? IRB members were, by comparison, more reserved. In particular, they worried about the rapidly changing technological landscape and discussed the complexities of maintaining sufficiently current knowledge to keep abreast of possible ethical and regulatory oversight issues. One IRB chair described his “resignation that we're never going to understand it.” Other IRB members explained that they rely on investigators to explain new technology to them while also recognizing the bind this places them in. IRB members and high-level administrators raised concerns about data ownership and security, as well as privacy and confidentiality.
Respondents explained that IRBs and institutions are still identifying best practices for ethical and regulatory oversight of PCOR. There is a lack of consensus among respondents that PCOR demands special oversight; however, given that many of the ethical and regulatory oversight issues are familiar ones, some IRBs have begun to develop policies on how best to proceed with PCOR studies, both to improve the efficiency of oversight for these projects and to ensure that research is conducted in an ethical manner. These efforts, however, are generally in the preliminary stages, and respondents declined to share their draft policies publicly. Few institutions in our sample have or are developing such policies.
Other strategies were offered as either complements or alternatives to formal policy development. For instance, many IRB members and investigators reported that their institutions encourage PCORI-funded investigators to talk to one another and share best practices. Several IRB members and investigators described the benefits of scheduling regular meetings between the study team and the IRB before a research proposal is submitted to PCORI and suggested continuing these meetings through IRB approval. Such meetings allow investigators to anticipate and address problems in a manner satisfactory to the IRB and ideally to avoid delays that could threaten PCORI-imposed deadlines. Several IRB members also suggested designating a single IRB member or staff person who could act as a point person for PCOR-related issues.
Finally, a handful of respondents stated that their institutions have policies on use of social media and mHealth in research (ie, not limited to PCOR), and several more reported having such policies in development. Drafting such policies was characterized as “challenging” and “not straightforward”: In particular, the technological landscape changes so fast that policies can quickly become outdated. Some respondents explained that their institutions rely on information technology experts to review research-related use of technology and expressed the value of having an identifiable expert to approach with questions and concerns. This was the exception, however, and many lamented not having access to such resources.
Of the 459 IRB chairpersons in the OHRP listing at research-intensive institutions, 5 were chosen for cognitive pretesting; to reach our goal of cognitive testing with 8 IRB chairpersons, we also included 3 who were former chairs (they were recruited through our stakeholders). Eighteen chairs were excluded due to unknown or missing addresses, leaving a final study sample of 436. Of these, 265 returned a questionnaire, for a response rate of 61%. Most IRB chairpersons were male (63%) and White (92%), and 93% were actively serving as chair at the time of the survey (Table 5). The average institutional protocol volume was 843 in the previous year. Approximately 68% of respondents had at least some experience reviewing PCOR submissions.
Characteristics of Respondents (N = 265).
Most respondents were supportive of studies having patients serve in nontraditional research roles (Table 6). Almost all thought this feature was meaningful and important to patients and caregivers (90%) and added value to research (83%); most felt it improved the quality of the research process generally (69%) and at their institution specifically (60%). These perceptions did not vary by the overall volume of research protocols. However, chairpersons with PCOR experience were more likely than others to agree or strongly agree that such studies added value to research (87% vs 75%), were beneficial to advancing science (82% vs 70%), and improved the quality of research at their institution (66% vs 42%; all P < 0.05). At the same time, a substantial number of respondents agreed or strongly agreed that PCOR protocols, compared with non-PCOR, were less scientifically sound (27%) and were not adequately thought through with respect to human subject protection issues (46%; not in tables).
Perceived Value of PCOR by IRB Chairs (N = 265).
IRB chairpersons' views of the challenges posed by PCOR varied widely (Figure 2). For example, approximately three-fourths felt very or somewhat challenged by reviewing PCOR studies in which patients in nontraditional roles had access to identifiable data that can be attached to a person (84%) or served in dual roles as both subject and researcher (78%), and most identified challenges stemming from inadequately trained investigators (73%). In contrast, a substantial minority reported feeling very or somewhat challenged by patients serving on advisory boards (24%) or by the creation of local registries for recruitment (31%). Overall, 61.5% found PCOR protocols more difficult to review compared with other types of protocols (not shown in Figure 2).
Level of Challenge in Review of PCOR Protocols.
More than a quarter (27%) of chairpersons reported that their IRB considers patients serving in nontraditional roles to be research subjects even if they were not formally enrolled as subjects in the same study. We found wide variation in IRB requirements for patients engaged in nontraditional roles (Figure 3). For example, 59% and 50%, respectively, said their IRBs always or often required patient members of research teams to undergo CITI or HIPAA training, and 53% always/often required a COI disclosure. A substantial minority always/often required patients in these roles to be listed as study staff (43%) or to provide formal informed consent (37%).
Participation Requirements for Patients in Nontraditional Roles.
More than half of respondents (60%) felt that their IRB had a lot of responsibility to protect patients serving in nontraditional roles for patients in research, and 55% felt that the IRB had a lot of responsibility to protect the institution from problems that might be created by such activity (55%; Table 7). When asked about ensuring that patients in nontraditional roles were adequately trained in research ethics, nearly all placed a lot of responsibility on the PI (97%), with somewhat lower numbers indicating responsibility rested with the IRB (60%), the funder (27%), the patients themselves (49%), and their institutions (48%).
Perceived Responsibility of the IRB (N = 265).
IRB chairpersons were supportive of the need for additional guidance (Table 8). Two-thirds to four-fifths felt it would be very or somewhat helpful for the federal government to provide guidance on the topics listed. Chairpersons from high-volume institutions were less likely (67.5%) to perceive a need for guidance with respect to training IRB members than low-volume institutions (85.9%; P = 0.003). Chairpersons from high-volume institutions were also less likely (66.2%) to perceive a need for guidance on informed consent for patients serving in nontraditional roles than low-volume institutions (80.9%; P = 0.030). On the contrary, chairpersons with no PCOR experience were more likely (88.0%) to express a need for guidance with respect to training IRB members than those with any level of PCOR experience (76.6%; P = 0.041). Chairpersons with no PCOR experience were also more likely (90.5%) than those with experience (72.2%) to express a need for guidance around COI policies related to patients in nontraditional research roles (P = 0.002). Chairpersons with no PCOR experience were additionally more likely (85.1%) to express a need for guidance on policies regarding the use of electronic health records than chairpersons with experience (68.1%; all P = 0.006).3
Perceived Desire for IRB Guidance From the Federal Government (N = 265).
Aim 2 was to develop guidelines and recommendations for IRBs, investigators, and patient advisors to use when designing or reviewing human subjects research aspects of PCOR. Twenty-one recommendations for PCOR oversight ultimately achieved the prespecified criteria for consensus via the Delphi process (Table 9). We summarize them here organized by the 3 domains identified earlier.
Delphi Panel Consensus Recommendations.
The panel endorsed 10 recommendations addressing oversight of PCOR involving patients in roles other than as a research subject. These recommendations include guidance on classifying patient research partners according to a formal taxonomy of research roles; appropriate ethics and scientific training for patients in nontraditional roles; the permissibility of patients occupying multiple research roles such as a research participant and research team member simultaneously and switching between roles in the same study; and providing compensation to patient research partners (Table 9, domain 1).
The oversight challenges posed by emerging technologies such as mobile apps and social media are not unique to PCOR, as these technologies are also used in other research; however, the relatively high prevalence of these technologies in PCOR made the topic important to address in this context. Delphi consensus was achieved on 6 recommendations addressing this domain. The endorsed recommendations include guidance on educating PCOR stakeholders about protecting participant privacy and confidentiality when using emerging technologies; best practices for data collection using emerging technologies; protections around the use of social media in recruitment efforts; and the use of electronic platforms for obtaining consent in PCOR studies (Table 9, domain 2).
Five recommendations addressing the identification and engagement of patient research partners achieved consensus among the Delphi panel. These address the need for education of investigators and sponsors in identifying and selecting potential research partners, the importance of achieving a diverse mix of patient voices in PCOR, the need for education on identifying and mitigating financial and nonfinancial COIs among patient research partners, and advice to IRBs on how to approach financial COIs among patient partners occupying study personnel roles (Table 9, domain 3).
IRBs review human subjects research protocols, consistent with federal regulations and guidance documents, to protect the rights, safety, and well-being of research participants and ensure the integrity of the research process. However, the rapid advent of patients taking on new, nontraditional roles in PCOR may pose a challenge for IRBs, investigators, and institutions.10 Our initial motivation for this project was anecdotal evidence provided by colleagues at our institution and at peer institutions who were conducting PCORI-funded research or reviewing it in their capacity as an IRB member/chair suggesting that ethical and regulatory oversight of PCOR studies was proving burdensome for IRBs and, by extension, for other stakeholders.4
Although we found a general lack of familiarity with the definition of PCOR, there was agreement that PCOR raises challenging and potentially novel oversight issues due primarily to the engagement of patients in nontraditional roles other than research subject, such as co-investigator or advisor. This is an important finding because it means that existing ethical and regulatory frameworks can facilitate oversight if further developed, appropriately adapted, and made available to PCOR stakeholders.4 It is helpful to encourage IRBs to recognize that they already have many of the tools needed to review and oversee PCOR, even though PCOR may at first seem new and different.
Although not our focus, in the course of our investigation we uncovered concerns about accountability for meaningful patient engagement. Patients' involvement in the conduct of PCOR varied widely, and most respondents also stated that ensuring high-quality patient engagement in research (whether as advisor or research team member) is not an IRB's role, raising the question of how to hold investigators accountable for the quality of patient engagement and how accountability can be achieved.4
Our survey of IRB chairs from US medical schools, schools of public health, and hospitals engaged in research affirmed the variability in practices and opinions. Nearly a quarter of chairpersons considered patients serving in nontraditional roles to be research subjects even when they are not formally enrolled in the study per the regulatory definition of a human subject, a position in conflict with national regulatory definition; such a stance potentially threatens to reduce the ability of patient partners to be viewed as equals within a study team by exceptionalizing the conditions of their participation. In addition, there was wide variation in the types of training, protections, and safeguards IRBs should require for patients participating in research in these nontraditional roles. What constitutes appropriate protection for patients likely depends on details of the nontraditional roles that patient partners occupy in the context of a specific study. But when similar situations are treated differently by different IRBs—for example, when 1 IRB requires patient collaborators to engage in training and sign consent documents while another treats collaborators like investigators—problems are likely to arise regarding fairness, efficiency, and quality.
Despite these variations, most respondents who had experience with reviewing PCOR protocols indicated that PCOR is meaningful and adds value to science. Given respondent views about perceived value of PCOR, the difficulty with its review, and oversight, it was not surprising that respondents expressed the need for additional guidance from national bodies. Such guidance can improve consistent treatment of patient partners in research and help clarify when inconsistent treatment can be justified.10
Disambiguating the various roles that patients may fill in PCOR will bring needed clarity. To address this, the recommendations emanating from our Delphi process endorsed a formal taxonomy of patient involvement in research (recommendations 1-5 in Table 9) that distinguishes 3 broad roles patients might occupy—study personnel, advisor, and research subject—and whose main purpose is to allow IRBs to determine adequate protections and training requirements for patients depending on the role and activities or functions that they assume, bringing greater uniformity to practice. This taxonomy is complemented by recommendation 9, which clarifies that the typical IRB oversight protections should not be applied to patients involved in research unless they meet the regulatory definition of “subject,” thereby guarding against overprotection.3
At the same time, the role of patient partners in PCOR studies may be fluid. PCOR studies may be more likely than traditional clinical research to feature patients occupying multiple roles (eg, research subject and study personnel), simultaneously or at different phases of the study, and switching between roles (eg, from subject to advisor) in the same study, which raises oversight challenges.4 Although there is potential value in leveraging patients' perspectives on different aspects of the research, caution is needed to prevent role conflicts and safeguard scientific integrity. Problems might arise, for example, if patients are both research subjects and personnel in the same study; their obligations as study personnel to ensure scientific integrity (including maintaining a blind and the use of randomization and placebo) may conflict with their interests as patient subjects in knowing which intervention they are receiving and receiving treatment for their medical condition. Recommendations 6 to 8 in Table 9 provide actionable guidance for how IRBs and investigators should approach these situations.3
Additionally, although oversight bodies are accustomed to evaluating and managing researchers' COIs, they may be less familiar with recent empirical evidence of strong financial connections between industry and certain patient advocacy groups, which may give rise to COIs for patient research partners.3 A different challenge is making sure that investigators receive the input of a diverse set of patients in research design and implementation. Recommendations 17 to 21 in Table 9 address these issues, rejecting the high bar of patient partners being “representative” of the population from which they come, and instead recommending that investigators cultivate a diverse mix of patient partners. Further, these recommendations call for guidance from a high-level policy body, such as PCORI, on identifying and managing COIs among patient partners.3
Further research is required to understand how the recommendations from our Delphi process can be made specific and actionable for subpopulations of interest, such as investigators, IRB members, and funders. Education is a thread running through the recommendations. To facilitate implementation of the recommendations, researchers need additional information on how educational models should be developed and what they should look like, as well as what kind of patients are acting as research partners and how to manage their COIs.
We also suggest appropriately defining and, if required, revising terminology to refer to patients who are involved in research studies beyond being a participant. The term nontraditional in our research was based on input from our stakeholder advisory panel that included patients, but it is possible that future research will identify the need for a different term. In addition, future studies might also contribute to PCOR by comparing and learning from the institutions or individuals who have experience with engaging community residents and participants in research, as part of community-based participatory research (CBPR). Although PCOR poses its distinct challenges, parallels between CBPR and PCOR might be able to offer valuable insights into the ethical oversight of PCOR.
The study adopted a mixed-methods approach to identify oversight challenges, if any, posed by PCOR and to determine the need as well as areas for additional guidance. Although the sample for qualitative data was based on purposive and snowball sampling, and the small sample sizes limit generalizability of the findings to a broader population, a diversity of opinions included in our study enabled us to identify a range of issues for best informing the design of the national survey of IRB chairs. Our use of qualitative methods aligned with our study goals. We aimed to acquire an in-depth understanding of the issues surrounding ethical oversight of PCOR; hence, our goal was to gather diverse opinions rather than to ensure generalizability.
We surveyed the universe of IRB chairs at major research institutions in the United States, a strength of our design. However, the findings cannot be extrapolated to IRBs in less research-intensive medical schools, teaching hospitals, or schools of public health. We also could not account for the opinions of survey nonrespondents. We conducted an available case analysis of survey data and did not account for missing data using statistical methods. The survey was also limited in how precisely it described specific patient roles and responsibilities. Lack of a detailed description of specific patient roles might have impacted the responses to some of the questions because several practices and opinions about ethical oversight of PCOR can depend on the level of patient engagement in research.
In general, there is no standard definition of PCOR, which can influence the perceptions among respondents. However, to reduce this variability, we offered a definition of PCOR in the questionnaire. Survey results might also be subject to differing interpretation of questions owing to the way they were worded. Words such as “challenging” instead of a more neutral term may have impacted respondents' perception of the topic.
Finally, our Delphi panel used a purposive sample of identified experts and patient partners. Although this approach to Delphi processes is well accepted in the health care field, there has been limited use of Delphi in bioethics. We had no guidelines for conducting a Delphi panel to develop ethical recommendations. Hence, our methods were adapted from the existing clinical Delphi guidelines, along with incorporating feedback from a few investigators who have used Delphi in the ethical realm.
For PCOR to realize its full potential and yield knowledge that can be used for patient decision-making, the ethical and regulatory oversight challenges it raises must be addressed systematically and with transparency. Only in this way will it earn the confidence of the broader research enterprise. PCOR oversight issues can sometimes be complex, and IRBs often differ in their interpretations of the relevant regulatory concepts and ethical issues. The consensus developed by content-matter experts from diverse PCOR stakeholder groups provides a solid basis for IRBs to adopt our recommendations for addressing challenging aspects of PCOR oversight. The policy recommendations advanced here, which target IRBs, investigators, funders, and regulatory bodies, set an agenda for immediate implementation of uniform practice as well as for additional study. Such implementation would ideally be carried out by PCORI or other high-level policy bodies.3
Citation 3.
Citation 2.
Citation 1.
Patient Partners: Martie Carnie, Karen Spikes, Barry Nielsen
Stakeholder Panel: Elisa Hurley, Pearl O'Rourke, Richard Platt, Ronen Rozenblum, Steve Joffe, Sean Tunis, Maureen Fagan
Participants: Participants in focus groups, case study interviews, individual interviews, and national survey
Delphi Expert Panel: A full listing appears in Appendix 6.
Symposium Response Panel Members and Audience: A full description appears in Appendix 7. We also thank Cristine Jones-Hutchinson, Carmel Shachar, Alex Pearlman, and Wendy Williams for their help in organizing and publicizing the symposium.
Research reported in this report was funded through a Patient-Centered Outcomes Research Institute® (PCORI®) Award (#ME-1409-21701). Further information available at: https://www.pcori.org/research-results/2015/developing-recommendations-oversight-patient-centered-outcomes-research-pcoros
Symposium Agenda (PDF, 238K)
Interview Guide for Case Studies (PDF, 329K)
Interview Guide for Focus Groups (PDF, 430K)
Interview Guide for Individual Interviews (PDF, 182K)
IRB Protocol and Approval Letter (PDF, 262K)
Survey Instrument (PDF, 447K)
Full List of Delphi Panel Participants (PDF, 164K)
Weissman JS, Cohen IG, Campbell E, et al. (2020). Developing Recommendations for Oversight of Patient-Centered Outcomes Research—The PCOROS Study. Patient-Centered Outcomes Research Institute (PCORI). https://doi.org/10.25302/08.2020.ME.140921701
The [views, statements, opinions] presented in this report are solely the responsibility of the author(s) and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute® (PCORI®), its Board of Governors or Methodology Committee.
Senior patient experience advisor
This book is distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License which permits noncommercial use and distribution provided the original author(s) and source are credited. (See https://creativecommons.org/licenses/by-nc-nd/4.0/
Your browsing activity is empty.
Activity recording is turned off.
See more...
