An official website of the United States government
NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Marc Williams, MD, Janet Williams, MS, LGC, Alanna Rahm, PhD, MS, Michele Bonhag, Heather Stuckey, DEd, and Doris Zallen, PhD.
Author Information and AffiliationsGenetic and genomic information is increasingly used in clinical care. The challenge for patients and their providers is having ready access to the information that is necessary for appropriate management and coordination of care.
Although the number of users with outcome data was small, this study suggests that customizable, templated genetic reports may be a useful and durable source of information to supplement that provided by genetics professionals in traditional face-to-face encounters. Parents who accessed the report used it to enhance communication with a wide of variety of individuals who interact with their child; however, reports that did not provide clear-cut diagnostic information were less useful to parents.
The low rate of access of the online genetic report by parents may have been due to multiple factors, including genetic test results were initially returned to all patients and their parents during a visit with a geneticist before the study began, per standard clinical practice; there was lack of a definitive diagnosis in most of the patients; and unanticipated technical issues during the study.
Genetic disorders, while individually rare, are collectively common. Most are chronic, and they affect patients and their families for life. Few treatment protocols exist; providers, patients, and their families are left to try to create management and treatment plans on their own, leading to variable outcomes.
Making a specific diagnosis is key to developing effective plans for management and treatment. At present, only about 30% of patients evaluated by geneticists receive a specific diagnosis that explains the clinical presentation and allows the development of a condition-specific management plan. Genome sequencing leads to a causal diagnosis in an additional 30% to 40% of patients who have neurodevelopmental brain disorders evaluated.1,2
The challenge for patients and their providers is having ready access to the information that is necessary for appropriate management and coordination of care of rare conditions. In genetic disease, guidelines are available for only a handful of the more common rare diseases (eg, Down syndrome, Marfan syndrome). Knowledge and resources are extremely limited for providers who treat a patient with a genetic condition they have not previously encountered. This puts patients (and families) in the position of having to become the “expert” in the specific disease, which can lead to uncomfortable and sometimes dangerous encounters with the medical system.
Wilson recognizes the role a patient's knowledge base contributes to successful chronic disease management in the following statement: “By living with and learning to manage a long-term illness, many people develop a high degree of expertise and wisdom.”3 This statement is even more relevant in the rare disease area, given that the disorders are not only chronic but also lacking in the treatment guidelines that exist for common chronic diseases such as diabetes.
One potential solution to lower the barriers experienced by providers caring for patients with genetic conditions is the use of fully functional electronic health records (EHRs)4-6 that include patient-facing components. The capabilities provided by such EHRs, particularly through knowledge management systems and clinical decision support systems, have been demonstrated to significantly improve process outcomes, although the evidence of impact on clinical outcomes is less robust.7
Another possible solution may involve the role of the laboratory report in communicating information. The purpose of the laboratory report is to transmit the results of laboratory tests to providers. Genetic test reports contain information about changes in DNA structure that require significant content knowledge to correctly interpret the results. Studies have shown that this is a challenge for nongenetics providers—leading to a significant error rate that can negatively affect patient care.8-10 The introduction of genomic sequencing into clinical care will further complicate the interpretation of laboratory reports.
Studies have shown that the laboratory report itself has the potential to provide information critical to clinical decision-making at the point of care that leads to improved patient outcomes.11,12 This potential has been studied for genetic test results, leading to a proposed format for genetic test reporting with a design based on extensive input and feedback from genetic and nongenetic providers.13,14 Testing of this report showed significantly higher satisfaction, ease of use, and efficiency for the formatted genetic test report compared with standard reporting.15 The authors noted, “Physicians least familiar with genetic test reports, and possibly having the greatest need for better communication, were best served by the template reports.”
We hypothesize that a new form of functional genetic test report, presented through an EHR to patients, could dramatically improve the shared decision-making, planned management, and overall outcomes from the perspectives of both the patient/family and the provider. In this study, we propose to study the question: Can a genomic laboratory report tailored to both providers and families of patients improve interpretation of complex results and facilitate recommended care by enhancing communication and shared decision-making?
A patient co-investigator (Michele Bonhag) was involved throughout the project. She participated in the design of the project and writing the initial proposal, including authoring the public abstract. Before the study, she received training from other co-investigators (Drs Rahm and Stuckey) in semistructured interviews in order to participate in this aspect of the study. During the study itself, she participated in interviews and focus groups, was a key part of the analytic team that coded the interviews and focus groups, analyzed the survey data, and developed conclusions based on the data. Ms Bonhag ensured that a general patient and a parent perspective were available for all discussions, evaluations, analysis, and writings, and at all study meetings. She co-authored all papers and manuscripts from the project (except for the technical manuscript) and has had the opportunity to present the study and her role at several meetings, including the inaugural PCORI annual meeting. The project has also been presented to the Geisinger Return of Results external advisory board, which includes 4 Geisinger patients as representatives. Results of the project have been presented at these meetings (held 2-3 times a year), and input from the group has been used to inform the project. The decision to use a brief phone interview to increase the number of respondents described below came from this group. Additional key stakeholders and their involvement, detailed in Table 1, included researchers experienced in methods, health literacy, patient engagement, communication, genomics, report content, and electronic report design.
Key Stakeholders Participating in All Aspects of Study Design, Analysis, and Reporting by Role and Contribution to Overall Project.
Parents and health care providers of children with rare disorders are the stakeholders of greatest importance in this study. A detailed description of these parent and provider stakeholders is included in the study cohort sections.
Monthly phone conference calls were held, in which all team members participated. Each stakeholder on the team was encouraged to voice opinions and concerns during each discussion. Specific contributions were expected of each stakeholder.
The parent stakeholder perspective was paramount in this project and, therefore, parents were both stakeholders and participants. Qualitative interviews and focus groups with our patients' parents were completed before the provider interviews, and they defined the outcomes of importance for this project.
The report developed for providers was based on an analysis of the parent interviews. The balance in patient and provider perspectives was one of the most interesting aspects of this study. This tip in balance toward the parent stakeholder perspective was exemplified in comments by many providers who requested the ability to access the parent report as well as the provider report, primarily in order to be prepared for parent questions. The providers also noted that several elements suggested by the parents (eg, prognosis table) were not ones they had considered but were found to be the most valuable components of the report in terms of provider ability to care for the patient and to facilitate communication with the parents.
The patient co-investigator was a member of the whole-genome sequencing (WGS) study oversight committee and was recruited via personal invitation to contribute to all phases of the study. Other key stakeholders were recruited by the principal investigator (PI), based on their previous work expertise including national publications, system leadership in health literacy, expertise with technical components and content of the report, and personal expertise.
Parent stakeholders (as participants for aim 1) were recruited based on their child's participation in a separate research study using WGS. Parents were invited during the in-person WGS results visit to participate in the PCORI study. The study coordinator followed up on the clinic visit discussion and contacted parents to confirm their willingness to participate in an interview, a focus group, or both. Throughout the study, parents were informed that participation was voluntary and were assured that they or their child would face no loss of health care should they decline to participate. Provider stakeholders (as participants for aim 1) were recruited via personal invitation from the PCORI PI.
Table 2 provides an overview of the engagement processes. The design of this study was such that engagement for the parents as stakeholders and participants was intertwined. The development of the report called for engagement as participants in focus groups and interviews, while the testing of report impact engaged parents as research participants in a randomized trial. Stakeholders were engaged through different modes and for differing intensities, depending on the point in the study/study aim in progress and the purpose of the engagement as detailed in Table 2.
Overview of Stakeholder Engagement by Mode and Intensity Throughout the Project.
This research and the resulting enhanced genomics results report could not have been completed without the engagement of parents and providers of children with rare genetic conditions. Throughout the initial development and deployment of the enhanced report, parents and providers expressed the need for a report that could improve their understanding of the result and facilitate communication between parents and providers—thus reinforcing the importance and relevance of the overall project research question to these stakeholders.
Ms Bonhag, as patient investigator, provided critical information about the burden of study interventions such as interviews, focus groups, and surveys, which were incorporated into the design of the study processes so that we could minimize the burden of participating while still capturing the outcomes of most interest and relevance to the research question. Likewise, the engagement of our patient co-investigator and other key stakeholders (represented in Table 1) had a critical impact on the study rigor and outcomes during the development of the enhanced report. At several points in the study, we experienced challenges with participant contact, survey completion, and apparent attrition. Stakeholder input was essential in developing alternative approaches that improved the participant experience without affecting collection of study data. Monthly meetings with the entire research group, as noted in Table 2, facilitated sharing of results and problem solving in a transparent and rigorous manner.
An example of the importance of parents as stakeholders was noted in the first round of development interviews in aim 1. Parents told us they wanted information on the condition, including what steps to take immediately and what to expect over time. Members of the study team experienced with creating these types of materials distilled this “prognostic information” into 3 different views for parents to review in the next round of engagement (aim 1 focus groups). However, our patient co-investigator and patient content stakeholder advocated for the inclusion of a very long, very detailed prognostic table (example shown in Figure 1) to be tested along with the other views. Had these stakeholders not been on the study team, this comprehensive prognostic table would not have been tested with the parent stakeholders, because the expert genetics researchers considered it to be “too much information” based on their many years of clinical experience. When parents were given the choice (details found in Table 8), they overwhelmingly endorsed the table with the most detailed information. While parents liked some aspects of 2 of the other 3 versions of prognostic information, Concept 4, the comprehensive prognostic table was the most important, most valued, and most appreciated component of the report for both parents and providers. Indeed, one provider said, “If this were my child, I'd pin this table to their clothes every morning.” This is only one of many examples where this type of diverse stakeholder input led to decisions that were of high value to the patient, family, and provider end-users.
Prognostics Table Concept Example With Participant Feedback.
Parent Stakeholder Response to Different Conceptual Formats of the Prognostic Table (Aim 1 – Report Development),.
Another particularly poignant example of the impact of this patient-centered enhanced genomic report, as finally implemented, occurred when 1 of the families in our WGS study was accepted into an NIH research study that included a weeklong intensive evaluation at the NIH clinical center. After this parent and child arrived at the clinic, they were told that the genetics results of the WGS had not been forwarded along with the other medical records documenting the child's care. The parent promptly pulled up the GenomeCOMPASS Report through the MyGeisinger online patient portal and downloaded the entire report, which included the comprehensive prognostic table of all possible symptoms by how common they are and the typical age of onset (Figure 1). Having this enhanced report shaped the entirety of the research visit at NIH, after which the NIH providers informed us directly that the report was instrumental in determining the follow-up care for the patient. Therefore, this enhanced report—available through the EHR patient portal and including the prognostic table—addressed and solved the common health care problem of barriers to sharing information between systems. This problem often leads to suboptimal and duplicative care, generating waste and increased costs of care with no improvement in patient outcomes. The patient is the only common element in the delivery of health care, so by putting information into their hands, under their control, we can alleviate some of the issues illustrated in this example.
As there was little available research on the direct use of laboratory reports by patients, we chose a mixed-methods study design exclusively utilizing qualitative data collection to develop the report (aims 1 and 2) and quantitative surveys to collect information during the randomized trial (aim 3), followed again by qualitative data collection to allow for a richer understanding of the impact of this new enhanced results report (aim 3). Our hypothesis was that if reports were developed through extensive stakeholder engagement and by employing user-centered design principles, such reports would have the potential to improve understanding of rare diseases, aid patient satisfaction, and enhance effective communication. Therefore, the development of an appropriate report that is patient-centered and provides effective communication and subsequent impact on patient and provider outcomes is largely a qualitative question, resulting in the following methodology choices as appropriate to each study aim:
Develop a genomic report with advanced functionality, including point of care education and clinical decision support. Development will use providers and parents of affected patients to provide feedback on the desired elements for the provider and patient views and the usability of the report.
Semistructured qualitative interviews were used to gather information deemed essential by parents; such interviews informed the development of an enhanced results report, followed by structured focus groups to refine specific components of the report suggested as being deficient in the interviews. Semistructured interviews were also conducted with providers to determine how well the report met their needs and if additional information (or separate report) were necessary (Figure 2). These iterative steps resulted in the creation of an enhanced genomic report with the potential to improve communication and affect outcomes important to families and providers.16,17
Flow Diagram to Develop and Deploy a Patient-Centered Enhanced Report (Aim 1 and Aim 2 Processes in Sequence).
Deploy the report for patients and families and their providers. The report will be presented to clinicians in the EHR and to patients either through a secure patient portal or through mailing a printed copy.
An online beta-test version of the enhanced report derived from aim 1 was created. A user-centered design approach was important to qualitatively capture the user experience of the technology within the EHR environment. This approach is widely used to gather data during product design and development, as it invites participants to indicate what they are seeing, thinking, doing, or feeling, as well as where they are encountering difficulty. This process improved the look, feel, and function of the enhanced report in the online environment, resulting in a final report that was deployed via MyGeisinger messaging (the patient portal to the EHR) to patient participants as the intervention for aim 3. As noted above, access to MyGeisinger is optimized for smartphone use to address potential barriers of internet connectivity in the population. Deploying through MyGeisinger allowed for the interaction and accessibility requested by parents during design. A print copy was available if requested by parents during the trial (aim 3).
Study the impact of the tool from the perspective of providers and family of affected patients.
A randomized prospective pre-post asymmetric crossover design paired with additional qualitative evaluation was employed to study the marginal impact of the enhanced report over the clinical summary letter alone for parents and providers (Figure 3).
Aim 3 Randomized Trial Design to Test the Impact of GenomeCOMPASS Report Compared With Usual Care.
This mixed-methods approach was designed to evaluate the impact of the enhanced report compared with the usual care process of returning genomic results. The schema depended on stratification of parents by result type (causal variant vs uninformative result). Randomization occurred after the return-of-results visit when parents would receive the enhanced genomic report (after baseline survey: intervention arm or at 3 months: usual care arm).
Quantitative measures in the baseline and 3-month surveys were chosen based on parent interviews in aim 1. Aim 3 employed an explanatory sequential design,18 a research methodology in which a researcher obtains quantitative data in a first phase (baseline and 3-month surveys); then, in the second phase, qualitative data are collected (interviews) and related to the quantitative data. This methodology is particularly useful in in guiding analysis of the responses on the surveys when combined with additional data gained through qualitative feedback in the form of short interviews and in-depth interviews. The surveys and interviews were designed to determine factors that influenced parent use of the enhanced report and parent comfort in communicating about the child's health. Qualitative interviews followed the baseline and 3-month surveys, as it was unlikely that quantitative measures alone would offer complete understanding of the overall report experience due to the novel nature of this intervention.
The study cohort was comprised of parents of children with rare, unexplained, intellectual disability (ID) and autism spectrum disorder (ASD) enrolled in the Whole Genome Sequencing Research Study and health care providers who were associated with the WGS research study (utilizing a study within a study approach). The inclusion criteria for participation in the WGS Research Study were: child under the age of 21 years; both parents available and consent to being sequenced along with the child; child with a diagnosis of ID, ASD, or multiple congenital anomalies suspected to be genetic in origin; child does not yet have a genetic diagnosis; and child has had a normal chromosomal microarray.
These children present with complex medical care requirements and, because they have no identified etiology, often there is little or no specific guidance to coordinate appropriate care. The study offered the family genome sequencing for their child with the potential to find a diagnosis that could offer important outcomes, including (1) unified efforts at care management, (2) family planning, (3) educational planning, and (4) anticipation of future health issues. Establishing a diagnosis ends the diagnostic odyssey, which saves unnecessary medical costs and burden for the parents.2 Application of genome sequencing approaches in the context of neurodevelopmental brain disorders (such as ID) has consistently yielded diagnoses in 30% to 40% of individuals tested.1,2 Each of the diagnostic outcomes referred to above relies on parents and providers understanding the genomic results and the child's diagnosis associated with them. Therefore, this population exemplified the need for a vehicle to enhance communication of information related to rare genetic diagnosis in the absence of best practices or evidence-based clinical recommendations.
These parents were chosen as most appropriate participants for this study because they have the primary responsibility for the care and management of their children with chronic, rare conditions and are the recipients of communication regarding their children's test results. Both parents were required to be enrolled in the WGS study for purposes of genomic results interpretation, and thus both parents were invited to participate in this study. Having both parents available offered the opportunity to capture potential differences in response to the enhanced report between fathers and mothers, something that has not been studied previously in the context of return of results to pediatric patients.
Eighty-four parents participating in the WGS study were eligible to participate in the development and design of the enhanced genomic report. Parents were contacted by the study coordinator and invited to come in for either an interview (development round 1) or focus groups (development round 2).
The provider cohort included physicians who care for children with ID and ASD—many of these providers were associated with the WGS Study Oversight Committee. Only 1 provider in the interview group cared for a child who was enrolled in the WGS study. Table 3 describes the characteristics of the providers who participated as stakeholders in the report design. These providers were invited to participate in individual interviews through personal email contact by the study PI or study staff. The primary reason for not participating in an interview was lack of availability due to clinical responsibilities.
Providers Involved in Development of the Enhanced Report Content and Design (Aim 1).
A convenience sample of department research assistants who were not involved in this study and representative patients was selected to perform the iterative user testing of the enhanced report within the EHR system.
Parents who had completed a genetic results return visit with the WGS genetics team were invited to participate in the randomized trial of the enhanced results report. The family genetics visit included a review of the research laboratory report, limitations of sequencing, and the result for the child—that is, a causal variant (diagnostic result) was found to explain the child's set of symptoms or no causal variant (uninformative result) was found and no explanation was currently apparent. A comprehensive summary letter describing the discussion and results was composed by the medical geneticist and sent by mail to the family after the session. This process is representative of current standard practice in which laboratory reports are designed for and delivered to health care providers who then report the information to the patient/parent. Summary letters are commonly sent by genetics professionals to their patients. Parents were informed during one of their WGS study visits of the potential to participate in a new study about test reports.
After the WGS results visit, the study coordinator sent a letter and baseline survey to parents. Parents were given the option to signal their participation in the trial of the enhanced report by completing and returning the baseline survey. Once the baseline survey was returned, the parent was considered enrolled in this phase of the study; those who did not complete a baseline survey did not participate in the trial of the enhanced report.
Eighty-four eligible parents were stratified by genomic test result (diagnostic vs uninformative) and then randomized (as couples, n = 42) within each group by the study coordinator using a random number generator to receive either the enhanced genomic report (intervention) or usual care (control). Randomization by couples was chosen, recognizing that randomization at the individual level could lead to contamination and spillover if 1 member of the couple were in the usual care arm and the other in the intervention arm. To test the impact of the enhanced genomic report regardless of test result, parent-couples who received a diagnostic result were evenly allocated to intervention or usual care arm—as were all parent-couples who received an uninformative result. Parents and investigators were blinded to the randomization.
Geisinger is a rural integrated health care delivery system that serves approximately 500 000 patients in central, south central, and northeast Pennsylvania and southern New Jersey. The study population was drawn from current Geisinger patients who are under age 21 and who are seen in outpatient pediatric clinics. Many of the patients in the study reside in areas classified as underserved by the Department of Health and Human Services. Issues of internet accessibility are a concern to the study population, as much of the Geisinger service area does not have broadband fiber-optic connectivity, nor in many cases even basic cable. Consequently, Geisinger's e-health strategy includes solutions that allow many patients to connect electronically with the health system even in underserved areas; 1 example is the ability to access the patient portal of the EHR securely via smartphone. In addition, the option to receive a print version of the genome results reports was offered.
The intervention consisted of deployment of an enhanced genomic results report, referred to as GenomeCOMPASS; it was delivered through the patient portal of the Geisinger EHR to patients and providers. As previously published,16 this enhanced results report was developed through engagement of parents and providers during aims 1 and 2 of this study, and is described in the results section of this report. Choice of comparators is relevant only to aim 3, the randomized trial to study the impact of the enhanced report created through aims 1 and 2 compared with usual care practice. In usual care practice, the laboratory reports are rarely given directly to patients/parents; rather, test results are discussed during a clinic visit and summary letters are sent to synthesize and interpret the complex and comprehensive nature of the information discussed at the visit. Therefore, as shown in Figure 3, the comparators for aim 3 were usual care (clinical return of results with summary letter) plus enhanced genomic report via the Geisinger EHR (intervention) vs usual care (clinical return of results with summary letter) without the enhanced genomic report (control). Crossover of the control group to intervention arm occurred at 3 months postbaseline.
The intervention, the enhanced online report, was not intended to replace the laboratory report or the clinical visit to discuss test results (usual care); rather, it was intended to present the genomic result from the laboratory in language accessible to patients, family members, and nongenetics providers. In addition, the enhanced genomic report provided access to supportive information and resources commonly discussed at the clinical visit but rarely included in the laboratory results report.
Exposure period to the intervention is specific to aim 3 only and, for study purposes, includes the time between the initial provision of the enhanced report and the 3-month postexposure survey and follow-up interviews. However, the enhanced report is a part of the child's medical record; therefore, exposure to the intervention is ongoing for both parents and providers as part of normal clinical care. Parents and providers could access the report through the Geisinger patient portal whenever it was thought necessary, as access to information was a key outcome of the study.
The primary outcomes were the development of a patient-centered report through user-centered design and the deployment of the report through the EHR system. Secondary outcomes for these aims included acceptability of electronic delivery, and how parents and providers envisioned using such a report. Parents and providers, independently and without solicitation, remarked that they envisioned such a report would be valuable and, as designed, would be an improvement over standard genetics laboratory reports.
Table 4 lists the primary and secondary outcomes for aim 3.
Data Collection Methods, Data Sources, and Survey Measures Used by Outcome to Test Impact of Enhanced Genomic Report (Aim 3 – Randomized Trial).
Survey measures were selected to represent the outcomes defined as most important by our patient investigator, the engaged parents, and the health care providers who participated in the qualitative work completed in aims 1 and 2. All team members also contributed to the outcomes chosen for aim 3. Primary outcomes included actual use of the enhanced report as measured by analytics within the GenomeCOMPASS tool and patient satisfaction as measured by surveys and interviews after the report was released. Secondary outcomes included how the report was used by patients and providers, unintended consequences of implementing the report through the EHR system, decision regret, patient-provider communication, and the impact of genetic test results. Survey scales (see Appendix for the complete survey) were chosen from existing validated measures (Table 5) to reflect themes identified by patients and providers during the initial report development (aim 1) as being of primary importance to them. Other measures, such as health literacy20 and numeracy,21 were chosen based on literature indicating the relevance of these measures to patient-facing tools. Decision regret24 was chosen to ascertain if parents regretted having participated in the WGS study, as we hypothesized that those who regretted participating may be less satisfied with any communication from the study, regardless of whether they received the enhanced report or the usual-care summary letter. As shown in Table 4, some of these outcomes are available from quantitative measures, while others are qualitative, in keeping with our overall explanatory sequential mixed-methods design.18
Survey Measures and Timing of Assessment (With Citations) Used to Evaluate the Impact of the Enhanced Genomic Report (Aim 3 – Randomized Trial).
Parent-participant interviews using a semistructured interview format were completed with 9 individuals. A copy of the semistructured interview guides can be found in the Appendix. Two focus groups using structured interview guides were conducted with 5 individuals. Qualitative interviews with 6 providers involved with the WGS study or who care for children with special health needs were conducted using a semistructured interview guide. In all cases, thematic saturation (no new information learned) was achieved with the reported population, obviating the need to engage more participants before moving forward to aim 2.
Structured usability sessions were conducted with a convenience sample of 5 individuals who were patients and/or departmental research assistants not involved with the WGS or PCORI study. Usability testing is a process for a researcher to evaluate a product or application with representative users. The content is not evaluated; rather, the process of “clicking” through a webpage and navigating a website are tested. Sessions were conducted iteratively until the enhanced report as deployed appeared to work properly in the EHR system with optimal usability from the parent perspective.
Data collection for the randomized prospective pre-post crossover trial included quantitative and qualitative methods. Quantitative data were collected through multiple combined validated survey scales chosen to represent the themes identified by parents as most important in aim 1. At baseline all scales were administered; however, the scales that collected demographics and assessment of health literacy20 and numeracy21 were not included on the 3-month surveys. The baseline measures relied on questions used in the Health Information National Trends survey,21 including the numeracy, general health, internet use, and information-seeking assessments. Two scales designed for assessment involving genetics were chosen. Psychological Adjustment to Genetic Information Scale (PAGIS)22 was used to ask about certainty of knowledge about genetic information, and an adapted form of the Multidimensional Impact of Cancer Risk Assessment (MICRA)19 provided responses relative to the impact of genetic testing. The Health Information Orientation Scale23 was used to assess health information preferences and engagement with information sources. The Decision Regret24 Scale was described above. Survey measures and the timing of their inclusion in the various survey instruments are listed in Table 5. The surveys were sent by US Postal Service with a return envelope at baseline and 3 months (intervention), or baseline, 3 months, and 3 months postintervention (control with crossover). Structured interviews were conducted at 3 months postintervention to further understand the utilization, impact, and unintended consequences of the enhanced report.
Thirteen of the 14 parents who participated in the design of the enhanced report (aim 1) were among the 52 parents who participated in the randomized trial to study impact of the report (aim 3). However, report design involved hypothetical examples of results and occurred before any parents receiving their own child's result. At the time of the aim 1 qualitative work, parents were not told that such a report would be developed relative to their child; rather, parents participated in order to develop an example report.
Recognizing that survey completion was crucial to aim 3, several processes were implemented to maximize completion of the baseline survey for enrollment and for the follow-up surveys. Phone calls were initiated 2 to 3 weeks following survey deployment to encourage parents to return the survey, and an offer was made to complete the survey over the phone. After 3 messages/contacts, a second mailing of the survey was sent to parents with outstanding surveys. To further maximize the follow-up rate, 1 last attempt was made in the final month of the project to allow parents with outstanding surveys to complete a short, structured interview via telephone. A comprehensive tracking database was employed to record parent dispositions and reasons for not returning the survey or not using the enhanced report as reported by parents.
After the 3-month post-enhanced report survey and/or short interview, parents in both arms were also offered the opportunity to participate in an in-depth semistructured interview. To maximize participation, interviews were conducted over the telephone at a time convenient to parents.
This study was initiated before the draft and finalized PCORI Methodology Standards; therefore, this study could not be designed to specifically include or address the current standards. Those that were addressed are detailed in the table of standards listed in the Appendix. One of the key methodology standards, heterogeneity of treatment effects analyses, was possible to consider, based on stratification by result (diagnostic vs uninformative) and planned surveys of both parents (mothers vs fathers). Such analyses were ultimately not appropriate due to the small sample sizes of parents who completed both baseline and postintervention surveys (n = 15 intervention arm, n = 20 crossover arm), the small sample size of parents who actually opened the enhanced report and completed the surveys from either arm (n = 15), and the even smaller sample of parents who opened the report, completed surveys, and had a diagnostic result (n = 4).
Given the lack of information on laboratory reports developed for patient and provider use, this aim utilized qualitative methods to most effectively engage these key stakeholders in developing a patient-centered report. All interviews and focus groups were audio-recorded and transcribed. Patient and provider responses were coded to capture language and information preferences of parents and providers based on the lived experiences of receiving genomic sequencing results, using an existential phenomenological conceptual framework.25
Standard practices of usability testing—such as observation, navigation tracking, and think aloud—were employed to iteratively test the integration of the enhanced report into the EHR system. Throughout usability testing, the content of the report did not require change or reorganization. Navigation challenges were noted (eg, links that did not connect) and resulted in changes until users could go through the report without any glitches. Usability testing continued until all testers reported that they could click through the entire report without problems. The COMPASS application recorded whether the report was actually accessed by the parent as part of deployment, thus providing objective use data for 1 of the primary outcomes of aim 3.
For the quantitative analysis (phase 3 experimental phase) it was predicted that it would be difficult to perform a precise power calculation until the specific survey scales were identified. The scales were to be chosen based on the patient input from stakeholder engagement via interviews and focus groups in the developmental phases of the study (aims 1 and 2). As a first approximation over the range of instruments that could be used, a 2-sided, 0.05 level Wilcoxon rank sum test indicated that we would have 80% power to detect a difference between groups in change from baseline on any continuous measure in SD units of 0.76 with 30 subjects per group, 0.65 with 40 per group, and 0.58 with 50 per group. We also planned to adjust for baseline characteristics, such as severity of disease and education level of parents, in regression models for the outcomes of interest and to assess suitability of linear regression models for continuous outcomes through diagnostics of residuals. Detected differences in parents' responses after access to the enhanced report could signify increased satisfaction and understanding of the result and improved communication with providers and could represent a clinically meaningful contribution to practice.
Unfortunately, final sample sizes were too small to detect change based on power calculations (n = 28 intervention arm, n = 24 control arm), meaning that only descriptive statistics were appropriate and conducted on baseline, 3 months postbaseline surveys (usual care arm before crossover), and 3 months postreport surveys (intervention and crossover). In the case of missing data, when survey measures contained summary scores, a mean score was calculated based on responses provided. This statistical method for handling missing information is typical for the scales used (Table 5); however, it may result in an underestimation of the respondent summary score. As described above, efforts to reduce missing surveys and clear instructions and pretesting of the survey instrument were used to reduce the potential for parents not to answer individual questions. We also examined respondent answers to individual questions in each scale and in summary scores to reduce the impact of using scale scores calculated only on responses provided. Write-in responses for the question regarding from whom respondents sought medical information were categorized by study staff for analysis. Inferential statistical tests were not conducted due to the small sample size of parents who actually opened the report (n = 15).
Subgroups included parents whose child received a causal variant result on WGS (a cause for their child's symptoms was found) and parents whose child received an uninformative nondiagnostic result (no causal variant). These subgroups were anticipated and accounted for by stratification before randomization, as described in Section 9; however, the number of children receiving a diagnostic result was much smaller than anticipated (n = 7). Additionally, 2 parents in the usual care with crossover arm who had children who received a diagnostic result did not answer any questions in the PAGIS subscale22 on the 3-month postreport survey. In this case, summary scores were analyzed only for the parents who completed these questions during the descriptive analyses.
Qualitative analyses on interviews after the enhanced report were conducted as in aim 1. Themes analyzed for aim 3 included satisfaction with report, use of the report to communicate with others, technical issues with the EHR report, reasons for using/not using, and suggestions for improvement. All parents were eligible and invited to complete a short semistructured interview and an in-depth interview.
Participants were recruited from an ongoing clinical research study, the WGS study mentioned previously and approved under the Geisinger Health System IRB (#2012-0187). The Geisinger IRB approved this project and all amendments as a separate study (#2013-0594). The randomized trial was registered at ClinicalTrials.gov (identifier NCT02504502).
Semistructured individual interviews were completed with parent participants; they were followed by 2 focus groups with parent participants for evaluation of additional concepts that emerged from the interviews, a prognostic table, and for design of the added information. Report samples for providers as well as the parent-designed reports were given to a convenience sample of medical providers comprised of pediatric specialists, med-peds specialists, and an internist for their response to the design.17 The end point for aim 1 was the development of the content for the enhanced report for use in aim 2 and aim 3 of the study. Parent participants were engaged via recruitment letter, followed by telephone call by study staff inviting participation in an interview or focus group. The study PI sent personal email invitations to participate to the providers; study staff coordinated interviews. Interviews and focus groups were conducted by study staff proficient in this type of data collection. Parents and providers were provided a $25 gift card on completion of the interview or focus group.
Participants engaged for usability testing were recruited by personal email from study staff and offered the opportunity to test the enhanced genomic report.
Parents (as couples) were first stratified by genomic test result (diagnostic vs uninformative) and then randomized by the study coordinator (as couples within each group) to receive either the enhanced genomic report (intervention) or the summary letter (usual care with crossover). Parents were not told to which arm they were randomized. All parents of children in the WGS research study were invited into the experimental arm (aim 3) via letter explaining the randomized trial along with the baseline survey. Parents were enrolled in the randomized trial if they completed the baseline survey. Over the course of the experimental phase, individuals completed either 2 survey instruments (baseline, 3 months post-enhanced report) for the intervention arm or 3 surveys (baseline, 3 months, 3 months post-enhanced report) for the crossover arm. At 3 months post-enhanced report (intervention or crossover), enrolled participants were also invited via recruitment letter, followed by phone call, to participate in an in-depth interview about using the enhanced report. All surveys were conducted via mail with the option to complete via telephone if desired; all interviews were conducted via telephone. Participants were provided a $25 gift card at the completion of each survey and interview.
One major change from the original study protocol as proposed involved removal of the requirement for a provider visit after the initial release of the enhanced report. Initially, it was thought that scheduling an extra visit focused on the reports for patient and provider would provide an opportunity to study its use in the clinic. However, in consulting with our participants and reviewing the number of visits these children have with the health system, it was decided that scheduling this extra visit was not practical or respectful of the parents' time. Therefore, we decided that due to the frequent encounters of the children with the medical system, requiring an additional visit just for purpose of reviewing the enhanced report would place an undue burden on both patients and providers. We felt the ability to track provider and patient access of the enhanced report was an acceptable alternative to the planned single visit assessment focused on the report, as it allowed data collection in a pragmatic way; it increased the generalizability of findings and reduced the potential Hawthorne effect from a visit focused on the report.26
Another small change to the protocol involved the addition of a final, short, structured interview administered by phone at the end of the study with parents who had not returned the 3-month postreport survey. Due to the low utilization of the report and return of surveys, this short, structured phone interview was added after the 3-month postreport survey to elicit information about access and barriers and then invite the parent to participate in an in-depth interview. Study staff contacted all parents by phone approximately 1 month after all 3-month post-enhanced report surveys had been sent. This resulted in the completion of 8 short interviews and 1 in-depth interview with the intervention group (Figure 3) plus 10 short interviews and 4 in-depth interviews with the crossover group. As shown in Figure 3, these short and in-depth interviews were completed both with parents who returned the surveys and with those who did not, in order to understand the utility and barriers of the enhanced report.
Develop a genomic laboratory report with advanced functionality, including point of care education and clinical decision support. Development will use providers and parents of affected patients to provide feedback on the desired elements for the provider and patient views and the usability of the report.
Results for this aim are available in greater detail elsewhere.16,17 Qualitative interviews were conducted with 9 parents, and 2 focus groups were conducted with 5 parents. Individual interviews with participants lasted 60 to 90 minutes; focus groups lasted approximately 90 minutes. The major reason given by parents who declined to participate in the interviews or focus groups was lack of availability—even though we offered times during evenings and Saturdays to increase access for working parents. Other parents chose not to participate in the report design due to lack of time, interest, or pressures associated with caring for their child's chronic condition. Three families moved out of the area and had no further contact with the health care system and were therefore ineligible.
The qualitative data were collected via audio-recording and transcribed by secure hospital transcription services. Transcriptions were evaluated using emergent categorical codes related to improvement in the report, communication success, and additional needs noted by participants. The patient investigator participated in all thematic analyses to ensure representation of the patient perspective in the thematic interpretation of data.
Three themes emerged (Table 6): (1) Parents described a continual search for valid information and resources regarding their child's condition, a need that prior reports did not meet; (2) parents believed that the genomic report would help facilitate communication with physicians and family members; and (3) parents identified specific items that they appreciated in a genomics report: simplicity of language, logical flow, visual appeal, information on what to expect in the future, and recommended next steps.
Interview Themes Expressed by Parents Used to Guide Report Design (Aim 1 – Report Development).
Additionally, parents identified the desired structural components (Table 7) that they deemed necessary to facilitate understanding and communication with providers and others.
Final Elements of the Enhanced Genome Report as Determined From Patient and Provider Stakeholder Engagement (Aim 1 – Report Development).
The 2 focus groups specifically tested potential designs for a prognostic table—a component that emerged during the interviews. Parent responses are shown in Table 8. Important elements of the prognosis table included a comprehensive list of all possible signs and symptoms of the condition, how often those signs and symptoms were found, and when the sign or symptom might appear. Depending on the condition, there is a possibility that “early death” could be among the list of signs and symptoms. We did not evaluate parental response to the inclusion of this specific information. Additional information desired included specific direction for action, such as specialists to see or tests to have completed, that related to signs and symptoms listed in the prognosis table. As described above, the testing of this comprehensive prognosis table was a direct result of the study patient co-investigator and other key stakeholders' involvement; without them, the comprehensive version of the table would not have been tested in the focus groups.
Ten providers were recruited for interviews about the report; however, information was repetitive after 5 interviews (saturation). Therefore, interviews with pediatric providers ceased, and 1 additional provider, from adult internal medicine, was added for contrast from the pediatric point of view; this change resulted in 6 physicians who participated in semistructured interviews. Analysis of the coded transcripts resulted in the recognition of 3 constructs around communication of genome sequencing results (Table 9): (1) Providers agreed that WGS results are complex, and they welcomed a report that provided supportive interpretation information to accompany sequencing results; (2) providers strongly endorsed a report that included active clinical guidance, such as reference to practice guidelines; and (3) providers valued the genomic report as a resource that would serve as the basis to facilitate communication of genome sequencing results with their patients and families. Providers also desired a provider-specific report containing information about the technical specifications of the genomic sequencing and the list of variants found in the patient for future reference, and they wanted this information to be in the patient's medical record and accessible during the patient visits. In response to the request by parents for more information about what to expect in the future, a table (see Figure 1 for excerpt) was developed to provide data on prognosis, physical findings, and some behavioral findings. Similarly, providers expressed that they found this prognosis table to be a valuable asset that they would use in their clinical visit with the family. They indicated that the table provided needed information about the rare condition in a format that they were accustomed to using and that it was organized in the way pediatricians think about child development. Providers also envisioned opening the report in the patient's EHR during a clinical visit and admitted they were unlikely to review it beforehand. Providers reported this enhanced report would help them confidently discuss the results with patients during the visit, particularly if they also had access to the patient version of the genomic report. Overall, providers indicated the report was better than the current laboratory report, with one provider stating, “I think this looks great, and I think it is much more helpful than what I currently receive.”
Themes Expressed by Providers Which Guided Report Design (Aim 1 – Report Design).
Deploy the report for patients and families and their providers. The report will be presented to clinicians in the EHR and to patients either through a secure patient portal or by giving access to parts of the EHR.
An electronic platform was required to convey the report content as preferred by parents and providers (aim 1 – development) and to make the information available to both patients and providers within the Geisinger EHR. A web-based tool, COMPASS, already utilized within Geisinger for other purposes, was selected as the vehicle upon which to build the enhanced genomic results report. COMPASS is a software platform developed at Geisinger Health System intended to work as an add-on to EHRs. Its main purpose is to manage secure data exchange as well as patient and provider access to patient-reported data capture and clinical display tools used within the Geisinger Health System. It is intended to improve patient engagement in their care and enhance patient-provider communication. This allowed for an enhanced report that is tied into the patient's medical record, accessible by providers, and available to patients and their parents through the electronic patient portal.
The COMPASS application relied on an administrative access point in which the report could be authored. The medical geneticist who returned results to the WGS study parents authored each of the reports in the administrative access suite of the application.27 The online enhanced genomic results report as deployed can be authored by designated authors for each individual genomic result. The COMPASS tool is set up so that the author can list the gene and variant result found after genetic testing. When genome sequencing is done, 2 types of results can be found. A genetic result that explains the physical findings that led to testing is called the primary result. However, genome sequencing may also lead to results for conditions that were not part of the reason for testing. These are often called incidental or secondary findings. In this population the primary results were expected to pertain to intellectual disability or autism. An example of a secondary finding in this population would be a variant in BRCA1 gene that confers an increased risk for breast and ovarian cancer. Both primary and secondary findings were reported in our study. For both types of results, the report can pull standardized information related to the specific patient from the EHR and relate the gene and variant found, specific diagnosis, prognosis associated with the diagnosis, and resources for parents and resources for providers, through use of the authoring tool. The report can also be updated and reissued as genomic information is revised, with older versions maintained in an archive. The enhanced report composition and display, dubbed GenomeCOMPASS, was found to be easy to navigate and informative by all beta-testers.
Study the impact of the tool from the perspective of providers and family of affected patients.
All parents from the WGS study (N = 84 individuals) were stratified by sequencing result and then randomized as couples before being invited to participate in the trial of the enhanced report. Forty-two parental dyads were eligible to receive enhanced reports (Figure 4). Four individual parents were ineligible to participate: 1 parent died—unrelated to the study—and 3 parents moved away from the study health care system (1 dyad divorced and only 1 parent moved away). Twenty-eight of the total 84 individuals did not complete the baseline survey and therefore did not participate in aim 3. In total, 52 individuals completed the baseline survey and entered the randomized trial (Figure 4). Of the 28 who received the enhanced report (intervention), 21 (75%) completed the 3-month postreport survey and 1 also completed an in-depth interview. Of the 24 parents in the usual care arm, 20 completed the 3-month postbaseline survey (83%) and were sent the enhanced report (crossover condition). Fifteen (75%) completed the 6-month (3 months post-enhanced report) survey as shown in Figure 4.
Randomized Trial (Aim 3) Participant Flow Diagram.
Of individual parents, slightly more mothers than fathers completed the baseline survey (58% vs 42%) and most were White (96%), were married (88%), had at least some college (63%), and were employed (75%). Health literacy and numeracy was high; however, fewer than half reported accessing their child's medical record through the MyGeisinger patient portal despite it being advised for the study and useful for caring for a child with undiagnosed ID or ASD. Only 7 individual parents (14%) received a diagnostic causal variant (CV) result for the child (Table 10).
Baseline Characteristics of Randomized Trial Individual Parent Participants (Aim 3 – Trial) as Reported to ClinicalTrials.gov.
More than 90% of the parent respondents described their own health as good, very good, or excellent. On a 5-point scale of “not confident at all” to “completely confident,” all respondents were a little confident or above in their ability to take good care of their health. This was not different between the intervention group and the usual care group. At baseline, 81.1% of parents rated their child's health as good or better, and parental confidence in their ability to care for their child's health was rated at 94.3% somewhat confident or above.
The participants in this study are admitted information seekers with an overall high information engagement subscale score (mean score = 3) and low information apprehension subscale scores (mean = 0.75). On all items related to information seeking, >90% of respondents affirmed information-gathering behavior. Most (83%) parents in both groups indicated that they use the internet to search for information about health or medical topics for their child, and 49% reported seeking health information from doctors or medical providers.
Most parents had no regret regarding their child's participation in the WGS clinical research study, with 90% agreeing/strongly agreeing that it was the right decision, 90% disagreeing/strongly disagreeing that they regretted the choice, and 89% saying they would do it again, regardless of whether a diagnostic result was found.
The 7 parents at baseline who received a diagnostic result for their child received this scale and reported certainty subscale score of 4.8 (range, 1-5), indicating high baseline certainty in their knowledge about the result before receiving the enhanced report.
Most parents (82.4%) were never or rarely upset or sad (74.5% rarely or never) by their child's result. Nearly all parents (97.9%) reported they never or rarely felt the result made it hard to cope with the child's diagnosis; however, 60.9% also reported sometimes or often feeling frustrated that there were no definite health guidelines for the child.
Based on data from the COMPASS tool, 15 of the 46 available reports were accessed (33%) by parents or providers (9 by parents, 7 by providers). For children with a noncausal variant (NCV) found (n = 39), 6 reports were accessed by parents and 6 by providers. For children where a CV was found that provided a diagnosis for the condition (n = 7), 3 reports were accessed by parents and 1 by providers. One CV parent accessed the report 2 times during the study period and the others 1 time each. Of the NCV reports, 1 report was accessed 3 times, 2 reports were accessed twice, and 3 reports were accessed once.
When the survey results for satisfaction were analyzed for only those parents who reviewed the report, 2 of the 9 parents in the enhanced genomic report first arm (intervention) reported that the report was NOT helpful, while no parents in the crossover arm indicated the report was “not helpful” (see Appendix for all survey data). Further examination revealed that these 2 parents describing the report as “not helpful” had received an NCV result for their child. In the crossover arm, only parents who received a CV result for their child (n = 4) responded to this question, while no parents who received an NCV result answered these specific survey questions. To further understand satisfaction with the report compared with the standard summary letter, we also examined the surveys of the 6 parents who received an NCV result and found that 4 of 6 (67%) reported lower helpfulness of the enhanced report compared with the summary letter (usual care), with 2 of 4 reporting the enhanced report as “not helpful.” One additional NCV parent rated the summary letter (usual care) as “very helpful” at baseline but did not rate the helpfulness of the enhanced report on the 3-month postreport survey (did not answer the survey question). Additional structured interviews with parents revealed that parents with an NCV result did not find value in the report (for those who accessed it) or did not feel the need to open the report in the first place because they had already received the summary letter from the provider and knew no result that could help them care for their child was found. One mother put it best: “The report I want is the one that helps my boys.” In one sense, this was an unexpected finding in that in aim 1, parents involved in the development of the report endorsed that the report would have value even if an NCV result were reported.
Due to the low number of parents who opened the enhanced report (n = 9) and the low number with a diagnostic result (n = 7), evaluating for differences between intervention and control at 3 months or for change between baseline and 3 months postreceipt of enhanced report (3-month survey for intervention, 6-month survey for control) was not informative. One measure, the PAGIS Scale, was included to measure potential impact of the enhanced report using the certainty subscale. This subscale measures understanding of genetic information and ability to explain implications of genetic information to others. Only 4 parents with a diagnostic result from the control arm completed this scale at all time points. Their scores indicated high certainty at baseline (mean scale score, 4.5), low certainty at 3 months postbaseline (no intervention; mean scale score, 2.6), and high certainty again at 3 months post-enhanced report (6 months postbaseline, 3 months postcrossover; mean scale score, 4.6).
Interviews with 2 mothers who both received diagnostic results for their children (both were randomized not to receive an enhanced report but received it 3 months later) best illustrate how this patient-centered enhanced report can be utilized by parents and how this enhanced report tailored for both providers and patients improves the interpretation of complex results and facilitates recommended care by reducing information asymmetry. Both mothers have utilized the enhanced report as a printout, have sent it via email, and/or have accessed it through a phone/tablet when meeting with other physicians inside and outside of Geisinger, when meeting with new teachers and other specialists, and when working with “anyone who is interested in [the child].” This utility for settings outside of health (eg school, therapy, caregivers) was anticipated by parents in the focus groups, as they regarded the content as relevant to myriad settings. The fact that parents used the report this way after implementation is an important finding that is rarely documented in studies of this type (based on our review of the literature to inform the proposal) and further underscores the importance of patient engagement in research like this. Some reports were the following:
She had a new speech therapist and a new occupational therapist, and they were both really grateful to have the information with her syndrome, as far as ways to treat her and help her, and then … she was in the hospital and … the hospitalist … had gone over everything, and used the information in the packet, so I thought that was pretty great.
We were actually at the National Institute of Health a few weeks ago … and they got all of her records from Geisinger except for anything with her mutation on it or saying anything about it. So, I was actually able to print [the report] out for them and give it to them while we were down there so … It's my like go-to place. You know, if I need something quick, I know where it's at, at least.
This access to the genomic testing results when outside Geisinger was critical for this mother, and the report format and prognostic table helped guide her daughter's care in directions that would not have been possible without the report. Importantly, this sentiment was expressed by the NIH providers through personal contact (unsolicited) with the study PI as well as the parent during the interview.
One mother best described the value to her as a parent in terms of being able to communicate with medical professionals, because:
[showing the report is] easier than me trying to explain it to medical professionals when I might not be able to explain it to the degree that they want me to or sometimes they don't think I know what I'm talking about.”
The report has helped her in:
Understanding and then being able to explain to others what's wrong with my daughter…. I felt like it evened the table…. I don't feel so overwhelmed when I'm discussing it with doctors or other medical professionals because I feel like I'm informed enough to know what I'm talking about.
The comment about not feeling overwhelmed because it “evened the table” is particularly salient, as one of the anticipated benefits of the report articulated in the proposal was reduction of information asymmetry. Most important, when asked to sum up the value of the enhanced report to her, this parent explained:
It makes me feel like the best advocate that I can be for my daughter … whether it's medical treatment or it's education, or therapy, I can help guide the direction it goes in because I'm learning and I'm using this tool that I have to understand what she needs.
The participants did not experience adverse events regarding receiving the GenomeCOMPASS report or related to any of the survey items in this phase of the study, nor were any adverse issues related to the report disclosed by parents during the brief or in-depth interviews. One parent appeared to regret the decision per the Decision Regret scale24 at baseline. This was explored further, and this parent was found to have received an uninformative result for a 4-year-old child. This parent's decision regret score did decrease at 3 months; however, it was still in the “regret” range for the scale score. The referent for the regret was the WGS test, indicating that the regret may be related to the genome sequencing not finding an answer for her child rather than to an issue with the enhanced genomic report.
The mixed-methods design of the randomized trial allowed for further evaluation of surveys and interviews. The possible reasons for the low parental access of the report of their child's results which may have been due to multiple factors including: genetic test results initially returned to all patients and their parents before the study began per standard clinical practice, lack of a definitive diagnosis in most of the patients, and unanticipated technical issues during the study
More parents responded that they had opened the enhanced report (n = 20 reports opened) than indicated by COMPASS utilization metrics (n = 15 reports opened). One possibility could be that the usual care condition (before the enhanced report randomized trial) as described above includes in-person discussion with genetics providers and a summary letter that could have been mistaken for the enhanced report by parents not involved in the development or testing of the report.
The generic COMPASS has previously been used almost exclusively for deployment of surveys to clinical care. In this study, the generic language (“a survey is available”) in the COMPASS message did not alert parents to the fact that their child's genome results were now available. Once this was reported, the team developed an instruction sheet to outline the step-by-step process to access the enhanced report; the sheet was sent to all parents when reports were launched, as this language in the subject line could not be changed due to institutional restrictions on patient portal messaging. This letter explained that they would be notified via a MyGeisinger (patient portal) message that “a survey was available” and to follow the instructions on the accompanying handout in order to access their child's genome results report. The letter included the contact information of study personnel for assistance if parents did not receive a message or if they had difficulty accessing the enhanced genomic report.
Follow-up interviews revealed that despite this additional information, parents were still confused about accessing the report. Parents reported that the tool name, COMPASS, had no meaning for them. Even though “COMPASS” is described in the introductory letter, mentioned in the email, and is available as a link in their portal menu, parents expressed varying ideas of what they thought it was—none of which were related to a test report. While parents reported this in interviews after the 3-month surveys were completed, no parents had reached out to project staff for assistance upon receiving the instructions and notice about the enhanced report availability.
Parents also reported that the standard subject line of the email message—”A survey is available”—was problematic because it did not differentiate the report from the other surveys they receive using the same subject line. Parents who do access their child's record through the patient portal commented that they receive so many inbox messages about surveys, lab results, and appointments from their own and their child's accounts that they often do not open the message unless the alert/subject line is distinct in some way.
Finally, despite most parents responding positively to use of the patient portal, MyGeisinger, during the results session of the WGS study and the engagement activities in aim 1 of this study, fewer than half reported accessing their child's record on the survey. Follow-up interviews clarified that some parents were unaware that they had to “connect” to their child's medical record to access that record; parents not connected to the child's MyGeisinger account did not receive the message that the enhanced report was available.
We hypothesized that an enhanced genomic result report (a report created with engagement of end-user parents and providers and available electronically through the medical record system and patient portal) would improve communication and engagement while reducing the information asymmetry that exists between patients, primary care providers, and genetics providers, particularly in the context of rare genetic disease. The primary outcomes included measures of the actual use of and satisfaction with the enhanced genomic report. As listed in Table 4, primary and secondary outcomes were obtained using qualitative and quantitative measures. Through the use of robust mixed-methods design, we found that when reporting on a diagnostic result, the online report facilitated communication between parents, medical providers, and other professionals caring for the affected child. This was evidenced in the qualitative interviews with parents who remarked that they now felt confident to be an advocate for their child. One parent described that she had shared the report with her daughter's occupational and speech therapists, who were also grateful for the information that the report contained.
The genomic report developed in this project relied on the parents' perspective of what constituted the important elements in a genomics result report. Parents envisioned a report that would enable them to understand their child's genomic result and feel confident about the implications for health care management. The themes that parents identified focused on wanting to understand their child's complex set of care requirements from the medical, social, and family perspectives and echoed what others have found in evaluation of families with children with special health care needs.28 The search for information and understanding is ever present in their lives, and our parents reported being driven to find whatever information is pertinent for their child's particular set of symptoms. They wanted to know “what the experts know.” If and when a diagnosis was reported (eg, a genetic explanation was found), parents wanted to have the actual genetic result and they wanted to know what it meant for their child. The parents were unanimous in their desire for a report containing clear, concise information about the genetic finding, the diagnosis, and what the diagnosis meant for the health management of their child.
Of greatest importance for the parents was the ability to understand the full scope of their child's medical diagnosis. During the design phase, parents articulated the desire to anticipate health issues and to plan for potential challenges in the future. Results from the randomized trial demonstrated that the GenomeCOMPASS report offered parents a central place to access specific and accurate diagnostic and prognostic information to anticipate and direct appropriate care, as well as a point of access to vetted web-based lay resources relevant to their child's disorder.
Another critical advantage that parents identified involved the use of the genomics report to aid in educational planning for their child. Parents reported that they would take it to their child's school to inform an Individualized Educational Plan, thereby enhancing understanding of this child's rare diagnosis; this use was not anticipated when the report was conceived but was identified as important by parents during the interviews in aim 1 and actualized during the randomized trial (aim 3). Finally, interview data with parents indicated that the low use of the GenomeCOMPASS report was due to a lack of utility when results were negative rather than a deficit in the enhanced report itself, while those who received a diagnostic result reported utilizing their GenomeCOMPASS report exactly as designed. The intentional use of mixed-methods design was critical to the evaluation and understanding of study results, which were hampered by limited sample size and unanticipated low number of patients who received a diagnostic result. Inclusion of qualitative assessment within the study design illuminated important issues of access and satisfaction—and revealed potential lack of value in a report to parents when their child's result was negative.
Providers readily admitted their lack of knowledge about specific genetic findings and associated rare genetic conditions, particularly regarding what to do for their patient given the genetic result. The location of the report within their patient's EHR and its accessibility at the point of care were critical to the acceptance of the GenomeCOMPASS report. The enhanced report offered providers a “one-stop shop” for information specific to the diagnosis and the option to go deeper via the weblinks included in the report, which allowed access to vetted content of relevance to providers. Most important, providers saw the report as an aid to communication for parents and for themselves: it offered a reference point for understanding the diagnosis, anticipating medical issues, and facilitating discussions about management. Through unsolicited contact with external providers who utilized the report, we also demonstrated that this functionality was transferrable, appreciated, and critical to providers outside the Geisinger system, thus addressing the anticipated need for availability beyond a single health care system.
Curative treatment is relatively rare in the setting of autism and intellectual disability. Establishing an etiologic diagnosis for autism and intellectual disability helps to guide medical management and anticipatory guidance (many syndromes have management guidelines), establish prognosis (something that our participants found most valuable about the report), provide information for reproductive decision-making, assist with planning for developmental and educational interventions, and end the diagnostic odyssey with its attendant costs and burden on the patient and family. These dimensions are frequently discounted or not measured in studies focused on traditional utility and cost of care; however, from a patient- and family-centered perspective, they are of high value. Indeed, communication of this critical information was the primary reason for the project. The GenomeCOMPASS report supports many of the tenets of patient-led care, an emerging paradigm for patient-centered care. This tool represents a scalable method to present genomic information to support increased understanding of rare disease by patients and providers. Many studies report the lack of genetic/genomic literacy among the US population and among nongenetics providers.14,15,29 Although the number of users was small, this study supports the hypothesis that customizable template reports may provide a beneficial and durable source of information. With additional study, online reports may be found to support and enhance the information provided by genetics professionals in traditional face-to-face encounters. This tool, which can be adapted in many EHR settings, could prove useful to disseminate accurate genetic information, provide real-time management support, and connect parents and nongenetics providers with appropriate resources regarding rare conditions. This is an important step in the process to make genomic information available and relevant for patients and providers.
Our parent participants and providers confirmed the challenges of interpreting and communicating genetic results, a point that is repeatedly made in the literature and clearly has not been addressed, since this concern has not changed over the past decade.30-32 The major focus is about advancing current understanding—that is, how providing the patient/family with the report empowers them, reduces information asymmetry, enhances communication, and allows them to be equal partners in the management decisions regarding their child. While previous research supported that behavior based on a hypothetical construct, we recognize that hypothetical behavior is often different from that undertaken once the real result is communicated. We anticipated that our report of negative results would be valuable to parents and families based on their feedback in the development phase of our report; however, in reality, the parents did not find the report useful when their child received an uninformative genomic result. It is possible that had the GenomeCOMPASS report been the communication from the results visit instead of the usual care summary letter, these parents would have found value in the report.
Likewise, as determined from the qualitative interviews, for parents who received a diagnostic result, the enhanced report as implemented did, in fact, empower parents, reduce information asymmetry (per a parent's own words), and enhance communication between parents and many different providers and caregivers inside and outside of the health care system.
After developing the reports (using user-centered design with patients and providers) and testing the report compared with standard of care, we approached the leadership of Geisinger about using the report for our large-scale exome sequencing project. We needed a process to return these results at scale, as we anticipate returning medically significant genetic results to several thousand Geisinger patient-participants. We presented the reports and the effectiveness results to large numbers of internal stakeholders, including several patient advisory groups, providers, and administrative and technical staff (probably 100-200 stakeholders in total). The patients were very enthusiastic about the report and, given our system's emphasis on the patient voice and experience, the leadership endorsed use of the report. Geisinger is a learning health care system, so once the reports are deployed at scale, we will continue to engage with patient and provider end-users to gather information on the reports and, using approaches grounded in improvement and implementation science, we will continue to improve the report (or if we find it's not working, explore other options).
Several external groups have asked about the possibility of using this report and participating in dissemination efforts. The involvement of clinicians involved in the return of genetic results (as was tested in this project) as well as evidence that reporting was extended to other types of genetic information (eg, pharmacogenomic results,3, tumor testing results) indicates the appeal of the GenomeCOMPASS generated results report. This could be viewed as surprising, given that the evidence generated by the study is difficult to characterize as definitive and given the small numbers and limitations discussed below. However, we believe several reasons could explain the enthusiasm for the report:
These data indicate that this report was useful for reporting genomic sequencing results for children with intellectual disability, autism spectrum disorder, multiple congenital anomalies, and seizures. The COMPASS tool has been integrated throughout Geisinger, and the GenomeCOMPASS report has been approved for use for return of all genomic results— not only for the rare disorder indication. The report design takes advantage of reusable text banks created and vetted for standard information that includes most common secondary findings (eg, BRCA-associated hereditary breast and ovarian cancer syndrome, Lynch syndrome, Familial Hypercholesterolemia), and the reports are available for use by all clinical genomic sequencing applications. To address barriers identified during the study, additional changes have been made related to email subject lines and messaging. Finally, technological barriers that have proved difficult to overcome in other large-scale genomic projects were minimized here by using a preexisting application with the ability to interface with several different vendor-based EHR systems beyond Geisinger. By building the enhanced genomic reports on this preexisting EHR-compatible application, we have dramatically increased the ability of the enhanced genome report to be used during routine clinical care within Geisinger and in other health care systems.
A significant implementation barrier existed because of the messaging available through the patient portal; the study team is using its experience to improve the messaging. At the outset of the study, parents reported that they used the patient portal for communication about health care; however, after the implementation of the tool for the randomized trial, we learned that parents may not be aware that they need to have proxy access to their child's record (in addition to their own) to receive messages and access the COMPASS genome report. As a work-around for the effectiveness trial and to try to improve uptake of the report, parents were sent instructions for connecting to their child's record and for accessing GenomeCOMPASS, and they were provided access to study staff for assistance when reports were released.
The rare disorders addressed by the enhanced genome report occur in all population groups regardless of race, ethnicity, or socioeconomic status. Currently, this application is available in English only. While the content can be translated into other languages, additional customization through appropriate engagement is required to define culturally sensitive and relevant presentation for other populations. Although we believe that the enhanced genomic report should perform well across all demographic groups, given the universal impact of rare genetic disorders, further deployment and evaluation of effectiveness in a broad range of patient end-users is desirable.
The COMPASS tool is constructed as an application that interacts with EHRs through a standardized application program interface. This dramatically reduces the barriers to the use of applications like GenomeCOMPASS. Therefore, enhanced GenomeCOMPASS reports should be functional for other health systems using those EHR systems. In anticipation of a proposal for the limited PCORI dissemination funding opportunity, conversations with information technology experts and informaticists at several other institutions about the technical compatibility of the GenomeCOMPASS tool with respective system information systems have been conducted. These consultations confirm that there are no substantial technical barriers to implementation with the information systems.
The GenomeCOMPASS reports are ready for dissemination with a broader range of genomic information types in a variety of health settings—provided that issues related to messaging about availability of the report and the need for parent proxy access to the child's EHR are addressed in the deployment plan. Use of the report can be evaluated in these new settings with regard to communication of genomic results to patients and the potential to facilitate communication between patients and providers. Improved communication is likely to lead to improved adherence to recommended care and to promote patient-centered outcomes.
Despite the small sample size, the survey results and interviews suggest that the report may not be useful or valued by the subpopulation that receives negative (uninformative) results on genomic sequencing.
The most significant limitation of the study stems from the limited sample of engaged and invested parents, a quarter of whom (13 of 52) served as both stakeholders in the report development and participants in the testing of the impact of the report as implemented. Furthermore, evaluation of the report impact was limited by the relatively small number of individuals who received a diagnostic result (n = 7). The initial report design and randomized trial analyses were based on receiving feedback for both the diagnostic and uninformative reports, an approach that was endorsed by the patients and families engaged at the outset of the project. However, upon clinical implementation, most (39/45 or 87%) parents who received an uninformative clinical result did not interact with the enhanced report as anticipated, limiting the numbers available for effectiveness analysis. Additionally, some of the parents who participated in the design of the report content also received the computerized enhanced report as part of the clinical trial. Furthermore, the final sample of 52 total participants (24 in 1 arm and 28 in the other) did not reach the statistical power to detect differences in any of the survey measures chosen for the study. While we planned to examine overall responses to survey measures, differences in responses between fathers and mothers, differences in responses based on result (causal variant and negative), and differences in responses to the enhanced report based on other demographic variables, only descriptive statistics could be utilized in the final analyses due to sample size. However, because of the mixed-methods design built a priori into the study methods, we were able to utilize qualitative data collected during the developmental phases and after deployment of the enhanced report to understand the quantitative survey results more robustly than would have been possible through descriptive statistical analysis alone.
Through interviews, the evaluators of the diagnostic result report did endorse the value and impact of the report in the domains defined by the evaluation framework. Additionally, in the small number of parents interviewed, the magnitude of the endorsement signal was qualitatively high and uniformly positive. The report has been presented in a variety of different venues engaging with multiple provider and patient stakeholder groups. All feedback from this engagement has been positive and reinforces the value of this type of reporting strategy.
The project also experienced technical difficulties that interfered with families' access to the report. While a work-around in the form of instructions sent to parents was used to address these issues in the trial, this did not address the unanticipated consequence that families receiving an uninformative result felt no need to access the report regardless of changes made to the messaging.
Our study population also scored high on literacy and numeracy scales at baseline despite the Geisinger rural catchment area; therefore, further evaluation of the enhanced report with individuals with lower health literacy/numeracy is needed. Finally, the WGS results return process (usual care arm) was very thorough and perhaps contributed to a ceiling effect on surveys for many outcome domains. As shown in the baseline and follow-up surveys, parents already felt highly informed and confident in dealing with their child's condition, and the summary letter of the results visit typically sent afterward may have been confused with the enhanced report, thus resulting in more parents reporting accessing the enhanced report and limiting the ability to detect change using the survey measures. The mixed-methods study design, including triangulation of the survey data with qualitative data obtained from postreport interviews, minimized the impact of this potential ceiling effect and allowed us to determine the value and impact of the enhanced report to parents and providers.
While we believe that the patient-facing report should perform well across all demographic groups, given the universal impact of rare genetic disorders in these groups, deployment and evaluation of the report to a broad range of patient end-users and assessing the utility considering factors—such as race/ethnicity, socioeconomic status, educational level and health literacy, and underserved and vulnerable populations—is desirable. Making the report available in different languages is important, although it does not define a research agenda in and of itself.
In the future, such an enhanced report could perhaps replace the summary letter that is currently sent by genetics professionals following clinic visits with the family. The report represents the opportunity for study of workflow improvement in genetics clinics, as it provides a report template with ready access to patient and provider resources that can be deployed through the patient-accessed site in the EHR. Future research could focus on dissemination and implementation in different settings and adaptation for diverse indications. We are pursuing these opportunities with multiple partners. Geisinger will be using this tool for return of actionable results in its large-scale sequencing project.34 Of significance was a request to present the tool to the Steering Committee of the NIH-funded Undiagnosed Disease Network (UDN). Following the presentation, UDN investigators expressed interest in partnering with the investigators in this project to use the report developed here for the patients and families evaluated through the UDN. In an entirely different application, the use of this report template is being adapted for the return of pharmacogenetic results,33 an application of the report template that could affect every patient who undergoes sequencing, as all of us carry genetic variants that influence response to medication. In yet another application, researchers returning cancer tumor testing results to patients and providers are looking to adapt the template and principles to meet the need of explaining this complex genomic technology and application for patients with cancer.
This project created and tested a patient-centered, interactive, accessible genomic test report for patients and providers, with the goals to explain results from WGS studies on children with intellectual disability, to facilitate understanding of the results, and to enhance patient-provider communication. Impact of the enhanced genomic report was assessed using a mixed-methods framework in a randomized asymmetric crossover trial. Of 46 enhanced reports, only 9 were accessed by parents. Because of the low uptake of the enhanced reports, the randomized trial was not informative. In-depth interviews with 2 mothers (both were randomized not to receive an enhanced report but received it 3 months later) best illustrate how parents can utilize the enhanced report. Both mothers used the report when meeting with other physicians and with teachers and other specialists. They indicated the report empowered them in these conversations with professionals. Unsolicited communication from external providers also confirmed the value of the enhanced report to providers for facilitating communication and guiding the most effective care for the child. Although the number of users was small, this study supports the hypothesis that customizable template reports may provide a useful and durable source of information that can support and enhance the information provided by genetics professionals in traditional face-to-face encounters. Reports that addressed negative findings (ie, uninformative results) were found to be less useful to parents.
Further research is needed to focus on the implementation and effectiveness of the enhanced report in different settings and in different populations. Confirmation of the impact, effectiveness, and utility across a broad range of patient end-users regarding race/ethnicity, education level, lower health literacy, and underserved and vulnerable populations is also desirable. We plan to facilitate dissemination of this approach in a variety of settings to enable this future research. This dissemination, coupled with enhancing the interoperability of the tool, should have a significant impact on the appropriate use of genomic information in clinical care.
Research reported in this report was [partially] funded through a Patient-Centered Outcomes Research Institute® (PCORI®) Award CD-1304-6987). Further information available at: https://www.pcori.org/research-results/2013/evaluating-impact-user-friendly-report-genetic-testing-results-parents
Baseline Questionnaire (PDF, 1.0M)
Supplementary Table 1. Survey results Intervention group at baseline and 3 months post report (PDF, 366K)
Appendix to Ancillary Information (PDF, 75K)
Williams M, Williams J, Rahm A, Bonhag M, Stuckey H, Zallen D. (2018). Evaluating the Impact of a User-Friendly Report of Genetic Testing Results for Parents of Children with Genetic Disorders. Patient-Centered Outcomes Research Institute (PCORI). https://doi.org/10.25302/9.2018.CD.13046987
The [views, statements, opinions] presented in this report are solely the responsibility of the author(s) and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute® (PCORI®), its Board of Governors or Methodology Committee.
This book is distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License which permits noncommercial use and distribution provided the original author(s) and source are credited. (See https://creativecommons.org/licenses/by-nc-nd/4.0/
Your browsing activity is empty.
Activity recording is turned off.
See more...
