Patient Input Information

Two patient groups, Cystic Fibrosis Canada (CF Canada) and the Cystic Fibrosis Treatment Society, responded to CADTH’s call for patient input. Patient perspectives were obtained from a national survey conducted in January 2018 and February 2018, testimonials from patients with CF and their families, CF Canada publications (e.g., 2016 CF Canada Registry Annual Data Report), and data from SickKids hospital and Genome Canada. The following is a summary of key input from the perspective of the patient groups:

• Managing CF requires a demanding treatment routine with regular visits to specialized CF clinics. CF treatments, CF-related infections, and hospitalizations take a toll on patients’ emotional stamina and have a significant impact on their day-to-day quality of life, affecting life decisions that include education, career, travel, relationships, and family planning. Treatments that may consume two to seven hours per day and may include hospitalizations disrupt family routines.
• Caregivers are faced with significant emotional, psychological, physical, and financial burdens. They may feel helpless and devastated, watching their loved ones cope with a life-threatening disease. Social activities and employment can be significantly affected in order to accommodate the treatment of a loved one with CF. It was also noted that limited access to new treatments (e.g., LUM/IVA) can have a negative impact on the mental health and well-being of patients and caregivers, due to perceptions of unfair access to what could be a life-altering drug.
• Patients indicated that there is a need for additional CF treatments that can prolong their lives, improve their quality of life, allow them to work, study, and participate more fully in social and physical activities, reduce the need for hospitalizations, avoid the need for lung transplantation, help them gain weight, and reduce the frequency and severity of pulmonary exacerbations. The majority of the 68 patients and caregivers who described their own or their loved ones’ experience with LUM/IVA reported improved lung function, a reduced rate of exacerbations, and improved nutritional status, and emphasized the cumulative positive effect on their quality of life.
• Patients are willing to tolerate adverse effects if they believe the potential benefits outweigh the possible side effects. The patient groups noted that some patients reported uncomfortable side effects, including tightness in their chest when starting treatment with LUM/IVA, but that the symptoms faded within weeks or months. Patients also cited the affordability of LUM/IVA as a significant concern.

Clinical Trials

The CDR systematic review included four double-blind, placebo-controlled, RCTs (TRAFFIC, TRANSPORT, Study 112, and Study 109) and one pivotal, single-arm, open-label trial (Study 11B). In addition, the CDR review included the following studies as supplemental information: two extension phase studies (PROGRESS and Study 110) and a single-arm study conducted in patients with severe lung disease (Study 106). The study populations consisted of patients who were either six years to 11 years of age (studies 109, 110, and 11B) or patients who were at least 12 years of age (TRAFFIC, TRANSPORT, PROGRESS, Study 106, and Study 112). All of the studies included a screening phase (up to 28 days), an investigational treatment period (24 weeks), and a safety follow-up phase (approximately four weeks). The use of a placebo as the comparator in the RCTs is appropriate as LUM/IVA is currently the only treatment approved in Canada for use in the treatment of CF in patients with F508del-CFTR mutations. All of the studies compared the addition of LUM/IVA (or placebo) to ongoing standard CF-management therapies, which is reflective of how LUM/IVA would be administered in routine clinical practice.

Outcomes

Outcomes were defined a priori in the systematic review protocol. Of these, the committee discussed the following:

• ppFEV₁ — calculated using the ratio of forced expiratory volume in one second (FEV₁) to the predicted FEV₁. Changes in ppFEV₁ were evaluated as follows:

  ▪

  Absolute change in ppFEV₁: Calculated as post-baseline value minus baseline value.

  ▪

  Relative change in ppFEV₁: Calculated and expressed in percentages as 100 × (post-baseline value – baseline value)/baseline value.

  ▪

  Proportion of patients with an improvement of ≥ 3%, ≥ 5%, and ≥ 10% in average absolute change from baseline in ppFEV₁ at week 16 and week 24.

  ▪

  Proportion of patients with an improvement of ≥ 5% and ≥ 10% in average relative change from baseline in ppFEV₁ at week 16 and week 24.

  At the time of this review, there is no established MCID for absolute or relative changes in ppFEV₁ for patients with CF.
• Pulmonary exacerbations were defined as a change in antibiotic therapy for any four or more of the following signs or symptoms: change in sputum; new or increased hemoptysis; increased cough; increased dyspnea; malaise, fatigue, or lethargy; temperature above 38°C; anorexia or weight loss; sinus pain or tenderness; change in sinus discharge; change in physical examination of the chest; decrease in lung function by at least 10%; or radiographic changes indicative of pulmonary infection. CDEC considered the following end points related to exacerbations:

  ▪

  number of pulmonary exacerbations from baseline to week 24

  ▪

  time to first pulmonary exacerbation

  ▪

  incidence of having at least one pulmonary exacerbation

  ▪

  pulmonary exacerbations requiring hospitalization

  ▪

  time to first hospitalization for pulmonary exacerbation

  ▪

  pulmonary exacerbations requiring intravenous (IV) antibiotics

  ▪

  time to first IV antibiotic therapy for pulmonary exacerbation.

• Lung clearance index is a multiple breath washout test that estimates the number of lung volume turnovers required to clear the lung of an inert gas. Absolute change from baseline in LCI2.5 and lung clearance index 5% (LCI5.0) were evaluated in studies 109 and 11B, and represent the number of lung turnovers that are required to reduce the end tidal concentration of the inert gas to 2.5% and 5.0% of the starting value, respectively. Each multiple breath washout assessment was performed three times at the study visit. At the time of this review, there is no established MCID for changes in LCI2.5 for patients with CF.
• Changes from baseline in BMI, body weight, and height — these end points were adjusted for age and sex, and analyzed as BMI-for-age z score, weight-for-age z score, and height-for-age z score for patients 12 years to 20 years of age in TRAFFIC and TRANSPORT and all patients in studies 109 and 11B.
• CFQ-R is a disease-specific instrument used to evaluate changes in respiratory symptoms, digestive symptoms, emotion, and health perception. A difference of at least four points in the respiratory domain score of the CFQ-R has been cited as the MCID.
• EuroQol 5-Dimensions 3-Levels questionnaire (EQ-5D-3L) — a generic utility measure of health-related quality of life used to evaluate the current health states of patients at least 12 years of age. The MCID for the EQ-5D-3L in patients with CF is uncertain.
• Total adverse events (AE), serious adverse events (SAEs), and withdrawals due to AEs.

Absolute change from baseline in ppFEV₁ was the primary end point of TRAFFIC and TRANSPORT. Both studies also included the following five key secondary end points in a statistical testing hierarchy: average relative change from baseline in ppFEV₁ at week 16 and week 24; absolute change from baseline in BMI at week 24; absolute change from baseline in CFQ-R respiratory domain at week 24; ≥ 5% increase in average relative change from baseline in ppFEV₁ at week 16 and week 24; number of pulmonary exacerbations through week 24. Failure to demonstrate statistically significant differences stopped the statistical testing hierarchy at BMI in TRAFFIC and at CFQ-R respiratory domain in TRANSPORT.

Efficacy

Harms (Safety and Tolerability)

Cost and Cost-Effectiveness

At the current marketed price of $170.54 per tablet, the daily cost of treatment per patient with LUM/IVA is $682, or $248,982 annually.

The manufacturer submitted a cost-utility analysis to assess the cost-effectiveness of LUM/IVA + standard of care (SoC) compared with SoC alone in patients with CF who are six years of age or older and homozygous for the F508del-CFTR mutation. SoC comprised nutritional support, airway clearance, and treatment of clinical manifestations such as lung infections. The analysis is based on an individual patient simulation model estimating long-term health care costs and quality-adjusted life-years (QALYs) over a lifetime horizon (119 years) from the perspective of the Canadian public health care payer. Clinical efficacy estimates (based primarily on ppFEV₁) were obtained from Study 109, the TRAFFIC and TRANSPORT clinical trials, and the PROGRESS extension study. In the manufacturer’s base case, it was reported that LUM/IVA + SoC was associated with greater QALYs and higher costs than SoC alone, with an estimated incremental cost per QALY gained of $446,529.

CADTH identified a number of limitations with the manufacturer’s submitted economic model:

• The model lacked transparency, which led to difficulties in assessing the validity of the coding within the model, with particular difficulties in assessing the methods used within the adopted micro-simulation technique. The complexity of the model ultimately led to the necessary probabilistic analysis being too unwieldy to run.
• The probabilistic analysis did not meet requisite standards as there were inappropriate assumptions relating to the specification of certain probability distributions and not all uncertain parameters were made probabilistic.
• No stratification by age was presented (i.e., results were presented with all patients aged six and over considered as a single group). The manufacturer’s submission from 2015 was for those aged 12 and over and no new data with respect to the relative efficacy of LUM/IVA compared with SoC were supplied for this subgroup. CADTH economic guidelines specifically indicate that when input parameters vary by characteristics of patients that are likely to impact results, the analysis should be stratified by these characteristics.
• The manufacturer assumed that over time the differences in ppFEV1 between LUM/IVA + SoC and SoC would increase. This assumption is not supported by the comparative clinical trial data given that within the clinical trials in both age groups, results suggested that the benefit from ppFEV1 changes occurs in the initial eight weeks of treatment with curves relating to ppFEV1 staying parallel after this period. This suggests a continuance of benefit but not an extension of benefit.
• The manufacturer assumed that after 12 years within the model, the cost of LUM/IVA + SoC would be reduced by 82% due to a generic equivalent becoming available. In addition, the manufacturer assumed patient compliance with LUM/IVA would be
• 96.46% each year and thus reduced drug costs accordingly.
• Finally, in addition to assuming a relationship to a reduction in exacerbations through improvements in ppFEV1, the manufacturer incorporated an additional assumption of a further 55% reduction in exacerbations with LUM/IVA + SoC after age 12. This is unlikely to be justified and could potentially lead to double counting the potential benefit from LUM/IVA + SoC. In addition, the results of the Study 109 pediatric efficacy trial found a higher rate of exacerbations on LUM/IVA + SoC, yet the manufacturer modelled a reduction in exacerbation rates in this group.

CDR addressed the issues pertaining to treatment costs, exacerbation effects, and ppFEV₁ effects in reanalyses. The incremental cost-utility ratio (ICUR) for LUM/IVA + SoC versus SoC alone in patients aged 12 and over is $3.8 million per QALY and $7.3 million per QALY for patients aged six years to 11 years. A 98.5% price reduction for LUM/IVA is required for the ICUR in both populations to be less than $50,000, or 97% for an ICUR of $100,000.

CDEC Response

CADTH’s clinical review report notes that there are differing opinions regarding the MCID for ppFEV₁ in CF patients. In the absence of a formal study to determine the MCID for ppFEV₁ in CF patients, CDEC’s interpretation of this end point was based on the clinical studies for LUM/IVA, the literature regarding ppFEV₁ that was summarized by CADTH, and the opinion of Canadian clinical experts with experience in the diagnosis and management of patients with CF. The following was noted by the committee:

• The sample size calculations used in the TRAFFIC and TRANSPORT studies were based in part on an absolute difference in ppFEV1 of 5%, which suggests that the investigators and manufacturer thought that such a change represented a clinically important difference when designing the studies.
• The clinical experts consulted by CADTH and CDEC noted that CF specialists would generally consider an absolute improvement in ppFEV1 of at least 5% to be clinically relevant; a threshold that was also cited by the clinical expert consulted by the National Institute for Health and Care Excellence (NICE) Evidence Review Group in the UK.
• Health Canada reviewers noted that a clinically relevant outcome for patients with CF could include stabilization of lung function, an improvement in the rate of decline of lung function, or a marginal improvement in lung function, but did not note a specific threshold.

Overall, the absence of a validated MCID for ppFEV₁ in CF remains a concern for the committee and makes it challenging to interpret the results of the clinical studies for LUM/IVA given that this pulmonary function measure was used as the primary end point in the trials conducted in patients who were at least 12 years of age (i.e., the TRAFFIC and TRANSPORT trials).

The manufacturer conducted a number of responder analyses based on a ≥ 3%, ≥ 5% and ≥ 10% absolute change from baseline in ppFEV₁ in both the TRAFFIC and TRANSPORT studies. CDEC carefully considered the results of the analyses based on three factors: the ability of the committee to draw inferences regarding the efficacy of LUM/IVA from these analyses; the magnitude of the observed responses; and the extent to which these improvements in ppFEV₁, a surrogate end point, would translate into improvements in clinical outcomes for Canadian CF patients. The committee noted the following:

• Although a greater proportion of patients achieved absolute improvements in ppFEV1 of ≥ 3%, ≥ 5% and ≥ 10%, these analyses were not primary or key secondary end points of the studies and were analyzed outside of the statistical testing hierarchy. Failure to conduct adjustment for multiplicity means that the responder analyses are inconclusive due to the inflated risk of a type I error (i.e., there was no control over the chance of a false-positive finding). This uncertainty limited the committee’s ability to evaluate the efficacy of LUM/IVA based on the responder analyses.

Based on the factors noted above, CDEC concluded that there is considerable uncertainty regarding the clinical relevance of the results for the responder analyses. Overall, these analyses were insufficient to support a reimbursement recommendation for LUM/IVA.

The embargoed recommendation notes that there is uncertainty regarding the clinical significance of the lower rate of pulmonary exacerbations reported in the TRAFFIC and TRANSPORT studies. Could CDEC provide clarity on the potential clinical relevance of the results for the other measures of pulmonary exacerbations, including those that required treatment with intravenous antibiotics (IV) and/or hospitalization? In addition, could the committee please comment on how the potential impact of a reduction in these events on the Canadian heath system was considered in their deliberation and recommendation?

CDEC Response

CDEC noted that the treatment effect sizes reported for reducing pulmonary exacerbations requiring hospitalization through 24 weeks (rate ratio: 0.39 [95% CI, 0.26 to 0.56]) and requiring IV antibiotics (rate ratio: 95% CI, 0.44 [95% CI, 0.32 to 0.59]) could be clinically relevant. However, these analyses were conducted outside of the statistical testing hierarchy; they were not key primary or secondary end points when the studies were designed. Similar to the ppFEV₁ responder analyses, failure to conduct adjustment for multiplicity means that the results for pulmonary exacerbations requiring IV antibiotics or hospitalization are inconclusive due to the inflated risk of a type I error (i.e., there was no control over the chance of a false-positive finding). Moreover, 24 weeks is likely too short of a study duration to assess exacerbations as an outcome; longer-term trial designs (ideally at least one year) are required. This uncertainty compromised the committee’s ability to evaluate the efficacy of LUM/IVA based on these analyses and it was concluded that these outcomes do not provide a sufficient evidentiary base to support a reimbursement recommendation for LUM/IVA.

CDEC noted that the potential impact of a reduction in pulmonary exacerbations requiring IV antibiotics or hospitalization on the Canadian heath system was incorporated into the manufacturer’s economic submission to CADTH. This is reflected in the cost-effectiveness estimates reported by CADTH and were considered in CDEC’s deliberations.

Could CDEC comment on the likelihood that improvements in exacerbations being seen in both the TRAFFIC and TRANSPORT studies were not caused by the treatment effect of Orkambi? Also, could CDEC clarify if the limitations of the studies (e.g., short duration of trials, lack of independent adjudication) would be expected to bias the results? If so, in what way would this bias the results?

CDEC Response

Regarding whether or not the reductions in pulmonary exacerbations reported in the TRAFFIC and TRANSPORT studies could be attributable to treatment with LUM/IVA requires consideration of the statistical methods used in the analysis (e.g., could it be explained by chance alone) and the internal validity of the study results (e.g., could the results be attributable to bias in the study). CDEC considered the following information:

• As noted in CDEC’s reasons for recommendation, failure of the manufacturer’s pre-defined statistical testing hierarchy prevents CDEC from concluding that these results were not due to chance alone.
• The clinical experts consulted by CADTH noted that the results are likely attributable to treatment with LUM/IVA. Limitations with the analyses include the statistical issues that were previously noted, the absence of an independent adjudication committee to confirm the events, and the short-term nature of the controlled portion of the TRAFFIC and TRANSPORT studies. It was noted that pulmonary exacerbation studies should be at least one year in duration in order to account for seasonal variation in the frequency of events.
• There were some imbalances between the treatment groups with respect to prior and concomitant usage of inhaled antibiotics and dornase alfa. CDEC noted that it is unlikely that such imbalances alone could explain the overall reduction in pulmonary exacerbations observed; however, it is possible that these imbalances had an impact on the magnitude of the effect size; that is, make the real magnitude of any observed effect uncertain. Moreover, given the nature of these imbalances, it is impossible to know if they biased the treatment effect in favour or against treatment with LUM/IVA (i.e., could the additional usage of these drugs have a protective effect or be a marker for treatment of a more severe CF disease state).

CDEC (and CADTH) noted that the magnitude of any potential bias (such as that from imbalances in baseline patient characteristics) on the estimated treatment effect of LUM/IVA for pulmonary exacerbations could not be calculated.

The embargoed CDEC recommendation stated that in a matched-registry cohort analysis that the slope of decline in lung function was reduced in patients treated with LUM/IVA by ‒1.33% versus ‒2.29%. Could CDEC clarify if this difference in the rate of the decline would be considered clinically meaningful? CDEC also noted concerns with the generalizability of this finding; could CDEC further elaborate on the implications this has for Canadian patients?

CDEC Response

CDEC carefully considered the results of the matched cohort study reported by the manufacturer and discussed the findings with Canadian clinical experts in CF. Overall, this study is not sufficient to conclude the LUM/IVA has a clinically relevant impact on the slope of decline in lung function. There are significant limitations to an analysis of registry patients matched to patients in a randomized controlled trial. As noted in CADTH’s report and by the clinical experts, there were many important issues with this analysis that limit the ability of CDEC to use this information for the purposes of making reimbursement recommendations in the Canadian context. The following key issues with the study may have biased the results in favour of L400/IVA: use of registry patients exclusively from the US as it has been documented that outcomes for CF patients in the US are worse than Canadian CF patients; the generation of propensity scores did not include important potential confounders (e.g., pulmonary exacerbation frequency and socioeconomic status); the balance across the full range of patients and important subgroups were not presented, thus whether balance was fully achieved and how this may have affected the study results is uncertain. Trial patients were selected based on enrolment in the PROGRESS open-label extension study, which excluded patients intolerant or non-adherent to LUM/IVA, and therefore may represent patients more likely to show positive treatment effects. Approximately 19% of all patients dosed in TRAFFIC or TRANSPORT were not included in the matched analysis. How these patients relate to those included in the analysis in terms of characteristics and outcomes is unknown and exclusion of these patients may introduce bias into the study, particularly given 7% were excluded for reasons related to pulmonary data during follow-up.

CDEC also noted that the authors of this study stated the following in the publication:¹

Although the slowed rate of lung function decline is probably related to treatment with lumacaftor/ivacaftor, causality cannot be definitively established in the context of this analysis. Patients who participate in clinical trials might differ systematically from those who do not, although clinical trial participation is higher in patients with cystic fibrosis than those with many other disease states. Furthermore, 21% of the registry patients in the matched control group were enrolled in at least one clinical trial in 2013 or 2014. A propensity score approach was used to balance covariates associated with the rate of lung function decline between groups; however, the ability to match patients on risk factors of lung function decline is limited to the variables collected in both the registry and the clinical study. Furthermore, the registry included data from patients in the USA, whereas the PROGRESS study enrolled patients not only in the USA but also in Canada, Europe, and Australia, where the cystic fibrosis population might differ. With respect to our statistical model, estimation of average annual rate of decline was based on FEV₁ measurements spanning different lengths of observation for different patients. The model also assumed that the rate of decline in FEV₁ was constant over the observation period for each patient.

In summary, these limitations, related to the observational nature of the analysis and mixing of two distinct cohorts, mean that this evidence is not strong enough to support causal inferences and decision-making, especially when considered in the context of the main study findings. Furthermore, any estimates of the magnitude of the effect for Canadian patients will be limited by the inclusion of only US patients in the matched cohort registry, as there are a number of differences between the US and Canada with regards to the management of CF and US patients have been reported to have a poorer prognosis compared with those in Canada.

The embargoed CDEC recommendation states that the clinical relevance of the results for several end points were uncertain (e.g., BMI and LCI2.5). Could CDEC provide clarity regarding the nature of this uncertainty? For example, is the uncertainty related to the magnitude of the effect observed, the characteristics of the study design (e.g., duration of follow-up), or the selection of the end point?

CDEC Response

The uncertainty with the results for BMI is attributable to three important considerations: failure to consistently demonstrate improvements across the studies; the small magnitude of the observed change; and the short-term duration of follow-up. CDEC noted the following for these issues:

1. As noted in CADTH’s clinical review report, there was inconsistency in the results for changes in BMI, with statistical significance being demonstrated in only one of the trials (TRANSPORT).
2. Although the pre-planned pooled analysis suggested that treatment with LUM/IVA was associated with improvements in BMI, the magnitude of improvement was of uncertain clinical significance. Given patients in TRAFFIC and TRANSPORT were at least 12 years of age, a change in BMI at 24 weeks from baseline of less than one pound/m² is of uncertain clinical significance.
3. Canadian clinical experts consulted by CADTH and CDEC noted that longer-term trials are required to demonstrate if treatment with LUM/IVA would result in clinically meaningful improvements in BMI for Canadian patients compared with current clinical practice.

As noted in CADTH’s clinical review report, there is no established MCID for LCI_2.5 and this end point is not used in Canadian clinical practice. Hence, it is not possible to conclude whether or not the results are clinically relevant, particularly in patients who have relatively normal lung function.

Could CDEC provide commentary regarding the type of clinical outcomes that would be required to address the uncertainty in the study results that was noted in the embargoed recommendation document (e.g., in a list of research gaps that could be addressed through future studies including real-world effectiveness data)?

CDEC Response

Data regarding the following end points would be beneficial in order to accurately assess the clinical benefit of LUM/IVA:

• Controlled studies evaluating adjudicated pulmonary exacerbations over at least one year of treatment.
• Controlled studies in patients with rapidly progressive disease, in those who experience a greater number of pulmonary exacerbations, in patients with severely impaired pulmonary function (FEV1 < 40%).
• Controlled studies evaluating slope of decline in lung function.
• Long-term end points such as time to lung transplantation and survival.

CDEC noted that some of these outcomes could be addressed through the collection of real-world effectiveness data in order to account for the long duration of follow-up and low event rates.

Could CDEC clarify what information would be required to develop clinical criteria that would facilitate the identification of patients who are at risk for rapid deterioration (i.e., the subgroup of patients identified by clinical specialists in whom there may be a place in therapy for LUM/IVA)?

CDEC Response

As noted in CADTH’s report and by the clinical experts, the controlled studies were limited to patients with stable disease; hence, these studies cannot be used to evaluate the effect of LUM/IVA on patients who are at risk for rapid deterioration. CDEC concluded that the available clinical and pharmacoeconomic evidence submitted by the manufacturer is insufficient to support reimbursement recommendations that include starting and/or stopping criteria for patients at risk of rapid deterioration.

For younger patients six to 11 years old, Orkambi was associated with a statistically significant improvement in lung function (based on LCI_2.5) after 24 weeks of treatment, but CDEC noted that the clinical relevance of this result could not be determined because the threshold for determining clinical importance of the results for the test that was used has not been established. Can CDEC clarify what information would be required to determine the clinical significance of this result?

CDEC Response

As noted in CADTH’s report and by the clinical experts, there is no established MCID for the LCI_2.5 and this end point is not used in Canadian clinical practice. CDEC noted that the following information would be necessary to determine whether LCI_2.5 is a clinically meaningful end point:

• Demonstration that LCI2.5 is a valid surrogate end point. Specifically, such studies should show that there is a strong, independent, consistent association between LCI2.5 and clinical end points in CF patients. Also, that there is evidence that improvement in LCI2.5 has consistently led to improvement in clinical end points.
• Establishment of the magnitude of improvement in LCI2.5 that is associated with improvement in clinical end points (i.e., the MCID).
• Standardized methods for LCI2.5 measurement and analysis.

Could CDEC please comment on whether or not the committee feels that the pivotal studies included in CADTH’s review were well-designed and well-conducted?

CDEC Response

CDEC concurs with CADTH and the clinical experts, in that the clinical studies submitted by the manufacturer were generally well-designed and well-conducted. However, CDEC has noted that the clinical studies must be interpreted in accordance with the statistical testing hierarchy that was pre-planned by the manufacturer. CDEC does not agree with the manufacturer’s decision to calculate and report P values for secondary end points after failure of the statistical testing hierarchy.

CDEC also noted that the design of the clinical trials left some important gaps in the evidence that should be addressed through further studies. These are documented above, in CDEC’s response to question six from the jurisdictions.