Resource Library

The EMA adopted the E9(R1) addendum on estimands and sensitivity analysis in clinical trials two years ago. Understanding the essense of estimands and the major aspects covered in E9(R1) is crucial for grasping their effect on clinical study design, statistical analyses and study outcomes. We interviewed Kristina Bondareva, Head of Biostatistics at OCT Clinical, top Eastern European clinical research organization, to shed some light on the role of biostatistics in general, and comment on the EMA’s adoption of the E9 addendum.   

— Kristina, the original ICH guideline, E9  “The Statistical Considerations for Clinical Trials” was published in 1998. And in 2020 it was amended for the first time in more than 20 years. What changes have been introduced by the addendum on estimands and sensitivity analysis?  

— Yes, it is big news that E9 has finally been amended. The addendum focuses on quantifying the effects of treatment. If all patients remained in the study and on their randomized treatment and did not take rescue medications or prohibited therapy, the task would be straightforward. However, in practice, these events, which affect the interpretation of the data collected or make them irrelevant for the evaluation of the treatment regimen we are interested in, do happen (both in clinical trials and in real life). The way that we handle these events during the analysis affects how the research question is answered. Instead, it should be the exact opposite, as the research question should define the way we analyze data and account for intercurrent events. Traditionally, we had somewhat vaguely formulated objectives in clinical study protocols, such as “To assess the efficacy and safety of test treatment compared to control”, and a list of endpoints and important decisions on how to account for these intercurrent events were made in terms of statistical analysis. At best they were only described in the statistical section of the protocol, under ‘Handling of missing data’ or elsewhere.   

In many cases it led to difficulties in interpretation and communication of study results and even disagreements between the sponsor and regulator, who could have different perspectives on what the trial was intended to estimate or what treatment effects are the most relevant for regulatory approval.  

The goal of the addendum is to reverse this practice. The research question should be formulated clearly and unambiguously in the clinical study protocol through specification of estimands. The addendum provides the framework, a common language which can be used and understood by different stakeholders.  

— Another aspect discussed in the addendum is the sensitivity analysis. How is it related to estimands?  

— Yes, the guideline also provides clarifications on conducting sensitivity analyses. The current practice of providing sensitivity analysis was imperfect and somewhat disordered. The expectations of regulatory agencies regarding how many sensitivity analyses should be carried out and what aspects they should cover were not clear either. In many cases it led to a situation where a lot of analyses were conducted, which again handled the intercurrent events differently and therefore answered different research questions. Ideally, the conclusion remains unchanged, but it is logical to assume that when the rate of intercurrent events is high or unequal between treatment groups, they are more likely to have a greater effect on the estimates and even change the conclusion.  

So, how do we interpret the results in this case?  

The addendum clarifies that sensitivity analyses should target the same estimand and answer the same research question and vary in the assumptions of statistical models. They should concentrate on untestable assumptions. Methods such as delta adjustment and tipping point analysis are useful in investigating these kinds of assumptions. Supportive analyses are all other analyses carried out in order to better understand data and treatment effects, and they are usually given lower priority in regulatory assessments.  

— How has the new estimand framework been accepted by the clinical trials community?  

— It was not love at first sight, I suppose. Some saw it as an unnecessary complication of an already complex process of designing clinical trials, while others claimed that there was nothing new or different about the approach. I liked the original title of one article “A narrative review of estimands in drug development and regulatory evaluation: old wine in new barrels?”.  

Overall, there was a lot of skepticism at first but, I believe, the more that articles, other material sand conferences broached the topic with clarifications and examples, the more that people started to see this more as an opportunity rather than a threat. And later, the framework further proved itself useful in discussing clinical trials affected by Covid-19.  

— Speaking about the impact of the Covid-19 pandemic, how does the estimand framework help address that?  

— The pandemic has caused unprecedented damage to healthcare systems, and clinical trials are no exception. There can be delays in drug supply, resulting in treatment interruptions, and patients may become infected and receive experimental treatment or even die of Covid-19. It is very natural to talk about the impact of the pandemic and other infectious disease research in terms of intercurrent events. In most cases it is also logical to consider pandemic-related intercurrent events and non-pandemic-related intercurrent events separately. In other words, treatment discontinuation due to Covid-19 or due to other reasons. The general idea is that the original objectives of a trial would remain the same.   

That would mean that we are interested in treatment effects unconfounded by the pandemic situation and related disruptions. Therefore, hypothetical strategy would be the most natural choice. It means that the data collected after the intercurrent event related to Covid-19 would be excluded from the analysis, and the outcomes would need to be modeled from the rest of the data. In this case the results would need to be accompanied by extensive sensitivity analysis. An additional consideration is that some less important intercurrent events due to Covid-19 could still be handled using a treatment policy. This would be a more balanced approach since it yields more data to model the outcomes.  

— So, how is this process of choosing the right estimand supposed to work in practice? Who would be responsible for it? And what are the main challenges associated with it?  

— The idea is that it should be a collaborative effort by both clinicians and statisticians withing a clinical research project. Ideally, the clinicians should formulate the research question, identify potential intercurrent events, and discuss the most clinically meaningful strategies to account for these events, while the statisticians should highlight when an estimand is difficult or impossible to estimate. Note that, despite this issue, the estimand framework is laid out in the addendum to statistical guidelines. And for a long time, the concept has only been familiar in the statistician community. If estimands continue to only be statistical issues, the positive impact of introducing them will be diminished. In order to make it work right, we need to ensure awareness and effective communication between different members of the study team. The role of biostatisticians would be to guide this discussion. You know, an ounce of prevention is worth a pound of cure. It is much better to have these discussions at the clinical study designt stage when it is still possible to align the conduct of the clinical trial with the research question and plan appropriate analyses. Eventually, these questions will arise anyway, maybe at the stage of CSR writing or communicating study results, so why not consider them earlier?  

— How have you been implementing the new guidelines at OCT Clinical?  

— We are planning a series of events. We have already conducted a webinar and now we are providing training and workshops for different staff within the company, including in the Biostatistics and Clinical Departments.