WHAT CLINICAL BENEFIT DO YOU WANT TO PROVE IN YOUR STUDY?

Defining the clinical benefit or the efficacy parameters for your product in a clinical study is a challenging but critical activity: Do I aim to show the effect of product itself or that of a treatment as a whole, what study parameter/ endpoint best reflects the benefit of my product, and should I assess the effect in a controlled clinical trial environment or in a real world setting? Especially for medical devices in Europe where upon market entry only clinical evidence is needed supporting product safety and performance, a whole range of options and therefore debate is possible and necessary in a postmarket setting. The fact that different stakeholders may have different opinions complicates the discussion even further, physicians told me several times that

”I check my patient to see and hear what (s)he feels and thinks, and that, in combination with test results, is what drives my diagnosis and treatment, not whether some number dropped or rose to a certain level”.

Such being the case for babies with severe eczema not being able to sleep, as well as for patients with end stage cancer, where quality of life generally prevails over other parameters. The problem with parameters such as quality of life, however, being that it can lead to large and long-term clinical studies unless using less conventional methodologies. In addition regulatory bodies or insurance companies may well have an interest in a different more clear cut study endpoints. In this second post in the clinical evidence dilemmas series, you find below some thoughts on 3 aspects that to my opinion need careful consideration when designing your medical device study:

Product effect

A clinical study is generally set-up to show efficacy of the product itself, excluding as much as possible other medical treatments or interventions that may affect the same study endpoints. This, indeed, is the most logical choice when aiming to show product efficacy for regulatory purposes, but this may well differ from how the product is or will be used in daily practise. In the real world, concomitant therapies may interfere and reduce the effect of the product, or in some cases treatments may even strengthen each others effect. Electroporation for example has been shown to destroy cancer cells, but also to enhance local chemotherapy. So here the question is whether in your clinical trial you want to show the effect of a product, or a treatment that includes that product? And again keep in mind the answer may depend on  the stakeholder: a physician aiming to treat his patient the best he can will see this different than regulatory body that wants to know the pure benefit of the product itself.

Patient outcome

So what about the patient? As the above quote already illustrates, daily practice is and should revolve around the patient rather than a medical product, and therefore prefers to see patient reported outcomes/ endpoints such as quality of life. Clinical trials, however, typically are designed to show a maximum effect at the smallest possible sample size and therefore study endpoints in medical device studies are often surrogate ones as these are better to measure. While this increases feasibility of a clinical trial, and therefore also addresses an ethical dilemma, the real world tends to ask for other endpoints than those addressed in a clinical trial.

Another challenge in this respect is that some medical products, such as diagnostic devices, do not treat the patients by itself and a potential effect on patient outcome(s) will only be achieved indirectly. In other words via the professional users who, based on the information provided by the device in combination with other information, decide to apply a certain therapeutic intervention. Such intervention may in turn lead to a certain outcome, and therefore it may be hard to show a product effect on patient outcome in a clinical study.

So what parameter/ endpoint to measure the effect of your product by, and in what setting, is not an easy question to answer. When, however, not choosing patient outcome as a primary study endpoint, at least include one or more as secondary.

Training

For many medical devices, adequate training on its use is key to a successful outcome. Whether it concerns a highly sophisticated surgical instrument like da Vinci robot or a diagnostic tool like a blood pressure monitor, correct handling of the device can have a strong impact on the product safety and efficacy. In clinical trials participating sites and users are well trained for patients safety as well as products outcome sake. With wide spread use, however, the device will be operated by less well trained experts for several reasons, thereby reducing efficacy and sometimes increasing risks for the patients involved. The question is whether and how to include this learning curve (which may be substantial for some surgical devices) in your clinical trial? This would make the outcome more representative for a real-world setting, but also increases the size of your study.

Conclusion

As will be clear from the above, unfortunately, there is not just one answer to the question(s) raised at the beginning of this post. A lot depends on the type of device and what you exactly want to prove and when in the product life cycle. Often, however, it is not a question of either/or, but more a question of how much you can afford to invest in your clinical trial to show product benefit.

Please feel free to contact me to discuss the above, or any other topic at: annet.muetstege@appliedclinicalservices.com.