Clinical evaluation is an ongoing process that’s conducted throughout the life cycle of a medical device. It is a structured, transparent, iterative, and continuous process that forms part of a device’s quality management system.
Whilst Software as a Medical Device (SaMD) is subject to the same general clinical evaluation principles as other medical devices, certain key additions apply. In our topical article below, our CEO Jörg Dogwiler takes a look at the requirements and considerations regarding clinical evaluation for SaMD in the EU and the US.
When it comes to undertaking a clinical evaluation for SaMD, manufacturers must comply with the same legal requirements as for the clinical evaluation of all other types of medical device.
As with other medical devices, SaMD manufacturers specify an intended medical purpose which has a clinical benefit, and therefore, their SaMD requires clinical evidence within its own conformity assessment.
Performing an effective clinical evaluation normally requires clinical data from the device itself. Manufacturers can obtain this clinical data through a clinical investigation, or they can utilise existing data from a proven equivalent product. For SaMD however, this evaluation approach is often not appropriate.
Whether you are producing SaMD for the EU or the US markets, the following three major components should be considered:
The general requirements for clinical evaluation of medical devices are outlined in Article 61 of the MDR 2017/745 (including Annex XIV).
These methodological principles are depicted in the diagram below:
The MDCG document MDCG 2020-1: Guidance on Clinical Evaluation (MDR)/Performance Evaluation (IVDR) of Medical Device Software outlines a possible alternative path for performing the clinical evaluation for SaMD.
According to the MDCG, three key elements should be considered when compiling clinical evidence for SaMD:
Valid clinical association is understood as the extent to which the SaMD’s output (e.g., concept, conclusion, calculations), based on the inputs and algorithms selected, is associated with the targeted physiological state or clinical condition. This association should be well-founded or clinically accepted. The valid clinical association of SaMD should demonstrate that it corresponds to the clinical situation, condition, indication or parameter defined in the stated intended purpose.
Evidence supporting valid clinical association can be generated through literature research, professional guidelines, proof of concept studies, or a manufacturer’s own clinical investigation studies.
Technical performance is validated by the demonstration of the SaMD’s ability to generate the intended output accurately, reliably, and precisely, from the input data.
Evidence supporting technical performance can be generated:
Clinical performance is validated by the demonstration of the SaMD’s ability to yield clinically relevant output in line with the stated intended purpose. The clinical relevance of the SaMD’s output is a positive impact:
Evidence supporting clinical performance can be generated by testing the SaMD under evaluation, or an equivalent device, with the target population for the intended use. The applied methodology should be appropriate for the device characteristics and intended purpose, and may include pre-clinical testing, a clinical investigation, or a clinical performance study.
Whilst valid clinical association, technical performance, and clinical performance portray a methodological principle for the generation of clinical evidence, they do not represent a distinct stepwise approach.
To determine and justify the level of clinical evidence, both the amount and quality of supporting data should be evaluated. This assessment can be guided by the following non-exhaustive list of questions:
In some cases, particularly for lower risk class I and IIa medical device software (SaMD), clinical data is often not adequate to demonstrate compliance. But both the MDR (Article 61, Section 10) and MDCG document 2020-1 allow an exception. In such cases, demonstration of compliance with general safety and performance requirements can be based solely on the results of non-clinical test methods including performance evaluation, technical testing (“bench testing”) and pre-clinical evaluation. However, this must be justified, covered in your risk management, and considered in line with your device’s intended clinical performance.
Take as an example, SaMD intended for image segmentation. Say a manufacturer develops an independent SaMD intended to allow automatic detection of organs and anatomical structures (such as the aorta) in CT scans with the accuracy of a radiologist.
In this example, the manufacturer claims that the SaMD:
The following depicts how the three key elements we covered earlier might be realised:
Method | Literature is reviewed to establish valid clinical association
Evidence | The normal shape and size of anatomy is well established, and the segmentation techniques on cross-sectional images correlates well with the actual size and shape
Result | The valid clinical association is established without any gaps being identified
Method | Verification and validation tests
Evidence | The basic technical performance such as display, modification, window levelling of images, measurements including confirmation of accuracy, sensitivity, and reliability of the SaMD are as per the expected performance
Result | The technical performance meets the expected performance
The undertaking of a usability assessment with the intended user groups, in conjunction with the valid clinical association and validation of technical performance results, is determined as sufficient to demonstrate conformity with relevant GSPRs.
In cases where data is available, a retrospective analysis could be performed. In cases where data does not represent the variability of input parameters, the missing data could be generated in a prospective clinical investigation to establish the clinical performance of the segmentation algorithm.
The safety, effectiveness, and performance of your SaMD should be actively and continuously monitored. Such data may include (but is not limited to) post-market information such as complaints, PMCF data, real-world performance data, direct end-user feedback or newly published research / guidelines. The data should be subject to the Clinical Evaluation principles depicted in our first diagram.
The unique level of connectivity of SaMD facilitates access to real-world performance data which can be used for multiple purposes, including, but not limited to:
SaMD can be released for CE marking with initially claimed and validated clinical benefits. Monitoring of real-world performance data can help to formulate hypotheses about future SaMD functionalities and intended uses.
Together with the IMDRF, the FDA has published a guidance document that is very similar to MDCG 2020-1-1. Software as a Medical Device: Clinical Evaluation describes a converged approach for planning the process for clinical evaluation of SaMD to establish:
The principle is the same as that proposed by MDCG for the EU. Manufacturers are required to verify the clinical validity of their product with the clinical evaluation of medical device software following the known steps as outlined in our EU section above. In doing so, as the manufacturer, you will effectively demonstrate that the intended purpose is fulfilled and that there are no unacceptable risks.
Should you have a Clinical or eHealth related challenge, our team is ready and happy to help. Simply get in touch to start the conversation.