Providing data from randomized controlled trials has long been considered the gold standard to demonstrate the safety and efficacy of a regulated medical product. However, for many medical devices, practical limitations related to the device or disease condition require alternative approaches and increased flexibility in trial design and statistical analysis.
Advances in the availability of Real-World Data (RWD) sources (such as electronic health records, registries, medical claims, pharmacy data and feedback from wearables and mobile technology), have increased the potential to generate robust Real-World Evidence (RWE) that can be used to support regulatory decisions.
In this topical article, our Regulatory and Clinical teams explore how RWE can be successfully leveraged in regulatory submissions for medical devices intended for the US and EU markets…
What is Real-World Evidence?
Real-World Evidence (RWE) is the clinical evidence regarding the usage, benefits, and risks of a medical product derived from the analysis of Real-World Data (RWD). The FDA defines Real-World Data (RWD) as:
“…data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources.”
Under the right conditions, data derived from real-world sources can be used to support regulatory decisions. RWD and associated RWE may constitute valid scientific evidence depending on the characteristics of the data.
The real-life clinical performance of a medical product may even be more clearly demonstrated through RWD/RWE than via a controlled clinical trial, because the latter often cannot evaluate all applications of a product in clinical practice across the full range of potential users.
Utilising RWE / RWD in FDA submissions
To determine the suitability of RWD for regulatory decision-making, the FDA assesses the relevance and reliability of the data source and its specific elements. This assessment is then used to determine whether the RWD source / sources and the proposed analysis can generate evidence that’s sufficiently robust for a regulatory purpose.
Whether the evidence is sufficiently relevant and reliable for use will, in part, depend on the level of quality necessary to make a particular regulatory decision. The FDA evaluates the same factors to assess RWD across all data sources and regulatory decisions.
In cases where your RWE is derived from multiple RWD sources, the FDA evaluates each RWD source both individually and in combination to determine the data’s relevance and reliability.
If you intend on developing a new RWD source, we’d recommend a consultation with the FDA to ensure that relevance and reliability are addressed in the initial design.
Determining the relevance of your Real-World Data
The FDA will assess the following important relevance factors to determine if your RWD are suitable for regulatory use:
- Whether your RWD contain sufficient detail to capture the use of the device, exposures, and the outcomes of interest in the appropriate population (i.e., that the data apply to the question at hand)
- Whether the data elements available for analysis can address the specified question when valid and appropriate analytical methods are applied (i.e., that the data are amenable to sound clinical and statistical analysis)
- Whether the RWD and RWE you provide are interpretable using informed clinical/scientific judgment
You can find a complete list of the relevance factors here.
Assessing the reliability of your Real-World Data
The primary factors FDA considers for assessing the reliability of RWD include data accrual (how the data were collected), and data assurance (whether the people and processes in place during the data collection and analysis can provide adequate assurance of error minimisation, and that the quality and integrity of the data is sufficient).
Additionally, you should prospectively define your RWD analysis protocol as described in Section III of Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices.
To ensure the reliability of your RWD, your RWD source should have an operational manual or other documentation that pre-specifies:
- The data elements to be collected
- Data element definitions via a data dictionary
- Methods for data aggregation and documentation such as a common case report form or an abstract from a verifiable source
- A common temporal framework for data element collection
Some RWD sources such as electronic health records (EHRs) or claims data may not fulfil all of these characteristics but may still demonstrate sufficient reliability to support your regulatory submission.
Data assurance – Quality Control
Data quality control is essential for instilling confidence in the reliability of RWD and RWE sources. RWD quality can generally be improved by following published recommendations concerning registries, such as those by the Agency for Healthcare Research & Quality, the Patient-Centered Outcomes Research Institute, the National Medical Device Registry Task Force, and the IMDRF Registry Working Group.
However, certain sources of RWD, such as some administrative and healthcare claims databases or EHRs, may not have established data quality control processes and as a result may not be capable of entirely fulfilling the recommendations.
As such, it’s important to consider any methods and systems used to generate your RWD to help ensure sufficient data quality. You should evaluate potential RWD sources in accordance with a data QA plan and develop procedures for the data source itself.
RWE / RWD under the EU MDR
When it comes to placing medical devices on the EU market, the EU MDR ensures the clinical performance and safety of medical devices throughout their lifetime by placing emphasis on appropriate Postmarket Clinical Follow up (PMCF) and postmarket studies.
The EU MDR details the procedural requirements of the PMCF process to be performed by manufacturers including PMCF plans and reports (depending on the device class). Processes must be effective and appropriate for the assessment of the collected data.
Then regarding the clinical data itself, Annex XIV, Part B of the MDR describes the aim for its collection and evaluation as;
“…confirming the safety and performance of the device, identifying unknown side effects, monitoring side effects, contraindications and risks, risk-benefit ratios, identifying misuse and off-label use.”
According to the EU MDR, the data should be analysed and reported, however the regulation does not specify expectations regarding the characteristics of the clinical data or post-market surveillance data itself and does not specifically mention “Real-World Data” or “Real-World Evidence” as potential sources of clinical data.
How does the supporting guidance available handle RWD/RWE?
MDCG guidance 2020-7 cites Real-World Data analyses as one of the activities to be considered by manufacturers when establishing a Post Market Clinical Follow-Up (PMCF) System under EU MDR. It points out that to ensure meaningful results, the manufacturer should be mindful of the quality of the Real-World Data and the reliability of the data sources.
MEDDEV 2.12/2 rev 2 provides further detail. Whilst this guidance predates EU MDR and has not been updated based on EU MDR requirements, it does however describe the design and methodologies that should be considered, as well as how obtained study data should be analysed and utilised to provide clinical evidence for medical devices on the market.
Even though the guidance does not use the terms “Real-World Data“ or “Real-World Evidence“, it does emphasise the importance of gathering real-world postmarket clinical data to ensure the long-term safety and performance of devices after they’ve been placed onto the market, and to ensure the acceptability of residual risks of devices in a real-world clinical setting.
Furthermore, MEDDEV 2.1/2 rev 2’s focus on studying problems or complications after wide-spread and long term use of devices, and the verification of safety and performance in a larger and more varied population of clinical users, arguably points toward the use of Real-World Data. Indeed, one of the PMCF study methods proposed in the guidance is the review of relevant retrospective data from patients previously exposed to the device, which could be interpreted as the use of a broader real-world data set, in real-world clinical situations, with a broad set of patients and clinicians as opposed to a narrow set of study participants.
What the future holds for leveraging RWE
From a US perspective, the FDA’s trust in RWE to support regulatory decisions has helped to fast-track patient access to innovative products that advance public health. The FDA is continuing to improve the opportunities for leveraging RWE in submissions for many types of devices. Leveraging RWE has provided CDRH with more timely access to a broader and richer set of safety information to better protect the public.
And in Europe, whilst MDR may not explicitly mention “Real-World Data” or “Real-World Evidence”, MDCG states the importance of high quality RWD and reliable data sources, and content included in MEDDEV 2.12/2 rev 2 can certainly be interpreted as nodding towards its utilisation.
Despite the varying ways in which RWE is interpreted by regulatory bodies in the US and EU, there is no doubt that by unleashing the power of RWE, medical device development can be accelerated – delivering innovation and advances faster and more efficiently to those in need, whilst maintaining patient safety.
Should you have a challenge relating to RWD / RWE, our Regulatory and Clinical teams are ready and happy to help. Simply get in touch to start the conversation.