Pragmatic Free Trial Meta
Pragmatic Free Trial Meta is a non-commercial open data platform and infrastructure that facilitates research on pragmatic trials. It collects and distributes cleaned trial data, ratings, and evaluations using PRECIS-2. This permits a variety of meta-epidemiological studies to evaluate the effects of treatment across trials of different levels of pragmatism.
Background
Pragmatic trials provide evidence from the real world that can be used to make clinical decisions. However, the use of the term "pragmatic" is not uniform and its definition and assessment requires further clarification. Pragmatic trials must be designed to inform policy and clinical practice decisions, not to confirm an hypothesis that is based on a clinical or physiological basis. A pragmatic trial should also aim to be as similar to real-world clinical practice as possible, such as the recruitment of participants, setting up and design as well as the implementation of the intervention, and the determination and analysis of outcomes as well as primary analysis. This is a major distinction between explanatory trials as defined by Schwartz and Lellouch1 that are designed to prove a hypothesis in a more thorough manner.
Trials that are truly practical should avoid attempting to blind participants or clinicians in order to lead to distortions in estimates of the effects of treatment. The trials that are pragmatic should also try to enroll patients from a variety of health care settings so that their results can be compared to the real world.
Furthermore, 프라그마틱슬롯 that are pragmatic must be focused on outcomes that matter to patients, like quality of life and functional recovery. This is particularly relevant when trials involve the use of invasive procedures or could have dangerous adverse consequences. The CRASH trial29, for instance, focused on functional outcomes to evaluate a two-page case report with an electronic system to monitor the health of patients admitted to hospitals with chronic heart failure. Similarly, the catheter trial28 utilized urinary tract infections that are symptomatic of catheters as its primary outcome.
In addition to these features, pragmatic trials should minimize the procedures for conducting trials and requirements for data collection to reduce costs and time commitments. Additionally, pragmatic trials should aim to make their results as relevant to actual clinical practice as is possible. This can be accomplished by ensuring that their primary analysis is based on an intention-to treat method (as defined in CONSORT extensions).
Many RCTs that don't meet the requirements for pragmatism but have features that are contrary to pragmatism have been published in journals of varying types and incorrectly labeled as pragmatic. This could lead to misleading claims of pragmatism and the use of the term should be standardized. The development of the PRECIS-2 tool, which provides an objective standard for assessing pragmatic features is a great first step.
Methods
In a pragmatic trial, the aim is to inform clinical or policy decisions by demonstrating how an intervention would be implemented into routine care. This is different from explanatory trials that test hypotheses about the causal-effect relationship in idealized settings. In this way, pragmatic trials can have less internal validity than explanatory studies and be more prone to biases in their design analysis, conduct, and design. Despite their limitations, pragmatic research can provide valuable information to make decisions in the context of healthcare.
The PRECIS-2 tool assesses the degree of pragmatism in an RCT by assessing it across 9 domains, ranging from 1 (very explicit) to 5 (very pragmatic). In this study, the recruitment, organisation, flexibility: delivery, flexible adherence and follow-up domains were awarded high scores, however the primary outcome and the method of missing data fell below the pragmatic limit. This suggests that it is possible to design a trial with good pragmatic features without damaging the quality of its outcomes.
However, it's difficult to determine how pragmatic a particular trial is, since pragmatism is not a binary quality; certain aspects of a study can be more pragmatic than others. A trial's pragmatism can be affected by modifications to the protocol or the logistics during the trial. Additionally 36% of the 89 pragmatic trials identified by Koppenaal et al were placebo-controlled or conducted before approval and a majority of them were single-center. They aren't in line with the usual practice and can only be considered pragmatic if the sponsors agree that these trials are not blinded.
Furthermore, a common feature of pragmatic trials is that researchers try to make their results more valuable by studying subgroups of the trial. This can lead to unbalanced results and lower statistical power, thereby increasing the chance of not or incorrectly detecting differences in the primary outcome. This was a problem during the meta-analysis of pragmatic trials as secondary outcomes were not adjusted for covariates' differences at the baseline.
In addition the pragmatic trials may have challenges with respect to the gathering and interpretation of safety data. This is due to the fact that adverse events are typically reported by participants themselves and are susceptible to reporting delays, inaccuracies, or coding variations. Therefore, it is crucial to enhance the quality of outcomes assessment in these trials, and ideally by using national registry databases instead of relying on participants to report adverse events in the trial's own database.
Results
Although the definition of pragmatism may not mean that trials must be 100 percent pragmatic, there are benefits to incorporating pragmatic components into clinical trials. These include:
By including routine patients, the trial results can be more quickly translated into clinical practice. However, pragmatic trials be a challenge. The right kind of heterogeneity for instance, can help a study generalise its findings to many different patients or settings. However the wrong kind of heterogeneity can reduce the assay sensitivity and thus lessen the power of a trial to detect even minor effects of treatment.
A variety of studies have attempted to classify pragmatic trials with various definitions and scoring systems. Schwartz and Lellouch1 developed a framework to differentiate between explanation studies that prove a physiological or clinical hypothesis, and pragmatic studies that guide the selection of appropriate therapies in real world clinical practice. The framework was comprised of nine domains evaluated on a scale of 1-5, with 1 being more explanatory while 5 being more pragmatic. The domains included recruitment of intervention, setting up, delivery of intervention, flex adherence and primary analysis.
The original PRECIS tool3 was built on the same scale and domains. Koppenaal et. al10 devised an adaptation of this assessment, dubbed the Pragmascope which was more user-friendly to use for systematic reviews. They found that pragmatic systematic reviews had higher average scores across all domains, but lower scores in the primary analysis domain.
This difference in primary analysis domains could be explained by the way most pragmatic trials analyze data. Certain explanatory trials however don't. The overall score was lower for systematic reviews that were pragmatic when the domains on organisation, flexible delivery and follow-up were merged.
It is important to remember that a pragmatic trial doesn't necessarily mean a poor quality trial, and in fact there is an increasing rate of clinical trials (as defined by MEDLINE search, however this is not specific or sensitive) which use the word "pragmatic" in their title or abstract. The use of these terms in abstracts and titles could indicate a greater understanding of the importance of pragmatism however, it is not clear if this is evident in the content of the articles.
Conclusions
In recent times, pragmatic trials are becoming more popular in research as the value of real-world evidence is becoming increasingly acknowledged. They are clinical trials that are randomized that compare real-world care alternatives instead of experimental treatments in development, they include patients which are more closely resembling those treated in routine care, they use comparators which exist in routine practice (e.g., existing drugs), and they rely on participant self-report of outcomes. This method can help overcome the limitations of observational studies that are prone to biases associated with reliance on volunteers, and the limited accessibility and coding flexibility in national registry systems.
Other advantages of pragmatic trials are the ability to utilize existing data sources, as well as a higher likelihood of detecting meaningful changes than traditional trials. However, these trials could have some limitations that limit their validity and generalizability. Participation rates in some trials could be lower than expected due to the healthy-volunteering effect, financial incentives, or competition from other research studies. Many pragmatic trials are also limited by the need to recruit participants quickly. Certain pragmatic trials lack controls to ensure that observed differences aren't due to biases in the trial.
The authors of the Pragmatic Free Trial Meta identified RCTs published up to 2022 that self-described as pragmatic. The PRECIS-2 tool was employed to evaluate the pragmatism of these trials. It covers areas like eligibility criteria and flexibility in recruitment and adherence to intervention and follow-up. They discovered that 14 of these trials scored highly or pragmatic sensible (i.e. scoring 5 or higher) in one or more of these domains and that the majority were single-center.
Studies with high pragmatism scores are likely to have broader criteria for eligibility than traditional RCTs. They also include patients from a variety of hospitals. The authors suggest that these characteristics could make pragmatic trials more effective and relevant to daily practice, but they don't necessarily mean that a trial conducted in a pragmatic manner is completely free of bias. The pragmatism characteristic is not a definite characteristic and a test that does not possess all the characteristics of an explicative study can still produce reliable and beneficial results.