Evaluating Complex Interventions: Navigating the Landscape of Health Technology Assessment

Domains of Value and Evaluation Criteria in HTA

Health Technology Assessments (HTAs) of complex interventions grapple with a multifaceted challenge: how to measure value. Our review of 35 publications revealed two distinct categories of evaluation domains and criteria. One stems from methodological guides and theoretical frameworks, proposing a balanced assessment across various domains. The other reflects real-world HTA practice, often prioritizing economic evaluation, clinical effectiveness, and organizational impact.

This prioritization, while understandable given the tangible nature of economic and clinical data, risks overlooking crucial social, ethical, and legal considerations. As interventions become increasingly complex, so too must our evaluation methods. Balancing the pragmatic needs of funding decisions with the comprehensive understanding required for successful implementation remains a key challenge.

Methodological guides like Integrate-HTA champion a balanced approach, emphasizing context, stakeholder engagement, and patient preferences alongside the core domains of safety, effectiveness, and cost-effectiveness. However, translating this ideal into practice proves difficult, especially in resource-constrained environments.

Logic models offer a potential tool for navigating complexity, mapping contextual factors and causal pathways. However, criticisms regarding their theoretical nature and potential to oversimplify real-world complexities highlight the need for continuous refinement and integration of diverse stakeholder perspectives.

Approaches to HTA of Complex Interventions

Our review identified three distinct approaches to HTA of complex interventions: integrative, concurrent, and sequential.

The Integrative Approach

Integrative HTA aims for a holistic, interdisciplinary evaluation, recognizing the interconnectedness of a complex intervention's components. Integrate-HTA exemplifies this approach, offering a comprehensive five-step framework. However, its complexity and resource intensity pose challenges to practical application, highlighting the gap between theory and practice.

Bridging this gap requires ongoing collaboration between researchers, practitioners, and decision-makers to ensure HTA remains relevant and actionable.

The Concurrent Approach

Common in Australia and Canada, the concurrent approach evaluates different aspects of the intervention simultaneously, often using specialized teams. While this can expedite the HTA process, it necessitates robust synthesis and interpretation of diverse evidence streams. Without careful integration, the interplay and emergent effects of complex interventions may be missed.

Furthermore, stakeholder engagement in concurrent approaches often occurs later in the process, potentially limiting their influence on evaluation outcomes.

The Sequential Approach

Predominant in Ireland, the sequential approach takes a step-by-step evaluation process, thoroughly examining each dimension before moving to the next. While proponents argue for increased rigor, the extended timelines may hinder the holistic understanding of complex interventions' dynamic nature.

Complexity Thinking and the Future of HTA

The divergence between theoretical ideals and practical realities underscores the ongoing methodological challenge in complex intervention HTA. Balancing comprehensive evaluation with timely, actionable insights necessitates an agile framework that blends theoretical robustness with pragmatic feasibility.

Complexity thinking offers a promising paradigm shift, encouraging a holistic, systems-level understanding of interventions. By embracing this approach, HTA can move beyond linear cause-and-effect analyses to capture the intricate web of contextual factors, stakeholder interactions, and emergent properties that shape outcomes.

While operationalizing complexity thinking in HTA remains a challenge, approaches like realist evaluation and established HTA processes offer practical guidance. Further research is crucial to refine these methods and ensure their effective integration into future evaluations.

Limitations of the Review

Our scoping review, while potentially the first of its kind, has limitations. The focus on English-language literature, the inclusion of studies examining single factors, and potential biases in narrative synthesis may limit the generalizability of findings. Furthermore, the specific terminology used in the search strategy may not have captured all relevant studies.

Future research should employ broader search strategies, explore alternative terminologies, and strive for a universally accepted definition of "complex intervention" to enhance the accuracy and inclusivity of HTA evaluations.