literature review on project management

Agile Project Management and Project Success: A Literature Review

• Conference paper
• First Online: 27 June 2018
• Cite this conference paper

• Thomas Bergmann 17 &
• Waldemar Karwowski 17  

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 783))

Included in the following conference series:

• International Conference on Applied Human Factors and Ergonomics

8715 Accesses

35 Citations

Agile project management (APM) has recently emerged as a new and different way of managing projects. Some experts are already voicing the opinion that APM will become the project management of the 21st century. However, so far APM has not impacted project management as much as it should have. Its focus was mainly on IT projects. The literature on APM is still in its early stages, and more research needs to be done in areas other than software development. Therefore, the present paper provides a review of the existing literature in the agile project management domain. It compares traditional to modern project management, specifically APM, and discusses the influence of project complexity factors. Further, it reviews different frameworks of project success and critical success factors. Finally, it recommends APM dimensions irrespective of project types, which potentially could impact the success of a project.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime
• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Framework for Project Management in Agile Projects: A Quantitative Study

Assessment of Success Measures and Challenges of the Agile Project Management

Agile Project Management Methodology: Exploring Factors Which Foster Successful Introduction and Institutionalization of the Framework

Shahin, A., Jamshidian, M.: Critical success factors in project management: a comprehensive review. In: Proceedings of 1st International Project Management Conference, pp. 1–14 (2006)

Google Scholar  

Jackson, M.B.: Agile: a decade in. Project Management Institute (2012)

Saynisch, M.: Mastering complexity and changes in projects, economy, and society via project management second order (PM-2). Proj. Manage. J. 41 , 4–20 (2010)

Article   Google Scholar  

PMI: a guide to the project management body of knowledge (PMBOK guide). Project Management Institute, Inc., Newtown Square, PA (2012)

Hass, K.B.: The blending of traditional and agile project management. PM World Today 9 , 1–8 (2007)

Conforto, E.C., Amaral, D.C.: Evaluating an agile method for planning and controlling innovative projects. Proj. Manage. J. 41 , 73–80 (2010)

Conforto, E.C., Salum, F., Amaral, D.C., da Silva, S.L., de Almeida, L.F.M.: Can agile project management be adopted by industries other than software development? Proj. Manage. J. 45 , 21–34 (2014)

Williams, T.M.: The need for new paradigms for complex projects. Int. J. Project Manage. 17 , 269–273 (1999)

Baccarini, D.: The concept of project complexity - a review. Int. J. Project Manage. 14 , 201–204 (1996)

Mosaic-Project-Services: Project size and categorisation. White Paper, Mosaic Project Services Pty. Ltd. (n.d.)

Brown, A., Adams, J.: Measuring the effect of project management on construction outputs: a new approach. Int. J. Project Manage. 18 , 327–335 (2000)

Bryde, D.: Perceptions of the impact of project sponsorship practices on project success. Int. J. Project Manage. 26 , 800–809 (2008)

Fortune, J., White, D., Jugdev, K., Walker, D.: Looking again at current practice in project management. Int. J. Managing Proj. Bus. 4 , 553–572 (2011)

Müller, R., Turner, R.: The influence of project managers on project success criteria and project success by type of project. Eur. Manag. J. 25 , 298–309 (2007)

Atkinson, R.: Project management: cost, time and quality, two best guesses and a phenomenon, its time to accept other success criteria. Int. J. Proj. Manage. 17 , 337–342 (1999)

Lim, C.S., Mohamed, M.Z.: Criteria of project success: an exploratory re-examination. Int. J. Project Manage. 17 , 243–248 (1999)

Jugdev, K., Müller, R.: A retrospective look at our evolving understanding of project success. Proj. Manage. J. 36 , 19–31 (2005)

Lipovetsky, S., Tishler, A., Dvir, D., Shenhar, A.: The relative importance of project success dimensions, p. 97. Wiley Subscription Services, Inc. (1997)

Shenhar, A.J., Dvir, D., Levy, O., Maltz, A.C.: Project success: a multidimensional strategic concept. Long Range Plan. 34 , 699–725 (2001)

Mir, F.A., Pinnington, A.H.: Exploring the value of project management: linking project management performance and project success. Int. J. Project Manage. 32 , 202–217 (2014)

Pinto, J.K., Slevin, D.P.: Project success: definitions and measurement techniques. Proj. Manage. J. 19 , 67–72 (1988)

Schultz, R.L., Slevin, D.P., Pinto, J.K.: Strategy and tactics in a process model of project implementation, pp. 34. The Institute of Management Sciences and the Operations Research Society of America (1987)

Alexandrova, M., Ivanova, L.: Critical success factors of project management: empirical evidence from projects supported by EU programmes. In: 9th International ASECU Conference on “Systematic Economics Crisis: Current Issues and Perspectives”, Skopje, Macedonia (2012). http://www.asecu.gr/files/9th_conf_files/alexandrova-and-ivanova.pdf

Baccarini, D.: The logical framework method for defining project success. Proj. Manage. J. 30 , 25–32 (1999)

Baker, B.N., Murphy, D.C., Fisher, D.: Factors affecting project success. In: Project Management Handbook, 2nd (edn.), pp. 902–919 (2008)

Kerzner, H.R.: Project Management: A Systems Approach to Planning, Scheduling, and Controlling. Wiley (2013)

Milosevic, D., Patanakul, P.: Standardized project management may increase development projects success. Int. J. Project Manage. 23 , 181–192 (2005)

Fortune, J., White, D.: Framing of project critical success factors by a systems model. Int. J. Project Manage. 24 , 53–65 (2006)

Pinto, J.K., Slevin, D.P.: 20. Critical success factors in effective project implementation. In: Project Management Handbook, vol. 479. Wiley (1988)

Müller, R., Jugdev, K.: Critical success factors in projects: Pinto, Slevin, and Prescott-the elucidation of project success. Int. J. Manag. Proj. Bus. 5 , 757–775 (2012)

Cooke-Davies, T.: The “real” success factors on projects. Int. J. Proj. Manage. 20 , 185–190 (2002)

Stare, A.: Agile project management–a future approach to the management of projects. Dyn. Relat. Manage. J. 2 , 43–53 (2013)

Chin, G.: Agile Project Management: How to Succeed in the Face of Changing Project Requirements. AMACOM, New York (2004)

Agile-Alliance: Manifesto for agile software development (2001). http://www.agilemanifesto.org

Kidd, P.T., Karwowski, W.: Advances in Agile Manufacturing: Integrating Technology, Organization and People. IOS Press, Amsterdam, Washington, DC (1994)

Nerur, S., Mahapatra, R., Mangalaraj, G.: Challenges of migrating to agile methodologies. Commun. ACM 48 , 72–78 (2005)

Sherehiy, B., Karwowski, W., Layer, J.K.: A review of enterprise agility: concepts, frameworks, and attributes. Int. J. Ind. Ergon. 37 , 445–460 (2007)

Fernandez, D.J., Fernandez, J.D.: Agile project management-Agilism versus traditional approaches. J. Comput. Inf. Syst. 49 , 10–17 (2008)

Ceschi, M., Sillitti, A., Succi, G., Panfilis, S.D.: Project management in plan-based and agile companies. IEEE Softw. 22 , 21–27 (2005)

Augustine, S., Payne, B., Sencindiver, F., Woodcock, S.: Agile project management: steering from the edges. Commun. ACM 48 , 85–89 (2005)

Sherehiy, B., Karwowski, W.: The relationship between work organization and workforce agility in small manufacturing enterprises. Int. J. Ind. Ergon. 44 , 466–473 (2014)

Chow, T., Cao, D.-B.: A survey study of critical success factors in agile software projects. J. Syst. Softw. 81 , 961–971 (2008)

Download references

Author information

Authors and affiliations.

Department of Industrial Engineering and Management Systems, University of Central Florida, Orlando, FL, 32816-2993, USA

Thomas Bergmann & Waldemar Karwowski

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Thomas Bergmann .

Editor information

Editors and affiliations.

School of Technology and Innovations, University of Vaasa, Vaasa, Finland

Jussi Ilari Kantola

Institute of Maritime Operations, University College of Southeast Norway, Borre, Norway

Salman Nazir

Hungarian-Netherlands School of Educational Management, University of Szeged, Szeged, Hungary

Tibor Barath

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer International Publishing AG, part of Springer Nature

About this paper

Cite this paper.

Bergmann, T., Karwowski, W. (2019). Agile Project Management and Project Success: A Literature Review. In: Kantola, J.I., Nazir, S., Barath, T. (eds) Advances in Human Factors, Business Management and Society. AHFE 2018. Advances in Intelligent Systems and Computing, vol 783. Springer, Cham. https://doi.org/10.1007/978-3-319-94709-9_39

Download citation

DOI : https://doi.org/10.1007/978-3-319-94709-9_39

Published : 27 June 2018

Publisher Name : Springer, Cham

Print ISBN : 978-3-319-94708-2

Online ISBN : 978-3-319-94709-9

eBook Packages : Engineering Engineering (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

To read this content please select one of the options below:

Please note you do not have access to teaching notes, systematic literature review of project manager's leadership competencies.

Engineering, Construction and Architectural Management

ISSN : 0969-9988

Article publication date: 11 July 2020

Issue publication date: 3 February 2021

Over the last few decades, a large number of research studies have been carried out on project manager's leadership competencies. However, systematic literature reviews are still scarce in the project management literature. Therefore, the purpose of this article is to conduct a systematic literature review on project manager's leadership competencies based on published empirical research studies.

Design/methodology/approach

The authors employed a systematic literature review (SLR) methodology to synthesize research in a rigorous manner and a total of 1,780 articles were identified in the first step and a final sample of 60 research studies were synthesized.

Synthesis of the findings in this SLR on project manager's leadership competencies revealed: (a) there is a lack of categorization or ranking of leadership competencies; (b) 20 research studies (46%) were conducted with sample sizes of less than 100; (c) only a few research studies (<10%) used interview data for analysis; and (d) none of the research studies reported adoption of a triangulation method.

Research limitations/implications

This study synthesized clusters of leadership competencies and prioritized project manager's leadership competencies as “high priority”, “moderate priority” and “low priority”. We recommend a sample size between 200 and 300 to produce sophisticated results and enhance the credibility, generalizability and validity of clusters and priorities of project manager's leadership competencies through future research.

Originality/value

Future research studies are suggested to consider systematic literature review combined with face-to-face and group interview in addition to employing triangulation methods. Besides highlighting implications for practitioners, this SLR has advanced the understanding of how to conduct systematic literature reviews in a robust manner.

• Systematic literature review
• Project manager
• Leadership competencies

Acknowledgements

Our special thanks to the editor for guidance and support. We also acknowledge and appreciate the constructive comments, feedback and suggestions of anonymous reviewers for improving the quality of this SLR paper.

Ahmed, R. , Philbin, S.P. and Cheema, F.-e. (2021), "Systematic literature review of project manager's leadership competencies", Engineering, Construction and Architectural Management , Vol. 28 No. 1, pp. 1-30. https://doi.org/10.1108/ECAM-05-2019-0276

Emerald Publishing Limited

Copyright © 2020, Emerald Publishing Limited

Related articles

All feedback is valuable.

Please share your general feedback

Report an issue or find answers to frequently asked questions

Contact Customer Support

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

The project manager's leadership style as a success factor on projects: a literature review

2005, Project management journal

Related Papers

A STUDY ON LEADERSHIP INFLUENCES IN PROJECT MANAGEMENT

Sisira Rodrigo

With the passage of time in advances in the project management profession, research studies have shown that many projects fail, underlining the importance of the project manager’s role as manager. Specifically, the manager’s leadership role is of great importance in motivating people and creating an effective working environment in order for the project team to meet greater challenges in today’s global economy. In this research study, using the literature review, important people-related elements of project performance are identified, then surveys and structured, questionnaire is used to develop a priority list of these elements, which helped to analyze underlying interactions among these elements and the role of leadership in project performance. This research is implemented to identify the important characteristics that a construction project manager should possess to ensure successful completion of project. The characteristics of a project manager indicate the type of leader and leadership style that the project manager applies during the management of projects. A research questionnaire focusing of characteristic of project managers, in terms of importance for the successful management of projects was circulated. The results of the survey were compiled and analyzed. The results of the research illustrate those characteristics which are important for managing projects successfully. The characteristics further indicate by implication the project managers’ leadership styles with managing projects. The implications of this research may be in understanding the influence of the project manager characteristics. This may indicate to developers and construction companies that they should pay more attention to the characteristics of project managers’ and their skills in communication, in order to improve the outcome of projects and to benefit the construction industry as a whole. The value is that the building industry, specifically project managers, should know the influence and effect a style of leadership has on the management of projects and that these characteristics are essential for effective completion of projects. Conclusion and recommendation: Knowledge of the characteristics and leadership styles followed by construction project managers will contribute to solving problems. The project manager has to sum up the situation and then adapt a style according to that specific situation.

South African Journal of Industrial Engineering

Suzaan Pretorius

It is widely accepted that project leaders should adapt their behaviour to meet the unique leadership demands of a variety of situations. Recently, vertical, shared, and horizontal styles of leadership have gained prominence, especially in the project management literature. Several factors are believed to play a role in determining an appropriate balance between these leadership styles. This theoretical study explores the influence of project types, the stage in the project life cycle, organisational project management maturity, and the level of trust and collaboration between project team members on the appropriate balance of leadership styles in projects. This paper presents a conceptual framework of these factors, while empirical results will be reported on in the sequel to this paper.

SSRN Electronic Journal

Vittal Anantatmula

Research Journal of Recent Sciences

The aim of this article is to highlight the significance of leadership skills that are vital for project managers while managing projects effectively and efficiently. An interesting observation has been noted from literature in the field of project management that research on leadership of project managers is still limited even though calls have been made to conduct more research for more than one decade. It becomes important to understand and adopt leadership skills and competences to cope with modern challenges of project which affect on success or failure of project. The function of project managers is rapidly evolving from managing or directing to leading the projects who must possess essential knowledge, skills, and new emerging concept of leadership. The utility of project management will be more exciting, challenging and critical in coming years, while the position of project manager will also be much more diverse. Future research has been suggested in similar field.

American Journal of Trade and Policy

Fatemeh Hamidifar

This study validates a model for assessing the relationships among a project manager's leadership style, teamwork, project performance, and stakeholder satisfaction. It determines whether the effect of leadership style on project performance can be mediated by teamwork. An industry-wide survey measured a project manager's leadership style, teamwork, and project outcomes in project performance and stakeholder satisfaction. Structural Equation Modeling (SEM) approach confirmed this research model. Analyses show that a project manager who adopts transactional and transformational leadership can improve teamwork and capital facility project performance. Additionally, stakeholder satisfaction can be achieved with high levels of project success in schedule, cost, and quality performance. These findings show that teamwork can partially mediate relationships between leadership style and project performance.

Samuel Ekung

Project management, in recognising that leadership can contribute to successful project outcome has stopped short of specifically identifying leadership traits as a significant contributor to project success. This study assessed the leadership traits of construction project managers and its impact on project delivery. The objectives were to determine project team members&#39; perception of the relative importance of selected leadership traitsto construction project management; andto evaluate the level of effort of identified leadership traits in enhancingplanning works, controlling tasks and allocating resources as correlates of project outcome. It involved questionnaire survey administered to 105 project team members selected from five locations in Akwa Ibom State, Nigeria. Surveyed data was analysed using the mean item score and percentageswhile hypothesis testinginvolvedchi square. The result of the study reveals the need to improve project leadership and that the improvement wil...

awais tallha

The goal of this paper is to indicate possible ways to improve leadership qualities in Project Management. The approach to leadership is changing. The traditional concept of leadership perceived as " command and control " gradually gives way to a new, more flexible approach, which requires-on the one hand-the ability to predict (looking ahead) and-on the other-" courage " to delegate powers and the ability to motivate a team. The paper analyses two different approaches to leadership: transactional vs. transformational. The results of the studies prove that a modern leader should be a transformational rather than a transactional one. Transformational leadership is about making all project members creative, engage and full of initiative. It is project manager role as leader to integrate team members around defined goals and make them feel responsible for solving project problems. In regard to growing demand for versatility of Project Manager, a Project Manager has to continuously improve his/ her leadership skills. The author of this article, basing on her years' long experiences with running projects, formulates a recommendation concerning three ways of improving one's leadership skills. All three recommendations are practical and are aimed at helping Project Manager in engaging project team members in a joint effort to overcome difficulties, as well as looking for new challenges.

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

RELATED PAPERS

Public Policy and Administration Research

FarhatUllah Khan

Aleksandra Đurić

International Journal of Innovative Business Strategies

Alaba Adetola

Industrial Engineering Letters

abd ur rehman rehman

wan maimun wan abdullah

Rahmayanti wulansari

Project Management Journal

Mahmoud Yasin

Paul Tilley

Alexander Decker

Malaki Hussein

Operations and supply chain management

sarath herath

Ralf Müller

International Journal of Construction Management

Muath Abu Arqoub

LEADERSHIP STYLE, TEAM INTERACTION AND PROJECT MANAGEMENT SUCCESS IN INFORMATION TECHNOLOGY PROJECTS: LITERATURE REVIEW AND RESEARCH MODEL

Võ Thanh Hải

Omambia Jane

Janabi Mager

IRJET Journal

PhD Thesis: University of Pretoria

Masoodul Hassan

International Journal of Built Environment and Sustainability

Bruno Lot Tanko

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

A structural equation model on critical risk and success in public–private partnership: exploratory study.

1. Introduction

2. literature review, 2.1. risk factors, 2.2. success factors, 2.3. research gap, 3. methodology.

• Factor identification: identifying risk and success factors for PPP transportation projects;
• Data collection: collecting data through a questionnaire survey in the public and private sectors of transportation;
• Data analysis: analyzing data statistically using structural equation modeling.

3.1. Factor Identification

3.2. data collection, 3.3. data analysis, structural equation modeling (sem), 4. results and discussion, 4.1. critical risk factors (crfs), 4.2. critical success factors (csfs), 5. conclusions, data availability statement, conflicts of interest.

• Abbas, Jawad, and Mustafa Sağsan. 2019. Impact of knowledge management practices on green innovation and corporate sustainable development: A structural analysis. Journal of Cleaner Production 229: 611–20. [ Google Scholar ] [ CrossRef ]
• Abdul Nabi, Mohamad, Rayan H. Assaad, and Islam H. El-Adaway. 2024. Modeling and Understanding Dispute Causation in the US Public–Private Partnership Projects. Journal of Infrastructure Systems 1: 04023035. [ Google Scholar ] [ CrossRef ]
• Adiyanti, Nabila Puspa, and Reza Fathurrahman. 2021. Assessing critical success factors for PPP water project in Indonesia: Lessons from West Semarang. Policy & Governance Review 2: 164–81. [ Google Scholar ]
• Ahmad, Usman, Hamid Waqas, and Kashif Akram. 2021. Relationship between project success and the success factors in public–private partnership projects: A structural equation model. Cogent Business & Management 1: 1927468. [ Google Scholar ]
• Ahmed, Manik, and Michael Garvin. 2022. Review of critical success factors and key performance indicators in performance assessment of P3 transportation projects. Journal of Management in Engineering 5: 04022045. [ Google Scholar ] [ CrossRef ]
• Akhtar, Muhammad, Nadeem Ahmad Mufti, Sajjad Mubin, Muhammad Qaiser Saleem, Sadaf Zahoor, and Sanna Ullah. 2023. Identification of Various Execution Modes and Their Respective Risks for Public–Private Partnership (PPP) Infrastructure Projects. Buildings 8: 1889. [ Google Scholar ] [ CrossRef ]
• Akomea-Frimpong, Isaac, Xiaohua Jin, and Robert Osei-Kyei. 2021. A holistic review of research studies on financial risk management in public–private partnership projects. Engineering, Construction and Architectural Management 9: 2549–69. [ Google Scholar ] [ CrossRef ]
• Aladağ, Hande, and Zeynep Işik. 2018. The effect of stakeholder-associated risks in mega-engineering projects: A case study of a PPP airport project. IEEE Transactions on Engineering Management 1: 174–86. [ Google Scholar ] [ CrossRef ]
• Alaloul, Wesam Salah, and Muhammad Ali Musarat. 2020. Impact of zero energy building: Sustainability perspective. In Sustainable Sewage Sludge Management and Resource Efficiency . Edited by Başak Kiliç Taşeli. Norderstedt: BoD–Books on Demand. [ Google Scholar ]
• Alaloul, Wesam Salah, Mohd Shahir Liew, Noor Amila Wan Zawawi, Bashar S. Mohammed, Musa Adamu, and Muhammad Ali Musharat. 2020. Structural equation modelling of construction project performance based on coordination factors. Cogent Engineering 1: 1726069. [ Google Scholar ] [ CrossRef ]
• Alaloul, Wesam Salah, Muhammad Ali Musarat, Muhammad Babar Ali Rabbani, Qaiser Iqbal, Ahsen Maqsoom, and Waqas Farooq. 2021. Construction sector contribution to economic stability: Malaysian GDP distribution. Sustainability 9: 5012. [ Google Scholar ] [ CrossRef ]
• Almarri, Khalid, and Bassam Abu-Hijleh. 2017. Critical success factors for public private partnerships in the UAE construction industry-a comparative analysis between the UAE and the UK. Journal of Engineering, Project, and Production Management 7: 21–32. [ Google Scholar ] [ CrossRef ]
• Alotaibi, Saleh, Mohammed Quddus, Craig Morton, and Marianna Imprialou. 2022. Transport investment, railway accessibility and their dynamic impacts on regional economic growth. Research in Transportation Business & Management 43: 100702. [ Google Scholar ]
• Al-Saadi, Rauda, and Alaa Abdou. 2016. Factors critical for the success of public–private partnerships in UAE infrastructure projects: Experts’ perception. International Journal of Construction Management 16: 234–48. [ Google Scholar ] [ CrossRef ]
• Anago, Jude Chidiebere. 2022. How do adoption choices influence public private partnership outcomes? Lessons from Spain and Portugal transport infrastructure. International Journal of Managing Projects in Business 15: 469–93. [ Google Scholar ] [ CrossRef ]
• Babatunde, Solomon Olusola, Akintayo Opawole, and Olusegun Emmanuel Akinsiku. 2012. Critical success factors in public-private partnership (PPP) on infrastructure delivery in Nigeria. Journal of Facilities Management 10: 212–25. [ Google Scholar ] [ CrossRef ]
• Babatunde, Solomon Olusola, and Srinath Perera. 2017. Analysis of traffic revenue risk factors in BOT road projects in developing countries. Transport Policy 56: 41–49. [ Google Scholar ] [ CrossRef ]
• Babatunde, Solomon Olusola, Srinath Perera, and Onaopepo Adeniyi. 2019. Identification of critical risk factors in public-private partnership project phases in developing countries: A case of Nigeria. Benchmarking: An International Journal 26: 334–55. [ Google Scholar ] [ CrossRef ]
• Bae, So Young, and Po-Ju Chang. 2021. The effect of coronavirus disease-19 (COVID-19) risk perception on behavioural intention towards ‘untact’tourism in South Korea during the first wave of the pandemic (March 2020). Current Issues in Tourism 24: 1017–35. [ Google Scholar ] [ CrossRef ]
• Bagozzi, Richard P., and Youjae Yi. 1988. On the evaluation of structural equation models. Journal of the Academy of Marketing Science 16: 74–94. [ Google Scholar ] [ CrossRef ]
• Bentler, Peter M., and Douglas G. Bonett. 1980. Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin 88: 588. [ Google Scholar ] [ CrossRef ]
• Bentler, Peter M., and Eric J. Wu. 2005. EQS 6.1 for Windows. Structural Equations Program Manual . Mexico: Multivariate Software. [ Google Scholar ]
• Bonvillian, William B. 2021. Emerging Industrial Policy Approaches in the United States . Washington, DC: Information Technology and Innovation Foundation. [ Google Scholar ]
• Browne, Michael W., and Robert Cudeck. 1992. Alternative ways of assessing model fit. Sociological Methods & Research 21: 230–58. [ Google Scholar ]
• Burke, Richard, and Istemi Demirag. 2019. Risk management by SPV partners in toll road public private partnerships. Public Management Review 21: 711–31. [ Google Scholar ] [ CrossRef ]
• Byrne, Barbara M. 2013. Structural Equation Modeling with Mplus: Basic Concepts, Applications, and Programming . London: Routledge. [ Google Scholar ]
• Carbonara, Nunzia, Louis Gunnigan, and Roberta Pellegrino. 2012. Tendering Procedures in PPP: A Literature Review . Bari: COST Action TU1001. [ Google Scholar ]
• Carbonara, Nunzia, Nicola Costantino, Louis Gunnigan, and Roberta Pellegrino. 2015. Risk management in motorway PPP projects: Empirical-based guidelines. Transport Reviews 35: 162–82. [ Google Scholar ] [ CrossRef ]
• Carpintero, Samuel, and Ole Helby Petersen. 2016. Public–private partnerships (PPPs) in local services: Risk-sharing and private delivery of water services in Spain. Local Government Studies 42: 958–79. [ Google Scholar ] [ CrossRef ]
• Chan, Albert P., Robert Osei-Kyei, Yi Hu, and L. E. Yun. 2018. A fuzzy model for assessing the risk exposure of procuring infrastructure mega-projects through public-private partnership: The case of Hong Kong-Zhuhai-Macao Bridge. Frontiers of Engineering Management 1: 64–77. [ Google Scholar ] [ CrossRef ]
• Chan, Albert P. C., Patrick T. I. Lam, Daniel W. M. Chan, Esther Cheung, and Yongjian Ke. 2010. Critical success factors for PPPs in infrastructure developments: Chinese perspective. Journal of Construction Engineering and Management 136: 484–94. [ Google Scholar ] [ CrossRef ]
• Cheng, Zhe, Huanming Wang, Wei Xiong, Dajian Zhu, and Le Cheng. 2021. Public–private partnership as a driver of sustainable development: Toward a conceptual framework of sustainability-oriented PPP. Environment, Development and Sustainability 23: 1043–63. [ Google Scholar ] [ CrossRef ]
• Cheng, Zhe, Yongjian Ke, Zhenshan Yang, Jianming Cai, and Huanming Wang. 2020. Diversification or convergence: An international comparison of PPP policy and management between the UK, India, and China. Engineering, Construction and Architectural Management 27: 1315–35. [ Google Scholar ] [ CrossRef ]
• Cheung, Esther, Albert P. C. Chan, and Stephen Kajewski. 2012. Factors contributing to successful public private partnership projects: Comparing Hong Kong with Australia and the United Kingdom. Journal of Facilities Management 10: 45–58. [ Google Scholar ] [ CrossRef ]
• Cheung, Esther, and Albert P. C. Chan. 2011. Risk factors of public-private partnership projects in China: Comparison between the water, power, and transportation sectors. Journal of Urban Planning and Development 137: 409–15. [ Google Scholar ] [ CrossRef ]
• Chou, Jui-Sheng, and Dinar Pramudawardhani. 2015. Cross-country comparisons of key drivers, critical success factors and risk allocation for public-private partnership projects. International Journal of Project Management 33: 1136–50. [ Google Scholar ] [ CrossRef ]
• Chourasia, Avin Shekhar, Narendra N. Dalei, and Karunakar Jha. 2024. Assessment of critical success factors for PPP airports using the AHP. International Journal of Critical Infrastructures 20: 241–60. [ Google Scholar ] [ CrossRef ]
• Civelek, Mustafa Emre. 2018. Essentials of Structural Equation Modeling . Lincoln: Zea E-Books. [ Google Scholar ]
• Cotula, Lorenzo. 2019. Reconsidering sovereignty, ownership and consent in natural resource contracts: From concepts to practice. In European Yearbook of International Economic Law 2018 . Edited by Marc Bungenberg, Markus Krajewski, Christian J. Tams, Jörg Philipp Terhechte and Andreas R. Ziegler. Berlin: Springer, pp. 143–74. [ Google Scholar ]
• Cruz, Carlos Oliveira, and Joaquim Miranda Sarmento. 2021. The impact of COVID-19 on highway traffic and management: The case study of an operator perspective. Sustainability 13: 5320. [ Google Scholar ] [ CrossRef ]
• Cruz, Carlos Oliveira, and Rui Cunha Marques. 2013. Flexible contracts to cope with uncertainty in public–private partnerships. International Journal of Project Management 31: 473–83. [ Google Scholar ] [ CrossRef ]
• Darko, Deborah, Demi Zhu, Matthew Quayson, Md Altab Hossin, Osayuwamen Omoruyi, and Albert Kweku Bediako. 2023. A multicriteria decision framework for governance of PPP projects towards sustainable development. Socio-Economic Planning Sciences 87: 101580. [ Google Scholar ] [ CrossRef ]
• de Carvalho, Marly Monteiro, Leandro Alves Patah, and Diógenes de Souza Bido. 2015. Project management and its effects on project success: Cross-country and cross-industry comparisons. International Journal of Project Management 33: 1509–22. [ Google Scholar ] [ CrossRef ]
• De Matteis, Fabio, Elio Borgonovi, Giovanni Notaristefano, and Fabrizio Striani. 2024. The contribution of public-private partnership (PPP) to sustainability: Governance and managerial implications from a literature review. Corporate Governance: The International Journal of Business in Society . Available online: https://www.emerald.com/insight/content/doi/10.1108/CG-03-2023-0103/full/html (accessed on 5 July 2024). [ CrossRef ]
• Debela, Getachew Yilma. 2022. Critical success factors (CSFs) of public–private partnership (PPP) road projects in Ethiopia. International Journal of Construction Management 22: 489–500. [ Google Scholar ] [ CrossRef ]
• Deng, Binchao, Dongjie Zhou, Jiachen Zhao, Yilin Yin, and Xiaoyu Li. 2021. Fuzzy synthetic evaluation of the critical success factors for the sustainability of public private partnership projects in China. Sustainability 13: 2551. [ Google Scholar ] [ CrossRef ]
• Fan, Yi, Jiquan Chen, Gabriela Shirkey, Ranjeet John, Susie R. Wu, Hogeun Park, and Changliang Shao. 2016. Applications of structural equation modeling (SEM) in ecological studies: An updated review. Ecological Processes 5: 1–12. [ Google Scholar ] [ CrossRef ]
• Fathi, Medya, and Pramen P. Shrestha. 2019a. Review of previous studies on public-private partnership infrastructure transportation and success factors. Paper presented at International Road Federation (IRF) Global R2T Conference, Las Vegas, NV USA, November 19; pp. 282–93. [ Google Scholar ]
• Fathi, Medya, and Pramen P. Shrestha. 2019b. Review of public-private partnerships and the current status of PPP highway projects in the United States. Paper presented at International Road Federation (IRF) Global R2T Conference, Las Vegas, NV USA, November 19; pp. 271–81. [ Google Scholar ]
• Fathi, Medya, and Pramen P. Shrestha. 2020. Project performance of public-private partnership projects based on various contract agreements and correlation with funding mechanism. In Construction Research Congress 2020 . Reston: American Society of Civil Engineers, pp. 1081–90. [ Google Scholar ]
• Fathi, Medya, and Pramen P. Shrestha. 2021. Funding and Finance Analysis of Public–Private Partnership Highway Projects: Exploratory Study. Journal of Legal Affairs and Dispute Resolution in Engineering and Construction 13: 04521015. [ Google Scholar ] [ CrossRef ]
• Fathi, Medya, and Pramen P. Shrestha. 2022. Public–private partnership project performance analysis compared to design-build in highway projects. Journal of Construction Engineering and Management 148: 04022118. [ Google Scholar ] [ CrossRef ]
• Fathi, Medya, and Pramen P. Shrestha. 2023a. Identification of critical success and risk factors for public–private partnership highway projects. Journal of Legal Affairs and Dispute Resolution in Engineering and Construction 15: 04522051. [ Google Scholar ] [ CrossRef ]
• Fathi, Medya, and Pramen P. Shrestha. 2023b. Public–Private Partnership Contract Framework Development for Highway Projects: A Delphi Approach. Journal of Legal Affairs and Dispute Resolution in Engineering and Construction 15: 04522046. [ Google Scholar ] [ CrossRef ]
• Fathi, Medya, and Pramen P. Shrestha. 2023c. Public–Private Partnership Highway Projects: Identifying Project Selection Factors. Journal of Legal Affairs and Dispute Resolution in Engineering and Construction 15: 04522052. [ Google Scholar ] [ CrossRef ]
• Gade, Rohit. 2022. Achieving Sustainable Construction through Construction & Demolition Waste Minimisation inResidential Building Projects. Ph.D. thesis, Auckland University of Technology, Auckland, New Zealand. [ Google Scholar ]
• Gao, Yuan, and Chun Kit Lau. 2021. Risk assessment of urban rail transit project using interpretative structural modelling: Evidence from China. Mathematical Problems in Engineering 1: 5581686. [ Google Scholar ] [ CrossRef ]
• Gaol, Dody Kurnia Lumban, Ratna Ayu Damayanti, and Andi Kusumawati. 2023. Critical Success Factor (CSF) of Implementing PPP Road Sector in Indonesia. Jurnal Perencanaan Pembangunan: The Indonesian Journal of Development Planning 2: 209–30. [ Google Scholar ]
• Ghorbani, Alireza, Mehdi Ravanshadnia, and Mohamad Bagher Nobakht. 2014. A survey of risks in public private partnership highway projects in Iran. Paper presented at ICCREM 2014: Smart Construction and Management in the Context of New Technology, Kunming, China, September 27–28; pp. 482–92. [ Google Scholar ]
• Gunduz, Murat, Khalid K. Naji, and Samar Al-Sharafi. 2024. Assessment of the Critical Dispute Factors in Public–Private Partnership Infrastructure Projects. Journal of Legal Affairs and Dispute Resolution in Engineering and Construction 16: 04524016. [ Google Scholar ] [ CrossRef ]
• Hai, Dinh Tuan, Nguyen Quoc Toan, and Nguyen Van Tam. 2022. Critical success factors for implementing PPP infrastructure projects in developing countries: The case of Vietnam. Innovative Infrastructure Solutions 1: 89. [ Google Scholar ] [ CrossRef ]
• Hair, Joseph F. 2009. Multivariate Data Analysis . London: Pearson. [ Google Scholar ]
• Hair, Joseph F., Jr., G. Tomas M. Hult, Christian M. Ringle, Marko Sarstedt, Nicholas P. Danks, and Soumya Ray. 2021. Partial Least Squares Structural Equation Modeling (PLS-SEM) Using R: A Workbook . Berlin: Springer Nature. [ Google Scholar ]
• Hang, Dinh Thi Thuy. 2019. Approaching Value for Money to assess viability of Public-Private partnership projects. Journal of International Studies 12: 229–38. [ Google Scholar ] [ CrossRef ]
• Hayduk, Leslie A. 1987. Structural Equation Modeling with LISREL: Essentials and Advances . Baltimore: Johns Hopkins University Press. [ Google Scholar ]
• Heydari, Mohammad, Kin Keung Lai, and Zhou Xiaohu. 2020. Risk Management in Public-Private Partnerships . London: Routledge. [ Google Scholar ]
• Ho, S. Ping, and Liang Y. Liu. 2002. An option pricing-based model for evaluating the financial viability of privatized infrastructure projects. Construction Management & Economics 20: 143–56. [ Google Scholar ]
• Hou, Deyi, Abir Al-Tabbaa, Haiqin Chen, and Ivanka Mamic. 2014. Factor analysis and structural equation modelling of sustainable behaviour in contaminated land remediation. Journal of Cleaner Production 84: 439–49. [ Google Scholar ] [ CrossRef ]
• Hoyle, Rick H. 2012. Handbook of Structural Equation Modeling . New York: Guilford Press. [ Google Scholar ]
• Hsueh, Chuen-Ming, and Luh-Maan Chang. 2017. Critical success factors for PPP infrastructure: Perspective from Taiwan. Journal of the Chinese Institute of Engineers 40: 370–77. [ Google Scholar ] [ CrossRef ]
• Hu, Zhen, Qianmeng Li, Tingting Liu, Lu Wang, and Zhe Cheng. 2021. Government equity investment, effective communication and public private partnership (PPP) performance: Evidence from China. Engineering, Construction and Architectural Management 28: 2811–27. [ Google Scholar ] [ CrossRef ]
• Hwang, Bon-Gang, Xianbo Zhao, and Mindy Jiang Shu Gay. 2013. Public private partnership projects in Singapore: Factors, critical risks and preferred risk allocation from the perspective of contractors. International Journal of Project Management 31: 424–33. [ Google Scholar ] [ CrossRef ]
• Iacobucci, Dawn. 2009. Everything you always wanted to know about SEM (structural equations modeling) but were afraid to ask. Journal of Consumer Psychology 19: 673–80. [ Google Scholar ] [ CrossRef ]
• Jacobson, Carol, and Sang Ok Choi. 2008. Success factors: Public works and public-private partnerships. International Journal of Public Sector Management 21: 637–57. [ Google Scholar ] [ CrossRef ]
• Jayasuriya, Sajani, Guomin Zhang, and Rebecca Jing Yang. 2019. Challenges in public private partnerships in construction industry: A review and further research directions. Built Environment Project and Asset Management 9: 172–85. [ Google Scholar ] [ CrossRef ]
• Jiang, Xiaoyan, Kun Lu, Bo Xia, Yong Liu, and Caiyun Cui. 2019. Identifying significant risks and analyzing risk relationship for construction PPP projects in China using integrated FISM-MICMAC approach. Sustainability 11: 5206. [ Google Scholar ] [ CrossRef ]
• Ke, Yongjian, Shouqing Wang, Albert P. C. Chan, and Patrick T. I. Lam. 2010. Preferred risk allocation in China’s public–private partnership (PPP) projects. International Journal of Project Management 28: 482–92. [ Google Scholar ] [ CrossRef ]
• Khahro, Shabir Hussain, Tauha Hussain Ali, Shafiqul Hassan, Noor Yasmin Zainun, Yasir Javed, and Shahbaz Aziz Memon. 2021. Risk severity matrix for sustainable public-private partnership projects in developing countries. Sustainability 13: 3292. [ Google Scholar ] [ CrossRef ]
• Khan, Syed Abdul Rehman, Yu Zhang, Anil Kumar, Edmundas Zavadskas, and Dalia Streimikiene. 2020. Measuring the impact of renewable energy, public health expenditure, logistics, and environmental performance on sustainable economic growth. Sustainable Development 28: 833–43. [ Google Scholar ] [ CrossRef ]
• Kineber, Ahmed Farouk, Ayodeji Oke, John Aliu, Mohammed Magdy Hamed, and Eguonor Oputu. 2023. Exploring the adoption of cyber (digital) technology for sustainable construction: A structural equation modeling of critical success factors. Sustainability 15: 5043. [ Google Scholar ] [ CrossRef ]
• Kline, Rex B. 2023. Principles and Practice of Structural Equation Modeling . New York: Guilford Publications. [ Google Scholar ]
• Koppenjan, Joop, Erik-Hans Klijn, Stefan Verweij, Mike Duijn, Ingmar van Meerkerk, Samantha Metselaar, and Rianne Warsen. 2022. The performance of public–private partnerships: An evaluation of 15 years DBFM in Dutch infrastructure governance. Public Performance & Management Review 45: 998–1028. [ Google Scholar ]
• Kukah, Augustine Senanu Komla, De-Graft Owusu-Manu, Edward Badu, and David John Edwards. 2023. Evaluation of risk factors in Ghanaian public-private-partnership (PPP) power projects using fuzzy synthetic evaluation methodology (FSEM). Benchmarking: An International Journal 30: 2554–82. [ Google Scholar ] [ CrossRef ]
• Kukah, Augustine Senanu Komla, De-Graft Owusu-Manu, Edward Badu, and David John Edwards. 2024. Delphi study for evaluating key risk factors inherent in public-private partnership (PPP) power projects in Ghana. Journal of Facilities Management 22: 124–43. [ Google Scholar ] [ CrossRef ]
• Kumar, Lakshya, Apurva Jindal, and Nagendra R. Velaga. 2018. Financial risk assessment and modelling of PPP based Indian highway infrastructure projects. Transport policy 62: 2–11. [ Google Scholar ] [ CrossRef ]
• Kwak, Young Hoon, YingYi Chih, and C. William Ibbs. 2009. Towards a comprehensive understanding of public private partnerships for infrastructure development. California Management Review 51: 51–78. [ Google Scholar ] [ CrossRef ]
• Le, Phuong Thi, Konstantinos Kirytopoulos, Nicholas Chileshe, and Raufdeen Rameezdeen. 2022. Taxonomy of risks in PPP transportation projects: A systematic literature review. International Journal of Construction Management 22: 166–81. [ Google Scholar ] [ CrossRef ]
• Lefcheck, Jonathan S. 2016. piecewiseSEM: Piecewise structural equation modelling in r for ecology, evolution, and systematics. Methods in Ecology and Evolution 7: 573–79. [ Google Scholar ] [ CrossRef ]
• Lember, Veiko, Ole Helby Petersen, Walter Scherrer, and Robert Ågren. 2019. Understanding the relationship between infrastructure public–private partnerships and innovation. Annals of Public and Cooperative Economics 90: 371–91. [ Google Scholar ] [ CrossRef ]
• Leruth, Luc E. 2012. Public-private cooperation in infrastructure development: A principal-agent story of contingent liabilities, fiscal risks, and other (un) pleasant surprises. Networks and Spatial Economics 12: 223–37. [ Google Scholar ] [ CrossRef ]
• Liang, Yanhong, and Hongjun Jia. 2018. Key success indicators for PPP projects: Evidence from Hong Kong. Advances in Civil Engineering 1: 9576496. [ Google Scholar ] [ CrossRef ]
• Likhitruangsilp, Veerasak, Masamitsu Onishi, and Phong Thanh Nguyen. 2017. Impacts of risk factors on the performance of public-private partnership transportation projects in Vietnam. ASEAN Engineering Journal 7: 30–52. [ Google Scholar ]
• Liu, Jingkuang, Yanqing Yi, and Xuetong Wang. 2020. Exploring factors influencing construction waste reduction: A structural equation modeling approach. Journal of Cleaner Production 276: 123185. [ Google Scholar ] [ CrossRef ]
• Liu, Junxiao, Peter ED Love, Jim Smith, Michael Regan, and Peter R. Davis. 2015. Life cycle critical success factors for public-private partnership infrastructure projects. Journal of Management in Engineering 31: 04014073. [ Google Scholar ] [ CrossRef ]
• Liu, Yong, Chenjunyan Sun, Bo Xia, Sai Liu, and Martin Skitmore. 2018. Identification of risk factors affecting PPP waste-to-energy incineration projects in China: A multiple case study. Advances in Civil Engineering 1: 4983523. [ Google Scholar ] [ CrossRef ]
• Loehlin, John C. 2004. Latent Variable Models: An Introduction to Factor, Path, and Structural Equation Analysis . London: Psychology Press. [ Google Scholar ]
• Mainul Islam, M. D., and Olusegun O. Faniran. 2005. Structural equation model of project planning effectiveness. Construction Management and Economics 23: 215–23. [ Google Scholar ] [ CrossRef ]
• Meissner, Dirk. 2019. Public-private partnership models for science, technology, and innovation cooperation. Journal of the Knowledge Economy 10: 1341–61. [ Google Scholar ] [ CrossRef ]
• Meng, Xianhai, Qi Zhao, and Qiping Shen. 2011. Critical success factors for transfer-operate-transfer urban water supply projects in China. Journal of Management in Engineering 27: 243–51. [ Google Scholar ] [ CrossRef ]
• Mladenovic, Goran, Nevena Vajdic, Bjorn Wündsch, and Alenka Temeljotov-Salaj. 2013. Use of key performance indicators for PPP transport projects to meet stakeholders’ performance objectives. Built Environment Project and Asset Management 3: 228–49. [ Google Scholar ] [ CrossRef ]
• Molenaar, Keith, Simon Washington, and James Diekmann. 2000. Structural equation model of construction contract dispute potential. Journal of Construction Engineering and Management 126: 268–77. [ Google Scholar ] [ CrossRef ]
• Mueller, Ralph O., and Gregory R. Hancock. 2018. Structural equation modeling. In The Reviewer’s Guide to Quantitative Methods in the Social Sciences . London: Routledge, pp. 445–56. [ Google Scholar ]
• Mukherjee, Subhodeep, Manish Mohan Baral, Venkataiah Chittipaka, Surya Kant Pal, and Ramji Nagariya. 2022. Investigating sustainable development for the COVID-19 vaccine supply chain: A structural equation modelling approach. Journal of Humanitarian Logistics and Supply Chain Management 13: 199–215. [ Google Scholar ] [ CrossRef ]
• Nguyen, Anh, Abu Mollik, and Ying-Yi Chih. 2018. Managing critical risks affecting the financial viability of public–private partnership projects: Case study of toll road projects in Vietnam. Journal of Construction Engineering and Management 144: 05018014. [ Google Scholar ] [ CrossRef ]
• Nguyen, Phong Thanh, Veerasak Likhitruangsilp, and Masamitsu Onishi. 2020. Success factors for public-private partnership infrastructure projects in Vietnam. International Journal on Advanced Science, Engineering and Information Technology 10: 858–65. [ Google Scholar ] [ CrossRef ]
• Nunnally, Jum C. 1978. Psychometric Theory , 2nd ed. New York: McGraw-Hill. [ Google Scholar ]
• Nunnally, Jum, and Ira Bernstein. 2010. Psychometric theory, 3rd edn., internat. stud. ed. In McGraw-Hill Series in Psychology . New Delhi: Tata McGraw-Hill Ed. [ Google Scholar ]
• Osei-Kyei, Robert, Albert P. C. Chan, Yao Yu, Chuan Chen, and Ayirebi Dansoh. 2019. Root causes of conflict and conflict resolution mechanisms in public-private partnerships: Comparative study between Ghana and China. Cities 87: 185–95. [ Google Scholar ] [ CrossRef ]
• Osei-Kyei, Robert, and Albert P. C. Chan. 2017. Risk assessment in public-private partnership infrastructure projects: Empirical comparison between Ghana and Hong Kong. Construction innovation 17: 204–23. [ Google Scholar ] [ CrossRef ]
• Osei-Kyei, Robert, Xiaohua Jin, Chukwuma Nnaji, Isaac Akomea-Frimpong, and Ibrahim Y. Wuni. 2023. Review of risk management studies in public-private partnerships: A scientometric analysis. International Journal of Construction Management 23: 2419–30. [ Google Scholar ] [ CrossRef ]
• Paez-Perez, David, and Mauricio Sanchez-Silva. 2016. A dynamic principal-agent framework for modeling the performance of infrastructure. European Journal of Operational Research 2: 576–94. [ Google Scholar ] [ CrossRef ]
• Panda, Debadutta Kumar. 2016. Public private partnerships and value creation: The role of relationship dynamics. International Journal of Organizational Analysis 24: 162–83. [ Google Scholar ] [ CrossRef ]
• Prus, Piotr, and Marek Sikora. 2021. The impact of transport infrastructure on the sustainable development of the region—Case study. Agriculture 11: 279. [ Google Scholar ] [ CrossRef ]
• Public Works Financing. 2016. Reports. Available online: https://azdot.gov/sites/default/files/2019/08/p3-pwf-feb-2016.pdf (accessed on 5 March 2020).
• Queiroz, Cesar, and Goran Mladenovic. 2020. A review of tools for project financial assessments. Transportation Research Procedia 45: 54–61. [ Google Scholar ] [ CrossRef ]
• Ramli, Nabila Huwaina, Hamimah Adnan, Har Einur Azrin Baharuddin, Nor Azmi Bakhary, and Zul Zakiyuddin Ahmad Rashid. 2022. Financial risk in managing public-private partnership (PPP) project. IOP Conference Series: Earth and Environmental Science 1067: 012074. [ Google Scholar ] [ CrossRef ]
• Rangel, Thais, and José Manuel Vassallo. 2015. Modeling the effect of contractual incentives on road safety performance. Transport Policy 40: 17–23. [ Google Scholar ] [ CrossRef ]
• Rasheed, Nasir, Wajiha Shahzad, Malik Khalfan, and James Olabode Bamidele Rotimi. 2022. Risk identification, assessment, and allocation in PPP projects: A systematic review. Buildings 12: 1109. [ Google Scholar ] [ CrossRef ]
• Rejeb, Abderahman, Karim Rejeb, Andrea Appolloni, Suhaiza Zailani, and Mohammad Iranmanesh. 2024. Navigating the landscape of public–private partnership research: A novel review using latent Dirichlet allocation. International Journal of Public Sector Management . Available online: https://www.emerald.com/insight/content/doi/10.1108/IJPSM-09-2023-0263/full/html (accessed on 5 July 2024). [ CrossRef ]
• Ross, Thomas W., and Jing Yan. 2020. Comparing public-private partnerships and traditional public procurement: Efficiency vs. flexibility. In Institutions and Governance in Comparative Policy Analysis Studies . London: Routledge, pp. 67–85. [ Google Scholar ]
• Ruiz Díaz, Gonzalo. 2020. What drives the failure of private participation in infrastructure projects? International Journal of Managing Projects in Business 13: 1167–85. [ Google Scholar ] [ CrossRef ]
• Ruuska, Inkeri, and Robin Teigland. 2009. Ensuring project success through collective competence and creative conflict in public–private partnerships—A case study of Bygga Villa, a Swedish triple helix e-government initiative. International Journal of Project Management 27: 323–34. [ Google Scholar ] [ CrossRef ]
• Rybnicek, Robert, Julia Plakolm, and Lisa Baumgartner. 2020. Risks in public–private partnerships: A systematic literature review of risk factors, their impact and risk mitigation strategies. Public Performance & Management Review 43: 1174–208. [ Google Scholar ]
• Sarkar, Debasis, Naimish Bhatt, Kamal Singh, and Goutam Dutta. 2021. Critical Success Factors (Csfs) For Public Private Partnership Road Projects In Western India. International Journal of Construction Project Management 13: 89–114. [ Google Scholar ]
• Scalco, Andrea, Stefano Noventa, Riccardo Sartori, and Andrea Ceschi. 2017. Predicting organic food consumption: A meta-analytic structural equation model based on the theory of planned behavior. Appetite 112: 235–48. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Scott, W. Richard, Raymond E. Levitt, and Michael J. Garvin. 2019. Introduction: PPPs–theoretical challenges and directions forward. Public–Private Partnerships for Infrastructure Development . Available online: https://www.elgaronline.com/edcollchap/edcoll/9781788973175/9781788973175.00006.xml (accessed on 5 July 2024).
• Sehgal, Reetika, and Ashish Mohan Dubey. 2019. Identification of critical success factors for public–private partnership projects. Journal of Public Affairs 19: e1956. [ Google Scholar ] [ CrossRef ]
• Seo, Hee-Chang, Yoon-Sun Lee, Jae-Jun Kim, and Nam-Yong Jee. 2015. Analyzing safety behaviors of temporary construction workers using structural equation modeling. Safety Science 77: 160–68. [ Google Scholar ] [ CrossRef ]
• Shen, Wenxin, Wenzhe Tang, Wenyang Yu, Colin F. Duffield, Felix Kin Peng Hui, Yongping Wei, and Jun Fang. 2017. Causes of contractors’ claims in international engineering-procurement-construction projects. Journal of Civil Engineering and Management 23: 727–39. [ Google Scholar ] [ CrossRef ]
• Shi, Shiying, Heap-Yih Chong, Lihong Liu, and Xiaosu Ye. 2016. Examining the interrelationship among critical success factors of public private partnership infrastructure projects. Sustainability 8: 1313. [ Google Scholar ] [ CrossRef ]
• Shrestha, Noora. 2021. Factor analysis as a tool for survey analysis. American Journal of Applied Mathematics and Statistics 9: 4–11. [ Google Scholar ] [ CrossRef ]
• Siemiatycki, Matti, and Jonathan Friedman. 2012. The trade-offs of transferring demand risk on urban transit public–private partnerships. Public Works Management & Policy 17: 283–302. [ Google Scholar ]
• Sinha, Ajit Kumar, and Kumar Neeraj Jha. 2020. Dispute resolution and litigation in PPP road projects: Evidence from select cases. Journal of Legal Affairs and Dispute Resolution in Engineering and Construction 12: 05019007. [ Google Scholar ] [ CrossRef ]
• Soliño, Antonio Sánchez, and Pilar Gago de Santos. 2010. Transaction costs in transport public–private partnerships: Comparing procurement procedures. Transport Reviews 30: 389–406. [ Google Scholar ] [ CrossRef ]
• Sun, Yu, Lizhen Chen, Huaping Sun, and Farhad Taghizadeh-Hesary. 2020. Low-carbon financial risk factor correlation in the belt and road PPP project. Finance Research Letters 35: 101491. [ Google Scholar ] [ CrossRef ]
• Surachman, Eko Nur, Dian Handayani, Maman Suhendra, and Sakti Prabowo. 2020. Critical success factors on PPP water project in a developing country: Evidence from Indonesia. The Journal of Asian Finance, Economics and Business 7: 1071–80. [ Google Scholar ] [ CrossRef ]
• Tabachnick, Barbara G., Linda S. Fidell, and Jodie B. Ullman. 2013. Using Multivariate Statistics . Boston: Pearson. [ Google Scholar ]
• Tallaki, Mouhcine, and Enrico Bracci. 2021. Risk allocation, transfer and management in public–private partnership and private finance initiatives: A systematic literature review. International Journal of Public Sector Management 34: 709–31. [ Google Scholar ] [ CrossRef ]
• Tang, LiYaning, Qiping Shen, Martin Skitmore, and Eddie WL Cheng. 2013. Ranked critical factors in PPP briefings. Journal of Management in Engineering 29: 164–71. [ Google Scholar ] [ CrossRef ]
• Tsai, Wen-Hsien, Michael J. Shaw, Yi-Wen Fan, Jau-Yang Liu, Kuen-Chang Lee, and Hui-Chiao Chen. 2011. An empirical investigation of the impacts of internal/external facilitators on the project success of ERP: A structural equation model. Decision Support Systems 50: 480–90. [ Google Scholar ] [ CrossRef ]
• US Census Bureau. 2023. Construction Expenditure ; Washington, DC: US Department of Commerce. Available online: https://www.census.gov/construction/c30/historical_data.html (accessed on 20 February 2023).
• Välilä, Timo. 2020. An overview of economic theory and evidence of public-private partnerships in the procurement of (transport) infrastructure. Utilities Policy 62: 100995. [ Google Scholar ] [ CrossRef ]
• Wang, Chun Mei, Bing Bing Xu, Su Juan Zhang, and Yong Qiang Chen. 2016. Influence of personality and risk propensity on risk perception of Chinese construction project managers. International Journal of Project Management 34: 1294–304. [ Google Scholar ] [ CrossRef ]
• Wang, Jichuan, and Xiaoqian Wang. 2012. Structural Equation Modeling: Applications Using Mplus . New York: John Wiley & Sons. [ Google Scholar ]
• Wang, Yan, Yujie Wang, Xiuyu Wu, and Jiwang Li. 2020. Exploring the risk factors of infrastructure PPP projects for sustainable delivery: A social network perspective. Sustainability 12: 4152. [ Google Scholar ] [ CrossRef ]
• Wang, Yongqi, Zhe Shao, and Robert LK Tiong. 2021. Data-driven prediction of contract failure of public-private partnership projects. Journal of Construction Engineering and Management 147: 04021089. [ Google Scholar ] [ CrossRef ]
• Waring, Justin, Graeme Currie, and Simon Bishop. 2013. A contingent approach to the organization and management of public–private partnerships: An empirical study of English health care. Public Administration Review 73: 313–26. [ Google Scholar ] [ CrossRef ]
• Wong, Peter Shek Pui, and Sai On Cheung. 2005. Structural equation model of trust and partnering success. Journal of Management in Engineering 21: 70–80. [ Google Scholar ] [ CrossRef ]
• Yuan, Jingfeng, Wei Li, Jiyue Guo, Xianbo Zhao, and Mirosław J. Skibniewski. 2018. Social risk factors of transportation PPP projects in China: A sustainable development perspective. International Journal of Environmental Research and Public Health 15: 1323. [ Google Scholar ] [ CrossRef ]
• Zhang, Jing, Fangjian Chen, and X-X. Yuan. 2020. Comparison of cost and schedule performance of large public projects under P3 and traditional delivery models: A Canadian study. Construction Management and Economics 38: 739–55. [ Google Scholar ] [ CrossRef ]
• Zhang, Juan, and William X. Wei. 2012. Managing political risks of Chinese contracted projects in Libya. Project Management Journal 43: 42–51. [ Google Scholar ] [ CrossRef ]
• Zhao, Nan, and Fei Ying. 2019. Method selection: A conceptual framework for public sector PPP selection. Built Environment Project and Asset Management 9: 214–32. [ Google Scholar ] [ CrossRef ]
• Zuo, Jian, Xianbo Zhao, Quan Bui Minh Nguyen, Tony Ma, and Shang Gao. 2018. Soft skills of construction project management professionals and project success factors: A structural equation model. Engineering, Construction and Architectural Management 25: 425–42. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Fathi, M. A Structural Equation Model on Critical Risk and Success in Public–Private Partnership: Exploratory Study. J. Risk Financial Manag. 2024 , 17 , 354. https://doi.org/10.3390/jrfm17080354

Fathi M. A Structural Equation Model on Critical Risk and Success in Public–Private Partnership: Exploratory Study. Journal of Risk and Financial Management . 2024; 17(8):354. https://doi.org/10.3390/jrfm17080354

Fathi, Medya. 2024. "A Structural Equation Model on Critical Risk and Success in Public–Private Partnership: Exploratory Study" Journal of Risk and Financial Management 17, no. 8: 354. https://doi.org/10.3390/jrfm17080354

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Home > ETD > DISSERTATIONS > AAI10792825

A Systematic Literature Review of Project Management Tools and Their Impact on Project Management Effectiveness

Michelle Newton , Purdue University

This thesis was conducted to examine project effectiveness and the best methodologies to use for managing projects as supported by the literature. Project effectiveness depends on multiple dimensions and has many situational issues that demand flexibility in method selection and utilization. Three of the most important dimensions explored in this study included: organizational structures, cost and schedule management typologies and leadership styles. A systematic literature review was utilized to collect data on the most common project management methods found to be effective according to the current literature. As projects are complex and vary greatly it was concluded from the literature that project management effectiveness cannot be measured by a single metric. An outline of methods and tools that may best be utilized based on project type and needs was created to better understand potential applications for each method and tool.

Springer, Purdue University.

Subject Area

Management|Industrial engineering

Off-Campus Purdue Users: To access this dissertation, please log in to our proxy server .

Since June 18, 2018

Advanced Search

• Notify me via email or RSS
• Purdue Libraries
• Purdue University Press Open Access Collections

Links for Authors

• Policies and Help Documentation
• Collections
• Disciplines

Home | About | FAQ | My Account | Accessibility Statement

Privacy Copyright

Pardon Our Interruption

As you were browsing something about your browser made us think you were a bot. There are a few reasons this might happen:

• You've disabled JavaScript in your web browser.
• You're a power user moving through this website with super-human speed.
• You've disabled cookies in your web browser.
• A third-party browser plugin, such as Ghostery or NoScript, is preventing JavaScript from running. Additional information is available in this support article .

To regain access, please make sure that cookies and JavaScript are enabled before reloading the page.

Sweetspire Literature Management

Updated 1 month ago

Photos & videos

See all 24 photos

Location & Hours

Suggest an edit

155 Willowbrook Blvd

Wayne, NJ 07470

Amenities and More

Ask the community.

Ask a question

Yelp users haven’t asked any questions yet about Sweetspire Literature Management .

People also searched for

advertising services

brand management

digital marketing

professional services

Recommended Reviews

• Systematic review
• Open access
• Published: 07 August 2024

Models and frameworks for assessing the implementation of clinical practice guidelines: a systematic review

• Nicole Freitas de Mello   ORCID: orcid.org/0000-0002-5228-6691 1 , 2 ,
• Sarah Nascimento Silva   ORCID: orcid.org/0000-0002-1087-9819 3 ,
• Dalila Fernandes Gomes   ORCID: orcid.org/0000-0002-2864-0806 1 , 2 ,
• Juliana da Motta Girardi   ORCID: orcid.org/0000-0002-7547-7722 4 &
• Jorge Otávio Maia Barreto   ORCID: orcid.org/0000-0002-7648-0472 2 , 4  

Implementation Science volume  19 , Article number:  59 ( 2024 ) Cite this article

790 Accesses

6 Altmetric

Metrics details

The implementation of clinical practice guidelines (CPGs) is a cyclical process in which the evaluation stage can facilitate continuous improvement. Implementation science has utilized theoretical approaches, such as models and frameworks, to understand and address this process. This article aims to provide a comprehensive overview of the models and frameworks used to assess the implementation of CPGs.

A systematic review was conducted following the Cochrane methodology, with adaptations to the "selection process" due to the unique nature of this review. The findings were reported following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) reporting guidelines. Electronic databases were searched from their inception until May 15, 2023. A predetermined strategy and manual searches were conducted to identify relevant documents from health institutions worldwide. Eligible studies presented models and frameworks for assessing the implementation of CPGs. Information on the characteristics of the documents, the context in which the models were used (specific objectives, level of use, type of health service, target group), and the characteristics of each model or framework (name, domain evaluated, and model limitations) were extracted. The domains of the models were analyzed according to the key constructs: strategies, context, outcomes, fidelity, adaptation, sustainability, process, and intervention. A subgroup analysis was performed grouping models and frameworks according to their levels of use (clinical, organizational, and policy) and type of health service (community, ambulatorial, hospital, institutional). The JBI’s critical appraisal tools were utilized by two independent researchers to assess the trustworthiness, relevance, and results of the included studies.

Database searches yielded 14,395 studies, of which 80 full texts were reviewed. Eight studies were included in the data analysis and four methodological guidelines were additionally included from the manual search. The risk of bias in the studies was considered non-critical for the results of this systematic review. A total of ten models/frameworks for assessing the implementation of CPGs were found. The level of use was mainly policy, the most common type of health service was institutional, and the major target group was professionals directly involved in clinical practice. The evaluated domains differed between the models and there were also differences in their conceptualization. All the models addressed the domain "Context", especially at the micro level (8/12), followed by the multilevel (7/12). The domains "Outcome" (9/12), "Intervention" (8/12), "Strategies" (7/12), and "Process" (5/12) were frequently addressed, while "Sustainability" was found only in one study, and "Fidelity/Adaptation" was not observed.

Conclusions

The use of models and frameworks for assessing the implementation of CPGs is still incipient. This systematic review may help stakeholders choose or adapt the most appropriate model or framework to assess CPGs implementation based on their specific health context.

Trial registration

PROSPERO (International Prospective Register of Systematic Reviews) registration number: CRD42022335884. Registered on June 7, 2022.

Peer Review reports

Contributions to the literature

Although the number of theoretical approaches has grown in recent years, there are still important gaps to be explored in the use of models and frameworks to assess the implementation of clinical practice guidelines (CPGs). This systematic review aims to contribute knowledge to overcome these gaps.

Despite the great advances in implementation science, evaluating the implementation of CPGs remains a challenge, and models and frameworks could support improvements in this field.

This study demonstrates that the available models and frameworks do not cover all characteristics and domains necessary for a complete evaluation of CPGs implementation.

The presented findings contribute to the field of implementation science, encouraging debate on choices and adaptations of models and frameworks for implementation research and evaluation.

Substantial investments have been made in clinical research and development in recent decades, increasing the medical knowledge base and the availability of health technologies [ 1 ]. The use of clinical practice guidelines (CPGs) has increased worldwide to guide best health practices and to maximize healthcare investments. A CPG can be defined as "any formal statements systematically developed to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances" [ 2 ] and has the potential to improve patient care by promoting interventions of proven benefit and discouraging ineffective interventions. Furthermore, they can promote efficiency in resource allocation and provide support for managers and health professionals in decision-making [ 3 , 4 ].

However, having a quality CPG does not guarantee that the expected health benefits will be obtained. In fact, putting these devices to use still presents a challenge for most health services across distinct levels of government. In addition to the development of guidelines with high methodological rigor, those recommendations need to be available to their users; these recommendations involve the diffusion and dissemination stages, and they need to be used in clinical practice (implemented), which usually requires behavioral changes and appropriate resources and infrastructure. All these stages involve an iterative and complex process called implementation, which is defined as the process of putting new practices within a setting into use [ 5 , 6 ].

Implementation is a cyclical process, and the evaluation is one of its key stages, which allows continuous improvement of CPGs development and implementation strategies. It consists of verifying whether clinical practice is being performed as recommended (process evaluation or formative evaluation) and whether the expected results and impact are being reached (summative evaluation) [ 7 , 8 , 9 ]. Although the importance of the implementation evaluation stage has been recognized, research on how these guidelines are implemented is scarce [ 10 ]. This paper focused on the process of assessing CPGs implementation.

To understand and improve this complex process, implementation science provides a systematic set of principles and methods to integrate research findings and other evidence-based practices into routine practice and improve the quality and effectiveness of health services and care [ 11 ]. The field of implementation science uses theoretical approaches that have varying degrees of specificity based on the current state of knowledge and are structured based on theories, models, and frameworks [ 5 , 12 , 13 ]. A "Model" is defined as "a simplified depiction of a more complex world with relatively precise assumptions about cause and effect", and a "framework" is defined as "a broad set of constructs that organize concepts and data descriptively without specifying causal relationships" [ 9 ]. Although these concepts are distinct, in this paper, their use will be interchangeable, as they are typically like checklists of factors relevant to various aspects of implementation.

There are a variety of theoretical approaches available in implementation science [ 5 , 14 ], which can make choosing the most appropriate challenging [ 5 ]. Some models and frameworks have been categorized as "evaluation models" by providing a structure for evaluating implementation endeavors [ 15 ], even though theoretical approaches from other categories can also be applied for evaluation purposes because they specify concepts and constructs that may be operationalized and measured [ 13 ]. Two frameworks that can specify implementation aspects that should be evaluated as part of intervention studies are RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) [ 16 ] and PRECEDE-PROCEED (Predisposing, Reinforcing and Enabling Constructs in Educational Diagnosis and Evaluation-Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development) [ 17 ]. Although the number of theoretical approaches has grown in recent years, the use of models and frameworks to evaluate the implementation of guidelines still seems to be a challenge.

This article aims to provide a complete map of the models and frameworks applied to assess the implementation of CPGs. The aim is also to subside debate and choices on models and frameworks for the research and evaluation of the implementation processes of CPGs and thus to facilitate the continued development of the field of implementation as well as to contribute to healthcare policy and practice.

A systematic review was conducted following the Cochrane methodology [ 18 ], with adaptations to the "selection process" due to the unique nature of this review (details can be found in the respective section). The review protocol was registered in PROSPERO (registration number: CRD42022335884) on June 7, 2022. This report adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [ 19 ] and a completed checklist is provided in Additional File 1.

Eligibility criteria

The SDMO approach (Types of Studies, Types of Data, Types of Methods, Outcomes) [ 20 ] was utilized in this systematic review, outlined as follows:

Types of studies

All types of studies were considered for inclusion, as the assessment of CPG implementation can benefit from a diverse range of study designs, including randomized clinical trials/experimental studies, scale/tool development, systematic reviews, opinion pieces, qualitative studies, peer-reviewed articles, books, reports, and unpublished theses.

Studies were categorized based on their methodological designs, which guided the synthesis, risk of bias assessment, and presentation of results.

Study protocols and conference abstracts were excluded due to insufficient information for this review.

Types of data

Studies that evaluated the implementation of CPGs either independently or as part of a multifaceted intervention.

Guidelines for evaluating CPG implementation.

Inclusion of CPGs related to any context, clinical area, intervention, and patient characteristics.

No restrictions were placed on publication date or language.

Exclusion criteria

General guidelines were excluded, as this review focused on 'models for evaluating clinical practice guidelines implementation' rather than the guidelines themselves.

Studies that focused solely on implementation determinants as barriers and enablers were excluded, as this review aimed to explore comprehensive models/frameworks.

Studies evaluating programs and policies were excluded.

Studies that only assessed implementation strategies (isolated actions) rather than the implementation process itself were excluded.

Studies that focused solely on the impact or results of implementation (summative evaluation) were excluded.

Types of methods

Not applicable.

All potential models or frameworks for assessing the implementation of CPG (evaluation models/frameworks), as well as their characteristics: name; specific objectives; levels of use (clinical, organizational, and policy); health system (public, private, or both); type of health service (community, ambulatorial, hospital, institutional, homecare); domains or outcomes evaluated; type of recommendation evaluated; context; limitations of the model.

Model was defined as "a deliberated simplification of a phenomenon on a specific aspect" [ 21 ].

Framework was defined as "structure, overview outline, system, or plan consisting of various descriptive categories" [ 21 ].

Models or frameworks used solely for the CPG development, dissemination, or implementation phase.

Models/frameworks used solely for assessment processes other than implementation, such as for the development or dissemination phase.

Data sources and literature search

The systematic search was conducted on July 31, 2022 (and updated on May 15, 2023) in the following electronic databases: MEDLINE/PubMed, Centre for Reviews and Dissemination (CRD), the Cochrane Library, Cumulative Index to Nursing and Allied Health Literature (CINAHL), EMBASE, Epistemonikos, Global Health, Health Systems Evidence, PDQ-Evidence, PsycINFO, Rx for Change (Canadian Agency for Drugs and Technologies in Health, CADTH), Scopus, Web of Science and Virtual Health Library (VHL). The Google Scholar database was used for the manual selection of studies (first 10 pages).

Additionally, hand searches were performed on the lists of references included in the systematic reviews and citations of the included studies, as well as on the websites of institutions working on CPGs development and implementation: Guidelines International Networks (GIN), National Institute for Health and Care Excellence (NICE; United Kingdom), World Health Organization (WHO), Centers for Disease Control and Prevention (CDC; USA), Institute of Medicine (IOM; USA), Australian Department of Health and Aged Care (ADH), Healthcare Improvement Scotland (SIGN), National Health and Medical Research Council (NHMRC; Australia), Queensland Health, The Joanna Briggs Institute (JBI), Ministry of Health and Social Policy of Spain, Ministry of Health of Brazil and Capes Theses and Dissertations Catalog.

The search strategy combined terms related to "clinical practice guidelines" (practice guidelines, practice guidelines as topic, clinical protocols), "implementation", "assessment" (assessment, evaluation), and "models, framework". The free term "monitoring" was not used because it was regularly related to clinical monitoring and not to implementation monitoring. The search strategies adapted for the electronic databases are presented in an additional file (see Additional file 2).

Study selection process

The results of the literature search from scientific databases, excluding the CRD database, were imported into Mendeley Reference Management software to remove duplicates. They were then transferred to the Rayyan platform ( https://rayyan.qcri.org ) [ 22 ] for the screening process. Initially, studies related to the "assessment of implementation of the CPG" were selected. The titles were first screened independently by two pairs of reviewers (first selection: four reviewers, NM, JB, SS, and JG; update: a pair of reviewers, NM and DG). The title screening was broad, including all potentially relevant studies on CPG and the implementation process. Following that, the abstracts were independently screened by the same group of reviewers. The abstract screening was more focused, specifically selecting studies that addressed CPG and the evaluation of the implementation process. In the next step, full-text articles were reviewed independently by a pair of reviewers (NM, DG) to identify those that explicitly presented "models" or "frameworks" for assessing the implementation of the CPG. Disagreements regarding the eligibility of studies were resolved through discussion and consensus, and by a third reviewer (JB) when necessary. One reviewer (NM) conducted manual searches, and the inclusion of documents was discussed with the other reviewers.

Risk of bias assessment of studies

The selected studies were independently classified and evaluated according to their methodological designs by two investigators (NM and JG). This review employed JBI’s critical appraisal tools to assess the trustworthiness, relevance and results of the included studies [ 23 ] and these tools are presented in additional files (see Additional file 3 and Additional file 4). Disagreements were resolved by consensus or consultation with the other reviewers. Methodological guidelines and noncomparative and before–after studies were not evaluated because JBI does not have specific tools for assessing these types of documents. Although the studies were assessed for quality, they were not excluded on this basis.

Data extraction

The data was independently extracted by two reviewers (NM, DG) using a Microsoft Excel spreadsheet. Discrepancies were discussed and resolved by consensus. The following information was extracted:

Document characteristics : author; year of publication; title; study design; instrument of evaluation; country; guideline context;

Usage context of the models : specific objectives; level of use (clinical, organizational, and policy); type of health service (community, ambulatorial, hospital, institutional); target group (guideline developers, clinicians; health professionals; health-policy decision-makers; health-care organizations; service managers);

Model and framework characteristics : name, domain evaluated, and model limitations.

The set of information to be extracted, shown in the systematic review protocol, was adjusted to improve the organization of the analysis.

The "level of use" refers to the scope of the model used. "Clinical" was considered when the evaluation focused on individual practices, "organizational" when practices were within a health service institution, and "policy" when the evaluation was more systemic and covered different health services or institutions.

The "type of health service" indicated the category of health service where the model/framework was used (or can be used) to assess the implementation of the CPG, related to the complexity of healthcare. "Community" is related to primary health care; "ambulatorial" is related to secondary health care; "hospital" is related to tertiary health care; and "institutional" represented models/frameworks not specific to a particular type of health service.

The "target group" included stakeholders related to the use of the model/framework for evaluating the implementation of the CPG, such as clinicians, health professionals, guideline developers, health policy-makers, health organizations, and service managers.

The category "health system" (public, private, or both) mentioned in the systematic review protocol was not found in the literature obtained and was removed as an extraction variable. Similarly, the variables "type of recommendation evaluated" and "context" were grouped because the same information was included in the "guideline context" section of the study.

Some selected documents presented models or frameworks recognized by the scientific field, including some that were validated. However, some studies adapted the model to this context. Therefore, the domain analysis covered all models or frameworks domains evaluated by (or suggested for evaluation by) the document analyzed.

Data analysis and synthesis

The results were tabulated using narrative synthesis with an aggregative approach, without meta-analysis, aiming to summarize the documents descriptively for the organization, description, interpretation and explanation of the study findings [ 24 , 25 ].

The model/framework domains evaluated in each document were studied according to Nilsen et al.’s constructs: "strategies", "context", "outcomes", "fidelity", "adaptation" and "sustainability". For this study, "strategies" were described as structured and planned initiatives used to enhance the implementation of clinical practice [ 26 ].

The definition of "context" varies in the literature. Despite that, this review considered it as the set of circumstances or factors surrounding a particular implementation effort, such as organizational support, financial resources, social relations and support, leadership, and organizational culture [ 26 , 27 ]. The domain "context" was subdivided according to the level of health care into "micro" (individual perspective), "meso" (organizational perspective), "macro" (systemic perspective), and "multiple" (when there is an issue involving more than one level of health care).

The "outcomes" domain was related to the results of the implementation process (unlike clinical outcomes) and was stratified according to the following constructs: acceptability, appropriateness, feasibility, adoption, cost, and penetration. All these concepts align with the definitions of Proctor et al. (2011), although we decided to separate "fidelity" and "sustainability" as independent domains similar to Nilsen [ 26 , 28 ].

"Fidelity" and "adaptation" were considered the same domain, as they are complementary pieces of the same issue. In this study, implementation fidelity refers to how closely guidelines are followed as intended by their developers or designers. On the other hand, adaptation involves making changes to the content or delivery of a guideline to better fit the needs of a specific context. The "sustainability" domain was defined as evaluations about the continuation or permanence over time of the CPG implementation.

Additionally, the domain "process" was utilized to address issues related to the implementation process itself, rather than focusing solely on the outcomes of the implementation process, as done by Wang et al. [ 14 ]. Furthermore, the "intervention" domain was introduced to distinguish aspects related to the CPG characteristics that can impact its implementation, such as the complexity of the recommendation.

A subgroup analysis was performed with models and frameworks categorized based on their levels of use (clinical, organizational, and policy) and the type of health service (community, ambulatorial, hospital, institutional) associated with the CPG. The goal is to assist stakeholders (politicians, clinicians, researchers, or others) in selecting the most suitable model for evaluating CPG implementation based on their specific health context.

Search results

Database searches yielded 26,011 studies, of which 107 full texts were reviewed. During the full-text review, 99 articles were excluded: 41 studies did not mention a model or framework for assessing the implementation of the CPG, 31 studies evaluated only implementation strategies (isolated actions) rather than the implementation process itself, and 27 articles were not related to the implementation assessment. Therefore, eight studies were included in the data analysis. The updated search did not reveal additional relevant studies. The main reason for study exclusion was that they did not use models or frameworks to assess CPG implementation. Additionally, four methodological guidelines were included from the manual search (Fig.  1 ).

PRISMA diagram. Acronyms: ADH—Australian Department of Health, CINAHL—Cumulative Index to Nursing and Allied Health Literature, CDC—Centers for Disease Control and Prevention, CRD—Centre for Reviews and Dissemination, GIN—Guidelines International Networks, HSE—Health Systems Evidence, IOM—Institute of Medicine, JBI—The Joanna Briggs Institute, MHB—Ministry of Health of Brazil, NICE—National Institute for Health and Care Excellence, NHMRC—National Health and Medical Research Council, MSPS – Ministerio de Sanidad Y Política Social (Spain), SIGN—Scottish Intercollegiate Guidelines Network, VHL – Virtual Health Library, WHO—World Health Organization. Legend: Reason A –The study evaluated only implementation strategies (isolated actions) rather than the implementation process itself. Reason B – The study did not mention a model or framework for assessing the implementation of the intervention. Reason C – The study was not related to the implementation assessment. Adapted from Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. https://doi.org/10.1136/bmj.n71 . For more information, visit:

According to the JBI’s critical appraisal tools, the overall assessment of the studies indicates their acceptance for the systematic review.

The cross-sectional studies lacked clear information regarding "confounding factors" or "strategies to address confounding factors". This was understandable given the nature of the study, where such details are not typically included. However, the reviewers did not find this lack of information to be critical, allowing the studies to be included in the review. The results of this methodological quality assessment can be found in an additional file (see Additional file 5).

In the qualitative studies, there was some ambiguity regarding the questions: "Is there a statement locating the researcher culturally or theoretically?" and "Is the influence of the researcher on the research, and vice versa, addressed?". However, the reviewers decided to include the studies and deemed the methodological quality sufficient for the analysis in this article, based on the other information analyzed. The results of this methodological quality assessment can be found in an additional file (see Additional file 6).

Documents characteristics (Table  1 )

The documents were directed to several continents: Australia/Oceania (4/12) [ 31 , 33 , 36 , 37 ], North America (4/12 [ 30 , 32 , 38 , 39 ], Europe (2/12 [ 29 , 35 ] and Asia (2/12) [ 34 , 40 ]. The types of documents were classified as cross-sectional studies (4/12) [ 29 , 32 , 34 , 38 ], methodological guidelines (4/12) [ 33 , 35 , 36 , 37 ], mixed methods studies (3/12) [ 30 , 31 , 39 ] or noncomparative studies (1/12) [ 40 ]. In terms of the instrument of evaluation, most of the documents used a survey/questionnaire (6/12) [ 29 , 30 , 31 , 32 , 34 , 38 ], while three (3/12) used qualitative instruments (interviews, group discussions) [ 30 , 31 , 39 ], one used a checklist [ 37 ], one used an audit [ 33 ] and three (3/12) did not define a specific instrument to measure [ 35 , 36 , 40 ].

Considering the clinical areas covered, most studies evaluated the implementation of nonspecific (general) clinical areas [ 29 , 33 , 35 , 36 , 37 , 40 ]. However, some studies focused on specific clinical contexts, such as mental health [ 32 , 38 ], oncology [ 39 ], fall prevention [ 31 ], spinal cord injury [ 30 ], and sexually transmitted infections [ 34 ].

Usage context of the models (Table  1 )

Specific objectives.

All the studies highlighted the purpose of guiding the process of evaluating the implementation of CPGs, even if they evaluated CPGs from generic or different clinical areas.

Levels of use

The most common level of use of the models/frameworks identified to assess the implementation of CPGs was policy (6/12) [ 33 , 35 , 36 , 37 , 39 , 40 ]. In this level, the model is used in a systematic way to evaluate all the processes involved in CPGs implementation and is primarily related to methodological guidelines. This was followed by the organizational level of use (5/12) [ 30 , 31 , 32 , 38 , 39 ], where the model is used to evaluate the implementation of CPGs in a specific institution, considering its specific environment. Finally, the clinical level of use (2/12) [ 29 , 34 ] focuses on individual practice and the factors that can influence the implementation of CPGs by professionals.

Type of health service

Institutional services were predominant (5/12) [ 33 , 35 , 36 , 37 , 40 ] and included methodological guidelines and a study of model development and validation. Hospitals were the second most common type of health service (4/12) [ 29 , 30 , 31 , 34 ], followed by ambulatorial (2/12) [ 32 , 34 ] and community health services (1/12) [ 32 ]. Two studies did not specify which type of health service the assessment addressed [ 38 , 39 ].

Target group

The focus of the target group was professionals directly involved in clinical practice (6/12) [ 29 , 31 , 32 , 34 , 38 , 40 ], namely, health professionals and clinicians. Other less related stakeholders included guideline developers (2/12) [ 39 , 40 ], health policy decision makers (1/12) [ 39 ], and healthcare organizations (1/12) [ 39 ]. The target group was not defined in the methodological guidelines, although all the mentioned stakeholders could be related to these documents.

Model and framework characteristics

Models and frameworks for assessing the implementation of cpgs.

The Consolidated Framework for Implementation Research (CFIR) [ 31 , 38 ] and the Promoting Action on Research Implementation in Health Systems (PARiHS) framework [ 29 , 30 ] were the most commonly employed frameworks within the selected documents. The other models mentioned were: Goal commitment and implementation of practice guidelines framework [ 32 ]; Guideline to identify key indicators [ 35 ]; Guideline implementation checklist [ 37 ]; Guideline implementation evaluation tool [ 40 ]; JBI Implementation Framework [ 33 ]; Reach, effectiveness, adoption, implementation and maintenance (RE-AIM) framework [ 34 ]; The Guideline Implementability Framework [ 39 ] and an unnamed model [ 36 ].

Domains evaluated

The number of domains evaluated (or suggested for evaluation) by the documents varied between three and five, with the majority focusing on three domains. All the models addressed the domain "context", with a particular emphasis on the micro level of the health care context (8/12) [ 29 , 31 , 34 , 35 , 36 , 37 , 38 , 39 ], followed by the multilevel (7/12) [ 29 , 31 , 32 , 33 , 38 , 39 , 40 ], meso level (4/12) [ 30 , 35 , 39 , 40 ] and macro level (2/12) [ 37 , 39 ]. The "Outcome" domain was evaluated in nine models. Within this domain, the most frequently evaluated subdomain was "adoption" (6/12) [ 29 , 32 , 34 , 35 , 36 , 37 ], followed by "acceptability" (4/12) [ 30 , 32 , 35 , 39 ], "appropriateness" (3/12) [ 32 , 34 , 36 ], "feasibility" (3/12) [ 29 , 32 , 36 ], "cost" (1/12) [ 35 ] and "penetration" (1/12) [ 34 ]. Regarding the other domains, "Intervention" (8/12) [ 29 , 31 , 34 , 35 , 36 , 38 , 39 , 40 ], "Strategies" (7/12) [ 29 , 30 , 33 , 35 , 36 , 37 , 40 ] and "Process" (5/12) [ 29 , 31 , 32 , 33 , 38 ] were frequently addressed in the models, while "Sustainability" (1/12) [ 34 ] was only found in one model, and "Fidelity/Adaptation" was not observed. The domains presented by the models and frameworks and evaluated in the documents are shown in Table  2 .

Limitations of the models

Only two documents mentioned limitations in the use of the model or frameworks. These two studies reported limitations in the use of CFIR: "is complex and cumbersome and requires tailoring of the key variables to the specific context", and "this framework should be supplemented with other important factors and local features to achieve a sound basis for the planning and realization of an ongoing project" [ 31 , 38 ]. Limitations in the use of other models or frameworks are not reported.

Subgroup analysis

Following the subgroup analysis (Table  3 ), five different models/frameworks were utilized at the policy level by institutional health services. These included the Guideline Implementation Evaluation Tool [ 40 ], the NHMRC tool (model name not defined) [ 36 ], the JBI Implementation Framework + GRiP [ 33 ], Guideline to identify key indicators [ 35 ], and the Guideline implementation checklist [ 37 ]. Additionally, the "Guideline Implementability Framework" [ 39 ] was implemented at the policy level without restrictions based on the type of health service. Regarding the organizational level, the models used varied depending on the type of service. The "Goal commitment and implementation of practice guidelines framework" [ 32 ] was applied in community and ambulatory health services, while "PARiHS" [ 29 , 30 ] and "CFIR" [ 31 , 38 ] were utilized in hospitals. In contexts where the type of health service was not defined, "CFIR" [ 31 , 38 ] and "The Guideline Implementability Framework" [ 39 ] were employed. Lastly, at the clinical level, "RE-AIM" [ 34 ] was utilized in ambulatory and hospital services, and PARiHS [ 29 , 30 ] was specifically used in hospital services.

Key findings

This systematic review identified 10 models/ frameworks used to assess the implementation of CPGs in various health system contexts. These documents shared similar objectives in utilizing models and frameworks for assessment. The primary level of use was policy, the most common type of health service was institutional, and the main target group of the documents was professionals directly involved in clinical practice. The models and frameworks presented varied analytical domains, with sometimes divergent concepts used in these domains. This study is innovative in its emphasis on the evaluation stage of CPG implementation and in summarizing aspects and domains aimed at the practical application of these models.

The small number of documents contrasts with studies that present an extensive range of models and frameworks available in implementation science. The findings suggest that the use of models and frameworks to evaluate the implementation of CPGs is still in its early stages. Among the selected documents, there was a predominance of cross-sectional studies and methodological guidelines, which strongly influenced how the implementation evaluation was conducted. This was primarily done through surveys/questionnaires, qualitative methods (interviews, group discussions), and non-specific measurement instruments. Regarding the subject areas evaluated, most studies focused on a general clinical area, while others explored different clinical areas. This suggests that the evaluation of CPG implementation has been carried out in various contexts.

The models were chosen independently of the categories proposed in the literature, with their usage categorized for purposes other than implementation evaluation, as is the case with CFIR and PARiHS. This practice was described by Nilsen et al. who suggested that models and frameworks from other categories can also be applied for evaluation purposes because they specify concepts and constructs that may be operationalized and measured [ 14 , 15 , 42 , 43 ].

The results highlight the increased use of models and frameworks in evaluation processes at the policy level and institutional environments, followed by the organizational level in hospital settings. This finding contradicts a review that reported the policy level as an area that was not as well studied [ 44 ]. The use of different models at the institutional level is also emphasized in the subgroup analysis. This may suggest that the greater the impact (social, financial/economic, and organizational) of implementing CPGs, the greater the interest and need to establish well-defined and robust processes. In this context, the evaluation stage stands out as crucial, and the investment of resources and efforts to structure this stage becomes even more advantageous [ 10 , 45 ]. Two studies (16,7%) evaluated the implementation of CPGs at the individual level (clinical level). These studies stand out for their potential to analyze variations in clinical practice in greater depth.

In contrast to the level of use and type of health service most strongly indicated in the documents, with systemic approaches, the target group most observed was professionals directly involved in clinical practice. This suggests an emphasis on evaluating individual behaviors. This same emphasis is observed in the analysis of the models, in which there is a predominance of evaluating the micro level of the health context and the "adoption" subdomain, in contrast with the sub-use of domains such as "cost" and "process". Cassetti et al. observed the same phenomenon in their review, in which studies evaluating the implementation of CPGs mainly adopted a behavioral change approach to tackle those issues, without considering the influence of wider social determinants of health [ 10 ]. However, the literature widely reiterates that multiple factors impact the implementation of CPGs, and different actions are required to make them effective [ 6 , 46 , 47 ]. As a result, there is enormous potential for the development and adaptation of models and frameworks aimed at more systemic evaluation processes that consider institutional and organizational aspects.

In analyzing the model domains, most models focused on evaluating only some aspects of implementation (three domains). All models evaluated the "context", highlighting its significant influence on implementation [ 9 , 26 ]. Context is an essential effect modifier for providing research evidence to guide decisions on implementation strategies [ 48 ]. Contextualizing a guideline involves integrating research or other evidence into a specific circumstance [ 49 ]. The analysis of this domain was adjusted to include all possible contextual aspects, even if they were initially allocated to other domains. Some contextual aspects presented by the models vary in comprehensiveness, such as the assessment of the "timing and nature of stakeholder engagement" [ 39 ], which includes individual engagement by healthcare professionals and organizational involvement in CPG implementation. While the importance of context is universally recognized, its conceptualization and interpretation differ across studies and models. This divergence is also evident in other domains, consistent with existing literature [ 14 ]. Efforts to address this conceptual divergence in implementation science are ongoing, but further research and development are needed in this field [ 26 ].

The main subdomain evaluated was "adoption" within the outcome domain. This may be attributed to the ease of accessing information on the adoption of the CPG, whether through computerized system records, patient records, or self-reports from healthcare professionals or patients themselves. The "acceptability" subdomain pertains to the perception among implementation stakeholders that a particular CPG is agreeable, palatable or satisfactory. On the other hand, "appropriateness" encompasses the perceived fit, relevance or compatibility of the CPG for a specific practice setting, provider, or consumer, or its perceived fit to address a particular issue or problem [ 26 ]. Both subdomains are subjective and rely on stakeholders' interpretations and perceptions of the issue being analyzed, making them susceptible to reporting biases. Moreover, obtaining this information requires direct consultation with stakeholders, which can be challenging for some evaluation processes, particularly in institutional contexts.

The evaluation of the subdomains "feasibility" (the extent to which a CPG can be successfully used or carried out within a given agency or setting), "cost" (the cost impact of an implementation effort), and "penetration" (the extent to which an intervention or treatment is integrated within a service setting and its subsystems) [ 26 ] was rarely observed in the documents. This may be related to the greater complexity of obtaining information on these aspects, as they involve cross-cutting and multifactorial issues. In other words, it would be difficult to gather this information during evaluations with health practitioners as the target group. This highlights the need for evaluation processes of CPGs implementation involving multiple stakeholders, even if the evaluation is adjusted for each of these groups.

Although the models do not establish the "intervention" domain, we thought it pertinent in this study to delimit the issues that are intrinsic to CPGs, such as methodological quality or clarity in establishing recommendations. These issues were quite common in the models evaluated but were considered in other domains (e.g., in "context"). Studies have reported the importance of evaluating these issues intrinsic to CPGs [ 47 , 50 ] and their influence on the implementation process [ 51 ].

The models explicitly present the "strategies" domain, and its evaluation was usually included in the assessments. This is likely due to the expansion of scientific and practical studies in implementation science that involve theoretical approaches to the development and application of interventions to improve the implementation of evidence-based practices. However, these interventions themselves are not guaranteed to be effective, as reported in a previous review that showed unclear results indicating that the strategies had affected successful implementation [ 52 ]. Furthermore, model domains end up not covering all the complexity surrounding the strategies and their development and implementation process. For example, the ‘Guideline implementation evaluation tool’ evaluates whether guideline developers have designed and provided auxiliary tools to promote the implementation of guidelines [ 40 ], but this does not mean that these tools would work as expected.

The "process" domain was identified in the CFIR [ 31 , 38 ], JBI/GRiP [ 33 ], and PARiHS [ 29 ] frameworks. While it may be included in other domains of analysis, its distinct separation is crucial for defining operational issues when assessing the implementation process, such as determining if and how the use of the mentioned CPG was evaluated [ 3 ]. Despite its presence in multiple models, there is still limited detail in the evaluation guidelines, which makes it difficult to operationalize the concept. Further research is needed to better define the "process" domain and its connections and boundaries with other domains.

The domain of "sustainability" was only observed in the RE-AIM framework, which is categorized as an evaluation framework [ 34 ]. In its acronym, the letter M stands for "maintenance" and corresponds to the assessment of whether the user maintains use, typically longer than 6 months. The presence of this domain highlights the need for continuous evaluation of CPGs implementation in the short, medium, and long term. Although the RE-AIM framework includes this domain, it was not used in the questionnaire developed in the study. One probable reason is that the evaluation of CPGs implementation is still conducted on a one-off basis and not as a continuous improvement process. Considering that changes in clinical practices are inherent over time, evaluating and monitoring changes throughout the duration of the CPG could be an important strategy for ensuring its implementation. This is an emerging field that requires additional investment and research.

The "Fidelity/Adaptation" domain was not observed in the models. These emerging concepts involve the extent to which a CPG is being conducted exactly as planned or whether it is undergoing adjustments and adaptations. Whether or not there is fidelity or adaptation in the implementation of CPGs does not presuppose greater or lesser effectiveness; after all, some adaptations may be necessary to implement general CPGs in specific contexts. The absence of this domain in all the models and frameworks may suggest that they are not relevant aspects for evaluating implementation or that there is a lack of knowledge of these complex concepts. This may suggest difficulty in expressing concepts in specific evaluative questions. However, further studies are warranted to determine the comprehensiveness of these concepts.

It is important to note the customization of the domains of analysis, with some domains presented in the models not being evaluated in the studies, while others were complementarily included. This can be seen in Jeong et al. [ 34 ], where the "intervention" domain in the evaluation with the RE-AIM framework reinforced the aim of theoretical approaches such as guiding the process and not determining norms. Despite this, few limitations were reported for the models, suggesting that the use of models in these studies reflects the application of these models to defined contexts without a deep critical analysis of their domains.

Limitations

This review has several limitations. First, only a few studies and methodological guidelines that explicitly present models and frameworks for assessing the implementation of CPGs have been found. This means that few alternative models could be analyzed and presented in this review. Second, this review adopted multiple analytical categories (e.g., level of use, health service, target group, and domains evaluated), whose terminology has varied enormously in the studies and documents selected, especially for the "domains evaluated" category. This difficulty in harmonizing the taxonomy used in the area has already been reported [ 26 ] and has significant potential to confuse. For this reason, studies and initiatives are needed to align understandings between concepts and, as far as possible, standardize them. Third, in some studies/documents, the information extracted was not clear about the analytical category. This required an in-depth interpretative process of the studies, which was conducted in pairs to avoid inappropriate interpretations.

Implications

This study contributes to the literature and clinical practice management by describing models and frameworks specifically used to assess the implementation of CPGs based on their level of use, type of health service, target group related to the CPG, and the evaluated domains. While there are existing reviews on the theories, frameworks, and models used in implementation science, this review addresses aspects not previously covered in the literature. This valuable information can assist stakeholders (such as politicians, clinicians, researchers, etc.) in selecting or adapting the most appropriate model to assess CPG implementation based on their health context. Furthermore, this study is expected to guide future research on developing or adapting models to assess the implementation of CPGs in various contexts.

The use of models and frameworks to evaluate the implementation remains a challenge. Studies should clearly state the level of model use, the type of health service evaluated, and the target group. The domains evaluated in these models may need adaptation to specific contexts. Nevertheless, utilizing models to assess CPGs implementation is crucial as they can guide a more thorough and systematic evaluation process, aiding in the continuous improvement of CPGs implementation. The findings of this systematic review offer valuable insights for stakeholders in selecting or adjusting models and frameworks for CPGs evaluation, supporting future theoretical advancements and research.

Availability of data and materials

Abbreviations.

Australian Department of Health and Aged Care

Canadian Agency for Drugs and Technologies in Health

Centers for Disease Control and

Consolidated Framework for Implementation Research

Cumulative Index to Nursing and Allied Health Literature

Clinical practice guideline

Centre for Reviews and Dissemination

Guidelines International Networks

Getting Research into Practice

Health Systems Evidence

Institute of Medicine

The Joanna Briggs Institute

Ministry of Health of Brazil

Ministerio de Sanidad y Política Social

National Health and Medical Research Council

National Institute for Health and Care Excellence

Promoting action on research implementation in health systems framework

Predisposing, Reinforcing and Enabling Constructs in Educational Diagnosis and Evaluation-Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

International Prospective Register of Systematic Reviews

Reach, effectiveness, adoption, implementation, and maintenance framework

Healthcare Improvement Scotland

United States of America

Virtual Health Library

World Health Organization

Medicine I of. Crossing the Quality Chasm: A New Health System for the 21st Century. 2001. Available from: http://www.nap.edu/catalog/10027 . Cited 2022 Sep 29.

Field MJ, Lohr KN. Clinical Practice Guidelines: Directions for a New Program. Washington DC: National Academy Press. 1990. Available from: https://www.nap.edu/read/1626/chapter/8 Cited 2020 Sep 2.

Dawson A, Henriksen B, Cortvriend P. Guideline Implementation in Standardized Office Workflows and Exam Types. J Prim Care Community Heal. 2019;10. Available from: https://pubmed.ncbi.nlm.nih.gov/30900500/ . Cited 2020 Jul 15.

Unverzagt S, Oemler M, Braun K, Klement A. Strategies for guideline implementation in primary care focusing on patients with cardiovascular disease: a systematic review. Fam Pract. 2014;31(3):247–66. Available from: https://academic.oup.com/fampra/article/31/3/247/608680 . Cited 2020 Nov 5.

Article   PubMed   Google Scholar  

Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10(1):1–13. Available from: https://implementationscience.biomedcentral.com/articles/10.1186/s13012-015-0242-0 . Cited 2022 May 1.

Article   Google Scholar  

Mangana F, Massaquoi LD, Moudachirou R, Harrison R, Kaluangila T, Mucinya G, et al. Impact of the implementation of new guidelines on the management of patients with HIV infection at an advanced HIV clinic in Kinshasa, Democratic Republic of Congo (DRC). BMC Infect Dis. 2020;20(1):N.PAG-N.PAG. Available from: https://search.ebscohost.com/login.aspx?direct=true&db=c8h&AN=146325052&amp .

Browman GP, Levine MN, Mohide EA, Hayward RSA, Pritchard KI, Gafni A, et al. The practice guidelines development cycle: a conceptual tool for practice guidelines development and implementation. 2016;13(2):502–12. https://doi.org/10.1200/JCO.1995.13.2.502 .

Killeen SL, Donnellan N, O’Reilly SL, Hanson MA, Rosser ML, Medina VP, et al. Using FIGO Nutrition Checklist counselling in pregnancy: A review to support healthcare professionals. Int J Gynecol Obstet. 2023;160(S1):10–21. Available from: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85146194829&doi=10.1002%2Fijgo.14539&partnerID=40&md5=d0f14e1f6d77d53e719986e6f434498f .

Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne AM. An introduction to implementation science for the non-specialist. BMC Psychol. 2015;3(1):1–12. Available from: https://bmcpsychology.biomedcentral.com/articles/10.1186/s40359-015-0089-9 . Cited 2020 Nov 5.

Cassetti V, M VLR, Pola-Garcia M, AM G, J JPC, L APDT, et al. An integrative review of the implementation of public health guidelines. Prev Med reports. 2022;29:101867. Available from: http://www.epistemonikos.org/documents/7ad499d8f0eecb964fc1e2c86b11450cbe792a39 .

Eccles MP, Mittman BS. Welcome to implementation science. Implementation Science BioMed Central. 2006. Available from: https://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-1-1 .

Damschroder LJ. Clarity out of chaos: Use of theory in implementation research. Psychiatry Res. 2020;1(283):112461.

Handley MA, Gorukanti A, Cattamanchi A. Strategies for implementing implementation science: a methodological overview. Emerg Med J. 2016;33(9):660–4. Available from: https://pubmed.ncbi.nlm.nih.gov/26893401/ . Cited 2022 Mar 7.

Wang Y, Wong ELY, Nilsen P, Chung VC ho, Tian Y, Yeoh EK. A scoping review of implementation science theories, models, and frameworks — an appraisal of purpose, characteristics, usability, applicability, and testability. Implement Sci. 2023;18(1):1–15. Available from: https://implementationscience.biomedcentral.com/articles/10.1186/s13012-023-01296-x . Cited 2024 Jan 22.

Moullin JC, Dickson KS, Stadnick NA, Albers B, Nilsen P, Broder-Fingert S, et al. Ten recommendations for using implementation frameworks in research and practice. Implement Sci Commun. 2020;1(1):1–12. Available from: https://implementationsciencecomms.biomedcentral.com/articles/10.1186/s43058-020-00023-7 . Cited 2022 May 20.

Glasgow RE, Vogt TM, Boles SM. *Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322. Available from: /pmc/articles/PMC1508772/?report=abstract. Cited 2022 May 22.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Asada Y, Lin S, Siegel L, Kong A. Facilitators and Barriers to Implementation and Sustainability of Nutrition and Physical Activity Interventions in Early Childcare Settings: a Systematic Review. Prev Sci. 2023;24(1):64–83. Available from: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85139519721&doi=10.1007%2Fs11121-022-01436-7&partnerID=40&md5=b3c395fdd2b8235182eee518542ebf2b .

Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions. version 6. Cochrane; 2022. Available from: https://training.cochrane.org/handbook. Cited 2022 May 23.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372. Available from: https://www.bmj.com/content/372/bmj.n71 . Cited 2021 Nov 18.

M C, AD O, E P, JP H, S G. Appendix A: Guide to the contents of a Cochrane Methodology protocol and review. Higgins JP, Green S, eds Cochrane Handb Syst Rev Interv. 2011;Version 5.

Kislov R, Pope C, Martin GP, Wilson PM. Harnessing the power of theorising in implementation science. Implement Sci. 2019;14(1):1–8. Available from: https://implementationscience.biomedcentral.com/articles/10.1186/s13012-019-0957-4 . Cited 2024 Jan 22.

Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev. 2016;5(1):1–10. Available from: https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-016-0384-4 . Cited 2022 May 20.

JBI. JBI’s Tools Assess Trust, Relevance & Results of Published Papers: Enhancing Evidence Synthesis. Available from: https://jbi.global/critical-appraisal-tools . Cited 2023 Jun 13.

Drisko JW. Qualitative research synthesis: An appreciative and critical introduction. Qual Soc Work. 2020;19(4):736–53.

Pope C, Mays N, Popay J. Synthesising qualitative and quantitative health evidence: A guide to methods. 2007. Available from: https://books.google.com.br/books?hl=pt-PT&lr=&id=L3fbE6oio8kC&oi=fnd&pg=PR6&dq=synthesizing+qualitative+and+quantitative+health+evidence&ots=sfELNUoZGq&sig=bQt5wt7sPKkf7hwKUvxq2Ek-p2Q#v=onepage&q=synthesizing=qualitative=and=quantitative=health=evidence& . Cited 2022 May 22.

Nilsen P, Birken SA, Edward Elgar Publishing. Handbook on implementation science. 542. Available from: https://www.e-elgar.com/shop/gbp/handbook-on-implementation-science-9781788975988.html . Cited 2023 Apr 15.

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):1–15. Available from: https://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-4-50 . Cited 2023 Jun 13.

Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76. Available from: https://pubmed.ncbi.nlm.nih.gov/20957426/ . Cited 2023 Jun 11.

Bahtsevani C, Willman A, Khalaf A, Östman M, Ostman M. Developing an instrument for evaluating implementation of clinical practice guidelines: a test-retest study. J Eval Clin Pract. 2008;14(5):839–46. Available from: https://search.ebscohost.com/login.aspx?direct=true&db=c8h&AN=105569473&amp . Cited 2023 Jan 18.

Balbale SN, Hill JN, Guihan M, Hogan TP, Cameron KA, Goldstein B, et al. Evaluating implementation of methicillin-resistant Staphylococcus aureus (MRSA) prevention guidelines in spinal cord injury centers using the PARIHS framework: a mixed methods study. Implement Sci. 2015;10(1):130. Available from: https://pubmed.ncbi.nlm.nih.gov/26353798/ . Cited 2023 Apr 3.

Article   PubMed   PubMed Central   Google Scholar  

Breimaier HE, Heckemann B, Halfens RJGG, Lohrmann C. The Consolidated Framework for Implementation Research (CFIR): a useful theoretical framework for guiding and evaluating a guideline implementation process in a hospital-based nursing practice. BMC Nurs. 2015;14(1):43. Available from: https://search.ebscohost.com/login.aspx?direct=true&db=c8h&AN=109221169&amp . Cited 2023 Apr 3.

Chou AF, Vaughn TE, McCoy KD, Doebbeling BN. Implementation of evidence-based practices: Applying a goal commitment framework. Health Care Manage Rev. 2011;36(1):4–17. Available from: https://pubmed.ncbi.nlm.nih.gov/21157225/ . Cited 2023 Apr 30.

Porritt K, McArthur A, Lockwood C, Munn Z. JBI Manual for Evidence Implementation. JBI Handbook for Evidence Implementation. JBI; 2020. Available from: https://jbi-global-wiki.refined.site/space/JHEI . Cited 2023 Apr 3.

Jeong HJJ, Jo HSS, Oh MKK, Oh HWW. Applying the RE-AIM Framework to Evaluate the Dissemination and Implementation of Clinical Practice Guidelines for Sexually Transmitted Infections. J Korean Med Sci. 2015;30(7):847–52. Available from: https://pubmed.ncbi.nlm.nih.gov/26130944/ . Cited 2023 Apr 3.

GPC G de trabajo sobre implementación de. Implementación de Guías de Práctica Clínica en el Sistema Nacional de Salud. Manual Metodológico. 2009. Available from: https://portal.guiasalud.es/wp-content/uploads/2019/01/manual_implementacion.pdf . Cited 2023 Apr 3.

Australia C of. A guide to the development, implementation and evaluation of clinical practice guidelines. National Health and Medical Research Council; 1998. Available from: https://www.health.qld.gov.au/__data/assets/pdf_file/0029/143696/nhmrc_clinprgde.pdf .

Health Q. Guideline implementation checklist Translating evidence into best clinical practice. 2022.

Google Scholar  

Quittner AL, Abbott J, Hussain S, Ong T, Uluer A, Hempstead S, et al. Integration of mental health screening and treatment into cystic fibrosis clinics: Evaluation of initial implementation in 84 programs across the United States. Pediatr Pulmonol. 2020;55(11):2995–3004. Available from: https://www.embase.com/search/results?subaction=viewrecord&id=L2005630887&from=export . Cited 2023 Apr 3.

Urquhart R, Woodside H, Kendell C, Porter GA. Examining the implementation of clinical practice guidelines for the management of adult cancers: A mixed methods study. J Eval Clin Pract. 2019;25(4):656–63. Available from: https://search.ebscohost.com/login.aspx?direct=true&db=c8h&AN=137375535&amp . Cited 2023 Apr 3.

Yinghui J, Zhihui Z, Canran H, Flute Y, Yunyun W, Siyu Y, et al. Development and validation for evaluation of an evaluation tool for guideline implementation. Chinese J Evidence-Based Med. 2022;22(1):111–9. Available from: https://www.embase.com/search/results?subaction=viewrecord&id=L2016924877&from=export .

Breimaier HE, Halfens RJG, Lohrmann C. Effectiveness of multifaceted and tailored strategies to implement a fall-prevention guideline into acute care nursing practice: a before-and-after, mixed-method study using a participatory action research approach. BMC Nurs. 2015;14(1):18. Available from: https://search.ebscohost.com/login.aspx?direct=true&db=c8h&AN=103220991&amp .

Lai J, Maher L, Li C, Zhou C, Alelayan H, Fu J, et al. Translation and cross-cultural adaptation of the National Health Service Sustainability Model to the Chinese healthcare context. BMC Nurs. 2023;22(1). Available from: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85153237164&doi=10.1186%2Fs12912-023-01293-x&partnerID=40&md5=0857c3163d25ce85e01363fc3a668654 .

Zhao J, Li X, Yan L, Yu Y, Hu J, Li SA, et al. The use of theories, frameworks, or models in knowledge translation studies in healthcare settings in China: a scoping review protocol. Syst Rev. 2021;10(1):13. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7792291 .

Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging research and practice: models for dissemination and implementation research. Am J Prev Med. 2012;43(3):337–50. Available from: https://pubmed.ncbi.nlm.nih.gov/22898128/ . Cited 2023 Apr 4.

Phulkerd S, Lawrence M, Vandevijvere S, Sacks G, Worsley A, Tangcharoensathien V. A review of methods and tools to assess the implementation of government policies to create healthy food environments for preventing obesity and diet-related non-communicable diseases. Implement Sci. 2016;11(1):1–13. Available from: https://implementationscience.biomedcentral.com/articles/10.1186/s13012-016-0379-5 . Cited 2022 May 1.

Buss PM, Pellegrini FA. A Saúde e seus Determinantes Sociais. PHYSIS Rev Saúde Coletiva. 2007;17(1):77–93.

Pereira VC, Silva SN, Carvalho VKSS, Zanghelini F, Barreto JOMM. Strategies for the implementation of clinical practice guidelines in public health: an overview of systematic reviews. Heal Res Policy Syst. 2022;20(1):13. Available from: https://health-policy-systems.biomedcentral.com/articles/10.1186/s12961-022-00815-4 . Cited 2022 Feb 21.

Grimshaw J, Eccles M, Tetroe J. Implementing clinical guidelines: current evidence and future implications. J Contin Educ Health Prof. 2004;24 Suppl 1:S31-7. Available from: https://pubmed.ncbi.nlm.nih.gov/15712775/ . Cited 2021 Nov 9.

Lotfi T, Stevens A, Akl EA, Falavigna M, Kredo T, Mathew JL, et al. Getting trustworthy guidelines into the hands of decision-makers and supporting their consideration of contextual factors for implementation globally: recommendation mapping of COVID-19 guidelines. J Clin Epidemiol. 2021;135:182–6. Available from: https://pubmed.ncbi.nlm.nih.gov/33836255/ . Cited 2024 Jan 25.

Lenzer J. Why we can’t trust clinical guidelines. BMJ. 2013;346(7913). Available from: https://pubmed.ncbi.nlm.nih.gov/23771225/ . Cited 2024 Jan 25.

Molino C de GRC, Ribeiro E, Romano-Lieber NS, Stein AT, de Melo DO. Methodological quality and transparency of clinical practice guidelines for the pharmacological treatment of non-communicable diseases using the AGREE II instrument: A systematic review protocol. Syst Rev. 2017;6(1):1–6. Available from: https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-017-0621-5 . Cited 2024 Jan 25.

Albers B, Mildon R, Lyon AR, Shlonsky A. Implementation frameworks in child, youth and family services – Results from a scoping review. Child Youth Serv Rev. 2017;1(81):101–16.

Download references

Acknowledgements

Not applicable

This study is supported by the Fundação de Apoio à Pesquisa do Distrito Federal (FAPDF). FAPDF Award Term (TOA) nº 44/2024—FAPDF/SUCTI/COOBE (SEI/GDF – Process 00193–00000404/2024–22). The content in this article is solely the responsibility of the authors and does not necessarily represent the official views of the FAPDF.

Author information

Authors and affiliations.

Department of Management and Incorporation of Health Technologies, Ministry of Health of Brazil, Brasília, Federal District, 70058-900, Brazil

Nicole Freitas de Mello & Dalila Fernandes Gomes

Postgraduate Program in Public Health, FS, University of Brasília (UnB), Brasília, Federal District, 70910-900, Brazil

Nicole Freitas de Mello, Dalila Fernandes Gomes & Jorge Otávio Maia Barreto

René Rachou Institute, Oswaldo Cruz Foundation, Belo Horizonte, Minas Gerais, 30190-002, Brazil

Sarah Nascimento Silva

Oswaldo Cruz Foundation - Brasília, Brasília, Federal District, 70904-130, Brazil

Juliana da Motta Girardi & Jorge Otávio Maia Barreto

You can also search for this author in PubMed   Google Scholar

Contributions

NFM and JOMB conceived the idea and the protocol for this study. NFM conducted the literature search. NFM, SNS, JMG and JOMB conducted the data collection with advice and consensus gathering from JOMB. The NFM and JMG assessed the quality of the studies. NFM and DFG conducted the data extraction. NFM performed the analysis and synthesis of the results with advice and consensus gathering from JOMB. NFM drafted the manuscript. JOMB critically revised the first version of the manuscript. All the authors revised and approved the submitted version.

Corresponding author

Correspondence to Nicole Freitas de Mello .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

13012_2024_1389_moesm1_esm.docx.

Additional file 1: PRISMA checklist. Description of data: Completed PRISMA checklist used for reporting the results of this systematic review.

Additional file 2: Literature search. Description of data: The search strategies adapted for the electronic databases.

13012_2024_1389_moesm3_esm.doc.

Additional file 3: JBI’s critical appraisal tools for cross-sectional studies. Description of data: JBI’s critical appraisal tools to assess the trustworthiness, relevance, and results of the included studies. This is specific for cross-sectional studies.

13012_2024_1389_MOESM4_ESM.doc

Additional file 4: JBI’s critical appraisal tools for qualitative studies. Description of data: JBI’s critical appraisal tools to assess the trustworthiness, relevance, and results of the included studies. This is specific for qualitative studies.

13012_2024_1389_MOESM5_ESM.doc

Additional file 5: Methodological quality assessment results for cross-sectional studies. Description of data: Methodological quality assessment results for cross-sectional studies using JBI’s critical appraisal tools.

13012_2024_1389_MOESM6_ESM.doc

Additional file 6: Methodological quality assessment results for the qualitative studies. Description of data: Methodological quality assessment results for qualitative studies using JBI’s critical appraisal tools.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Freitas de Mello, N., Nascimento Silva, S., Gomes, D.F. et al. Models and frameworks for assessing the implementation of clinical practice guidelines: a systematic review. Implementation Sci 19 , 59 (2024). https://doi.org/10.1186/s13012-024-01389-1

Download citation

Received : 06 February 2024

Accepted : 01 August 2024

Published : 07 August 2024

DOI : https://doi.org/10.1186/s13012-024-01389-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Implementation
• Practice guideline
• Evidence-Based Practice
• Implementation science

Implementation Science

ISSN: 1748-5908

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Data, analysis, convening and action.

The world’s largest and most diverse environmental network.

• IUCN WORLD CONSERVATION CONGRESS
• REGIONAL CONSERVATION FORA
• CONTRIBUTIONS FOR NATURE
• IUCN ENGAGE (LOGIN REQUIRED)

IUCN tools, publications and other resources.

• News & Events
• Eastern and Southern Africa
• Eastern Europe and Central Asia
• Mediterranean
• Mexico, Central America and the Caribbean
• North America
• South America
• West and Central Africa
• IUCN Academy
• IUCN Contributions for Nature
• IUCN Library
• IUCN Red List of Threatened Species TM
• IUCN Green List of Protected and Conserved Areas
• IUCN World Heritage Outlook
• IUCN Leaders Forum
• Protected Planet
• Union Portal (login required)
• IUCN Engage (login required)
• Commission portal (login required)

Get Involved

Our resources share the knowledge gathered by IUCN’s unique global community of 16,000+ experts. They include databases, tools, standards, guidelines and policy recommendations. We author hundreds of books, assessments, reports, briefs and research papers every year.

IUCN Issues Briefs provide key information on selected issues central to IUCN’s work. They are aimed at policy-makers, journalists or anyone looking for an accessible overview of the often complex issues related to nature conservation and sustainable development.

Conservation tools

IUCN's conservation tools consist of conservation databases, metrics and other knowledge products. These products have helped hundreds of organisations design, monitor and implement just and effective conservation.

IUCN produces publications on a wide range of topics to share our expertise on nature, conservation and sustainable development. Publications include reports, analyses, best practices, standards, periodicals from IUCN Commissions and numerous other types of knowledge from the Union.

Search all resources

• Sunday 28 July 28 2024, 11:00 – 16:30 (TBC)
• Courtyard Marriot (Frangipani Room)