Critical evaluation of the Manual "Evaluation and categorization of obstetric risk in prenatal care" of the MSP using the AGREE II Instrument
DOI:
https://doi.org/10.16921/pfr.v10i3.394Keywords:
obstetric risk; prenatal care; clinical practice guidelines; AGREE II; methodological quality; EcuadorAbstract
Introduction: Stratification of obstetric risk during prenatal care is a critical strategy to identify women at a higher likelihood of adverse outcomes and to optimize timely referral and intervention. In Ecuador, the Ministry of Public Health implemented a national manual in 2019 to standardize this process; however, its methodological quality and operational applicability have not been formally evaluated.
Objective: To assess the methodological quality, structural design, and operational applicability of the MSP manual titled “Evaluation and categorization of obstetric risk in prenatal care”, using the AGREE II instrument.
Methods: A descriptive and evaluative study was conducted applying the AGREE II instrument, which includes 23 items across six domains. Three independent experts in public health and epidemiology appraised the manual, and a consensus process was used to resolve discrepancies. Domain scores were standardized on a 0-100 scale, and inter-rater reliability was calculated using the intraclass correlation coefficient under a random-effects model. In addition, a content-mapping exercise was performed to characterize the manual’s key technical recommendations by thematic area, clinical-operational objective, and level of action.
Results: The manual achieved its highest ratings in Clarity of Presentation (94.4) and Scope and Purpose (79.6), indicating strong communicative structure and defined objectives. Conversely, Rigor of Development (31.9) and Editorial Independence (36.1) scored significantly lower, reflecting significant methodological and governance limitations in evidence search and transparency of conflict of interest. Overall inter-rater reliability was moderate-to-high (ICC = 0.686), with excellent agreement observed in the domains of Applicability and Stakeholder Involvement (ICC > 0.9). Content-mapping identified six operational domains: initial risk assessment, risk categorization, prenatal follow-up, referral and counter-referral, complementary interventions, and monitoring.
Conclusions: While the MSP manual demonstrates notable strengths in clarity and operational design, it reveals critical deficits in methodological foundation and editorial transparency. These findings underscore the need to adopt international guideline-development frameworks, such as GRADE and ADAPTE, to enhance the manual’s scientific validity, traceability, and contextual relevance. A methodological revision and update of the manual could improve its effectiveness and significantly reduce maternal morbidity and mortality in Ecuador.
Downloads
References
Tunçalp Ӧ., Pena‐Rosas J, Lawrie T, Bucagu M, Oladapo O, Portela A, et al. WHO recommendations on antenatal care for a positive pregnancy experience—going beyond survival. BJOG [Internet]. 2017 May 9;124(6):860–2. Available from: https://obgyn.onlinelibrary.wiley.com/doi/10.1111/1471-0528.14599
World Health Organization (Institution). WHO recommendations on antenatal care for a positive pregnancy experience [Internet]. Geneva: World Health Organization; 2016. Available from: https://www.who.int/publications/i/item/9789241549912
van Montfort P, Scheepers HCJ, Dirksen CD, van Dooren IMA, van Kuijk SMJ, Meertens LJE, et al. Impact on perinatal health and cost-effectiveness of risk-based care in obstetrics: a before-after study. Am J Obstet Gynecol. 2020 Sep;223(3):431.e1-431.e18.
Pallepogula DR, Bethou A, Ballambatu VB, Dorairajan G, Saya GK, Kamalakannan S, et al. A Systematic Review of Antenatal Risk Scoring Systems in India to Predict Adverse Neonatal Outcomes. The Journal of Obstetrics and Gynecology of India. 2022 Jun 12;72(3):181–91.
Albarqi MN. The Impact of Prenatal Care on the Prevention of Neonatal Outcomes: A Systematic Review and Meta-Analysis of Global Health Interventions. Healthcare. 2025 May 6;13(9):1076.
Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. Development of the AGREE II, part 1: performance, usefulness and areas for improvement. Can Med Assoc J. 2010 Jul 13;182(10):1045–52.
Hoffmann-Eßer W, Siering U, Neugebauer EAM, Brockhaus AC, Lampert U, Eikermann M. Guideline appraisal with AGREE II: Systematic review of the current evidence on how users handle the 2 overall assessments. PLoS One. 2017 Mar 30;12(3):e0174831.
Ministerio de Salud Pública del Ecuador. Evaluación y categorización del riesgo obstétrico en el control prenatal [Internet]. Manual. Quito: Dirección Nacional de Normatización, Gerencia Institucional de Estrategias y Articulación de la Salud Materna y Neonatal; 2019. Available from: https://www.salud.gob.ec
Hatakeyama Y, Seto K, Amin R, Kitazawa T, Fujita S, Matsumoto K, et al. The structure of the quality of clinical practice guidelines with the items and overall assessment in AGREE II: a regression analysis. BMC Health Serv Res [Internet]. 2019 Dec 4;19(1):788. Available from: https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-019-4532-0
Hoffmann-Eßer W, Siering U, Neugebauer EAM, Brockhaus AC, McGauran N, Eikermann M. Guideline appraisal with AGREE II: online survey of the potential influence of AGREE II items on overall assessment of guideline quality and recommendation for use. BMC Health Serv Res [Internet]. 2018 Dec 27;18(1):143. Available from: https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-018-2954-8
Polus S, Lerberg P, Vogel J, Watananirun K, Souza JP, Mathai M, et al. Appraisal of WHO Guidelines in Maternal Health Using the AGREE II Assessment Tool. PLoS One. 2012 Aug 13;7(8):e38891.
Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. Can Med Assoc J [Internet]. 2010 Dec 14;182(18):E839–42. Available from: http://www.cmaj.ca/cgi/doi/10.1503/cmaj.090449
Carande-Kulis V, Elder RW, Koffman DM. Standards Required for the Development of CDC Evidence-Based Guidelines. MMWR Suppl. 2022 Jan 14;71(1):1–6.
Institute of Medicine. Clinical Practice Guidelines We Can Trust. Graham R, Mancher M, Wolman DM, Greenfield S, Steinberg E, editors. Washington, D.C.: National Academies Press; 2011.
Belizan M, Meier A, Althabe F, Codazzi A, Colomar M, Buekens P, et al. Facilitators and barriers to adoption of evidence-based perinatal care in Latin American hospitals: a qualitative study. Health Educ Res. 2006 Aug 8;22(6):839–53.
Rodríguez Quintana T, Dávalos-Batallas V, Vargas-Martínez AM, López L, Bonilla-Sierra P, Lomas-Campos M de las M, et al. Implementation and Knowledge of the Clinical Practice Guide for Palliative Care in the Ecuadorian Primary Care Level. Int J Environ Res Public Health. 2021 Nov 4;18(21):11573.
Wang T, Tan JY (Benjamin), Liu XL, Zhao I. Barriers and enablers to implementing clinical practice guidelines in primary care: an overview of systematic reviews. BMJ Open. 2023 Jan 6;13(1):e062158.
(INEC) IN de E y C. Principales Resultados del Registro Estadístico de Defunciones Generales 2024 [Internet]. Quito: Instituto Nacional de Estadística y Censos (INEC); 2025. Available from: https://www.ecuadorencifras.gob.ec/defunciones-generales/
(INEC) IN de E y C, Olivo V, Lugmaña G, Encalada E, Muñoz J. Boletín Técnico: Registro Estadístico de Defunciones Generales. Año 2023 [Internet]. Quito: Instituto Nacional de Estadística y Censos (INEC); 2024. (Población y Migración). Available from: https://www.ecuadorencifras.gob.ec/defunciones-generales/
Schünemann HJ, Wiercioch W, Etxeandia I, Falavigna M, Santesso N, Mustafa R, et al. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. Can Med Assoc J. 2014 Feb 18;186(3):E123–42.
Beauchemin M, Cohn E, Shelton RC. Implementation of Clinical Practice Guidelines in the Health Care Setting. Advances in Nursing Science. 2019 Oct;42(4):307–24.
Hurtado MM, Quemada C, García‐Herrera JM, Morales‐Asencio JM. Use of the ADAPTE method to develop a clinical guideline for the improvement of psychoses and schizophrenia care: Example of involvement and participation of patients and family caregivers. Health Expectations. 2021 Apr 23;24(2):516–24.
