Developing an implementation fidelity measure for a family healthy weight program

Abstract Background Measuring implementation fidelity is crucial yet proves challenging. While observational methods are considered the gold standard, their practicality in geographically dispersed community settings is often limited by resource constraints. Engaging community members as paid research staff is a potential strategy to develop local capacity to conduct direct observations. This paper reports on the development and preliminary utility of a fidelity measure for a community-based family healthy weight program (FHWP), Building Healthy Families (BHF), and a method to hire and train local community members to conduct direct observation. Methods A consensus process guided the development of a comprehensive fidelity measure for direct observation. We piloted and refined the measure using a qualitative iterative approach with observers. Communities delivering BHF were geographically dispersed up to 450 miles resulting in the development of a training protocol to hire and train local community members as direct observers. Inter-rater agreement of ≥ 85% with an expert observer was required for observers to independently assess BHF sessions. Results A multidimensional fidelity measure for direct observation was developed specific to the core components of BHF and the session structure and process. The training method successfully prepared community-based observers (n = 5) to conduct fidelity assessments with the same quality as the trained research team observers (n = 3). Inter-rater agreement ≥ 85% with an expert observer was achieved on all training sessions. The fidelity measure demonstrated strong utility, effectively capturing multiple dimensions of fidelity and provided actionable insights to support consistent and high-quality implementation across community settings. Conclusions This study provides a systematic approach to assessing implementation fidelity of a FHWP in micropolitan and surrounding rural areas. Our approach to hiring and training local community members as direct observers enhanced the feasibility of measuring implementation fidelity across multiple geographically dispersed settings and established a model for ongoing assessments.