Managing multiplicity in clinical vaccine studies – A case study using a gatekeeping testing strategy

Multiplicity issues are increasingly common in vaccine clinical studies. Common causes include multi-valent combinations/co-administrations requiring separate evaluation of each antigen; numerous efficacy endpoints; requests from regulatory authorities for inclusion of specific powered endpoints into registration studies; interim analyses to support early decision-making. In a Phase III study to evaluate safety and immunogenicity of the 4-component Neisseria meningitidis serogroup B vaccine (4CMenB) when co-administered with 13-valent pneumococcal conjugate vaccine (PCV13) to healthy infants, a total of 49 statistical hypotheses were identified for the primary objectives as requested by the health authority. We designed a sequential testing strategy with visualization using a graphical gatekeeping procedure. The 49 immunogenicity objectives related to evaluation of the sufficiency of the 4CMenB immune response; and demonstration of non-inferiority of PCV13 and 4CMenB when co-administered versus administration alone. We used a graphical shortcut display for closed families assuming that the multiple testing procedure is consonant and hypotheses that are rejected by a closed testing procedure are also rejected within the graphical short-cut. The 49 hypotheses were grouped into 10 families and distributed over 4 sequential steps following the clinical and statistical logical relationships agreed with the clinical team. Test decisions within the first 8 families will be made based on p-values with alpha propagation to subsequent families according to the tree structure. This tailored strategy allowed evaluation of all 49 statistical hypotheses individually, and more efficiently. The method avoided a rigid all-or-nothing approach whereby all endpoints fail if one or more null hypotheses cannot be rejected. Clinical input and agreement are critical for designing an efficient and fit-for-purpose strategy. Our experience could encourage more application of such strategies in increasingly complex clinical trials.