Quantitative framework for validating two methodologies that are used to enumerate viable organisms for type approval of ballast water management systems

The Stain-Motility (S-M) and Most Probable Number (MPN) methodologies may be used for enumerating viable organisms ≥ 10 μm to <50 μm in discharge water for administrative type approval of a ballast water management system (BWMS). Only MPN is suitable for assessing the efficacy of disinfection using UV radiation — a chemical-free treatment technology — but the U.S. Coast Guard has not approved an MPN-based approach as an alternate to their required S-M method. Approval depends on a demonstration of equivalence, but a framework for comparative validation is not well established. The purpose of this study is to provide such a framework. It is shown that the requirement for 5 consecutive successful results in BWMS type approval testing fundamentally changes the relationship between a method's precision and its effectiveness in ensuring compliance with regulations. False approval due to random measurement error is effectively eliminated for both methods because it requires 5 consecutive underestimates, and false rejection due to a single erroneously high measurement is more likely for the method with wider confidence limits, imposing an extra margin of safety for MPN. These results reverse conventional interpretations of efficacy based on method precision alone. Sources of systematic error (bias) are reviewed and methods for estimating the errors are described. If combined bias is positive (overestimation), either method would yield type-approval results fully compliant with regulations. Subject to similar negative bias, the less precise method (generally, MPN) would be more protective of the environment. An illustrative analysis of 64 paired MPN and S-M counts from BWMS trials indicates that neither method was significantly biased relative to the other. Considered in the framework for method validation described here, available evidence strengthens arguments that in a BWMS type approval testing regime, the efficacy of the MPN method is equivalent to that of Stain-Motility.