Letter to the Editor Regarding “Assessing Methods for Replacement of Soiled Bedding Sentinels in Cage-level Exhaust IVC Racks” by Eichner and Smith
Dear Editor,
We thank the authors Eichner and Smith for their recent article comparing their soiled bedding sentinel (SBS) program to iterations of the sentinel-free soiled bedding (SFSB) approach on a large-scale, institution-wide basis.3 To date, 6 publications have found SFSB to be superior to SBS.2,4,5,8-10 Nevertheless, we are very interested in literature with a solid methodology that identifies the limitations of SFSB methods. We caution that the study we are addressing with this letter has fatal flaws and does not provide any meaningful challenge to the current body of work supporting SFSB.
SFSB is a form of environmental health monitoring (EHM) that involves placing soiled bedding from colony animals into a dedicated animal-free cage or container for sampling. After exposure of media and/or swabs to dust, dander, urine, and feces, the media/swabs are submitted for PCR analysis for pathogens of interest. Since earlier publications initially introduced the concept of SFSB,2,4 it has become a topic of great interest for many research animal professionals who are responsible for oversight of biosecurity, animal welfare, and health monitoring programs for rodent colonies. We support the objective assessment of new methods performed in a real-life setting, as this information aids in shaping best practices. However, below we outline 4 flaws in this latest paper that invalidate its conclusions that “the soiled bedding sentinel method has highest concurrence with the expected health status of an animal room…” and “…the current study failed to demonstrate that environmental sampling methods were accurate replacement options for SBS surveillance…”3
Addressed below in more detail, the 4 flaws are as follows: 1) failure to consider the impact of sample pooling for SFSB but not for SBS samples; 2) failure to control for feces collected using supplemental direct colony sampling (DCS) for SBS samples but not for SFSB samples; 3) use of inappropriate statistics for the conclusions being drawn; and 4) performance of direct comparisons between a variety of treatment groups with incomplete data sets. The majority of these flaws are fatal and uncorrectable and make it impossible to reanalyze the data and draw any reasonable conclusions.
Pooling of Samples
The article describes the pooling of SFSB samples at the room level, whereas SBS samples were analyzed at the rack level. We consider this a fatal flaw in methodology. SFSB samples should have been analyzed in the same manner as the SBS samples. Pooling of diagnostic samples challenges the sensitivity of molecular assays. With pooling, the quantity of nontarget nucleic acid can increase disproportionally above that of the targeted template. In addition, the likely “sandwiching” of pooled ReemayR 2024 material in the submission vial impedes buffer access to the material surfaces for the equivalent collection of nucleic acids. Furthermore, room-level pooling inherently limits the number of data points and cannot be compared with nonpooled, rack-level sampling. Although likely performed as a cost-saving strategy, pooling of SFSB samples ultimately results in a biased comparison that would artificially compromise the sensitivity of SFSB and boost the sensitivity of the rack-level SBS program. This prevents valid comparisons and severely biases the outcome of the experiment.
SBS Samples Supplemented with DCS
Here we raise concerns about fecal pellets from colony cages (DCS) being combined with SBS samples but not with SFSB samples. As seen above with sample pooling, the analysis of samples that are supplemented for some groups but not others also results in an apples-to-oranges comparison. DCS is another form of EHM that does not rely on sentinel animals and has been proposed to increase the sensitivity of other EHM methods, including SFSB.2,6,10 This flaw in the experimental design could have been avoided had SBS samples been collected and analyzed separately from DCS samples. However, with the data reported, the authors cannot differentiate the true contribution of the SBS as compared with the DCS in the combined sample. Furthermore, they do not accurately report the combination of SBS and DCS, as seen in Figure 2, where they refer to SBS and DCS as “sentinel fecal PCR” or throughout the rest of the paper when they mention “soiled bedding sentinel method” or the “SBS program.” Even though DCS can be valuable as an adjunct method of health monitoring, it should not be considered to be an “SBS method” and precludes valid comparison because it was not performed in all groups. This design flaw negatively and artificially compromises the sensitivity of SFSB methods and boosts the sensitivity of SBS.
Inappropriate Use of Statistics
The use of Cohen’s κ statistic was inappropriate for this study. Kappa is a descriptive statistic used to assess interrater reliability of humans using ordinal scoring systems (e.g., whether in a particular dataset 2 observers agree on the score for a given animal) and was developed to account for chance agreement due to guessing.7 The authors of this manuscript were trying to answer the hypothesis-driven question of whether SBS performs better than multiple SFSB methods, with SBS considered to be the gold standard. Kappa, as a descriptive statistic, is inappropriate for this question as it cannot be directly interpreted to infer an answer to a hypothesis-based question. Rather, to test a hypothesis, inferential statistical methods (e.g., logistic regression) that use sample data to make predictions about a large population must be used. Furthermore, the methods of handling missing data for κ are complex and must be managed carefully as the choice radically affects outcomes.1 Finally, we are unable to determine what “average kappa” means, which further hinders statistical interpretation.
Use of Incomplete Data Sets
As the authors note in their discussion, not every facility was tested for every agent or using all methods, resulting in many incomplete data sets. Of the 5 facilities, only one facility with 6 rooms containing 31 racks provided data for all experimental groups.3 Further, as stated in the article discussion, the sample loss was attributed to human error and lack of staff compliance. From the many institutions that have completely switched from SBS to EHM methods, it is well known that staff education and training are instrumental to a reliable and effective EHM program.6 With a program-wide approach under real-world conditions, increased oversight and training of staff should have been implemented before beginning the experiment. Although human error and lack of staff compliance may indeed occur in the real world, these circumstances do not justify drawing conclusions based on incomplete data. Indeed, the gaps in data and acknowledged variability in facility compliance cripple a side-by-side comparison of results.
Use of ReemayR 2024 in Comparison Studies
Referred to as filter paper in the article, ReemayR 2024 is spun-bound polyester nonwoven fabric and has been used in recent articles comparing SBS to SFSB.5,8 To date, ReemayR 2024 performs comparable to Allentown’s Sentinel media but is not as effective when compared with passive and active flocked swabs.5,8 Nevertheless, diagnostic companies continue to put significant effort into the testing and selection of media for the sole purpose of optimizing dust and debris binding for detecting infectious agents by PCR. Materials new to the market and likely not available at the time of this study include the Charles River Pathogen Binder material and the IDEXX BioAnalytics REPLACE matrix. The advantages of these media through commercial studies have been presented at industry meetings, posted on vendor websites, and disseminated in white papers. Thus, the use of Reemayx 2024 may not be considered best practice and may likely compromise the sensitivity of SFSB methods moving forward.
In conclusion, the 4 flaws of the cited paper essentially prevent the formulation of any cohesive conclusions. Our consensus-driven group of subject-matter experts in rodent health monitoring holds that EHM is clearly superior to SBS when compared in an unbiased manner, using industry-wide accepted methods and appropriate statistical interpretation. Based on the evidence provided by well-executed and controlled experiments, we continue to support the use of EHM methods, which also have 3Rs benefits. The movement of the industry in this direction is vital for ensuring high-quality biosecurity, fulfillment of the 3Rs of animal research, and exceptional animal welfare.
Sincerely,
Chris Manuel, DVM, PhD, DACLAM
Office of Laboratory Animal Resources, University of Colorado Anschutz Medical Campus
Aurora, Colorado
Kerith Luchins, DVM, DACLAM
Rodent Clinical Services, The University of Chicago
Chicago, Illinois
Norman C Peterson, DVM, PhD, DACLAM
In Vivo Sciences and Technology, Seagen
Bothell, Washington
Aurore Dodelet-Devillers, DVM, MSc, DES Lab Animal
Research Institute of the McGill University Health Centre
Montreal, Canada
Christina Pettan-Brewer, DVM, MSc
Rodent Health Monitoring Program, University of Washington
Seattle, Washington
Lise Phaneuf, DVM, DVSc, DACLAM
Animal Resources and Compliance, The Centre for Phenogenomics
Toronto, Canada
Joseph P Garner, DPhil
Department of Comparative Medicine, Department of Psychiatry and Behavioral Sciences, Stanford University
Stanford, California
Megan LaFollette, PhD
The 3Rs Collaborative
Denver, Colorado
Abbreviations and acronyms: DCS, direct colony sampling; EHM, environmental health monitoring; SBS, soiled bedding sentinels; SFSB, sentinel-free soiled bedding