Replicability of Scientific Claims in Drosophila Immunity: A Retrospective Analysis of 400 Publications

Reviewer #1 (Public Review):

This work revisits a substantial part of the published literature in the field of Drosophila innate immunity from 1959 to 2011. The strategy has been to restrain the analysis to some 400 articles and then to extract a main claim, two to four major claims and up to four minor claims totaling some 2000 claims overall. The consistency of these claims with the current state-of-the-art has been evaluated and reported on a dedicated Web site known as Reprosci and also in the text as well as in the 28 Supplements that report experimental verification, direct or indirect, e.g., using novel null mutants unavailable at the time, of a selected set of claims made in several articles. Of note, this review is mostly limited to the manuscript and its associated supplements and does not integrally cover the Reprosci website.

Strengths:
One major strength of this article is that it tackles the issue of reproducibility/consistency on a large-scale. Indeed, while many investigators have some serious doubts on some results found in the literature, few have the courage, or the means and time, to seriously challenge studies, especially if published by leaders in the field. The Discussion adequately states the major limitations of the Reprosci approach, which should be kept in mind by the reader to form its own opinion.

This study also allows investigators not familiar with the field to have a clearer understanding of the questions at stake and to derive a more coherent global picture that allows them to better frame their own scientific questions. Besides a thorough and up-to-date knowledge of the literature used to assess the consistency of the claims with our current knowledge, a merit of this study is the undertaking of independent experiments to address some puzzling findings and the evidence presented is often convincing, albeit one should keep in mind the inherent limitations as several parameters are difficult to control, especially in the field of infections, as underlined by the authors themselves. Importantly, some work of the lead author has also been re-evaluated (e.g., Supplements S2-S4). Thus, while utmost caution should be exerted, and often is, in challenging claims, even if the challenge eventually proves to be not grounded, it is valuable to point out potential controversial issues to the scientific community.

While this is not a point of this review, it should be acknowledged that the possibility to post comments on the ReproSci website will allow further readjustments by the community in the appreciation of the literature and also of the Reprosci assessments themselves and of its complementary additional experiments. As science is evolving continuously, this site allows the current state of knowledge to be updated as needed, not necessarily by the authors themselves.

Comments on revised version:

The major weaknesses identified in the original version of the manuscript have been adequately addressed by the authors. In a few cases, additional KO mutants have been generated and analyzed thus further strengthening the manuscript.

Reviewer #2 (Public review):

Summary:

The authors present an ambitious and large-scale reproducibility analysis of 400 articles on Drosophila immunity published before 2011. They extract major and minor claims from each article, assess their verifiability through literature comparison and, when possible, through targeted experimental re-testing, and synthesize their findings in an openly accessible online database. The goal is to provide clarity to the community regarding claims that have been contradicted, incompletely supported, or insufficiently followed up in the literature, and to foster broader community participation in evaluating historical findings. The manuscript summarizes the major insights emerging from this systematic effort.

Strengths:

(1) Novelty and community value: This work represents a rare example of a systematic, transparent, and community-facing reproducibility project in a specific research domain. The creation of a dedicated public platform for disseminating and discussing these assessments is particularly innovative.

(2) Breadth and depth: The authors analyze an impressive number of publications spanning multiple decades, and they couple literature-based assessments with new experimental data where follow-up is missing.

(3) Clarity of purpose: The manuscript carefully distinguishes between assessing evidential support for claims and judging the scientific merit of historical work. This helps frame the project as constructive rather than punitive.

(4) Metascientific relevance: The analysis identifies methodological and contextual factors that commonly underlie irreproducible claims, providing a useful guide for future study design and interpretation.

(5) Transparency: Supplementary datasets and the public website provide an exceptional degree of openness, which should facilitate community engagement and further refinement.

Comments on revised version:

In the revised manuscript the authors addressed in a satisfactory manner the weaknesses I previously highlighted in my initial review.

Reviewer #3 (Public review):

Summary:

In this ambitious study the authors set out to analyse the validity of a number of claims, both minor and major from 400 published articles within the field of Drosophila immunity that were published before 2011. The authors were able to determine initially if claims were supported by comparing to other published literature in the field and if required by experimentally testing 'unchallenged' claims that had not been followed up in subsequent published literature. Using this approach the authors identified a number of claims that had contradictory evidence using new methods or taking into account developments within the field post initial publication. They publish their findings on a publicly available website designed to enable the research community to assess published work within the field with greater clarity.

Strengths:

The work presented is rigorous and methodical, the data presentation is high quality and importantly the data presented support the conclusions. The discussion is balanced and the study is written considerately and respectfully highlighting that the aim of the study is not to assign merit to individual scientists or publications but rather to improve clarity for scientists across the field. The approach carried out by the researchers focuses on testing the validity of the claims made in the original papers rather than testing whether the original experimental methods produced reproducible results. This is an important point since there are many reasons why the original interpretation of data may have understandably led to the claims made. These potential explanations for irreproducible data or conclusions are discussed in detail by the authors for each claim investigated.

The authors have generated an accompanying website which provides a valuable tool for the Drosophila Immunity research community that can be used to fact check key claims and encourages community engagement. This will achieve one important goal of this study - to prevent time loss for scientists who base their research on claims that are irreproducible. The authors rightly point out that it is impossible (and indeed undesirable) to avoid publication of irreproducible results within a field since science is 'an exploratory process where progress is made by constant course correction'. This study is however an important piece of work that will make that course correction more efficient.

Comments on revised version:

I'm impressed with the revisions carried out by the authors. No further comments.

Reviewer #4 (Public review):

Westlake, Lemaitre and colleagues have revised their manuscript regarding the reproducibility of studies on the Drosophila immune system. As was emphasized by all reviewers at the original submission of this manuscript, this paper captures a tremendous effort and makes a valuable contribution to the community.

One strong conclusion to be drawn from the overall effort is that the scientific literature is, for the most part, very good. The vast majority of claims made in the 400 papers assembled for this project were substantiated, either with previously published follow-up work or with new experimentation presented here. Although the authors have chosen not to quantify the overall validation rate of scientific claims in the present study, that information is available on the ReproSci website and in the companion paper.

In the revised manuscript, the authors make the distinction between repeatability/direct reproducibility and what they term "conceptual replication" and "indirect reproducibility", emphasizing that this study falls in the latter category because in many cases the present authors are not attempting to reproduce or replicate the original study. Instead, they are often using new tools to reinvestigate original claims and determine the accuracy of those claims, and whether they stand up to additional data. The authors have replaced the word "irreproducible" with "irreplicable" in the revised manuscript. I continue to believe the present work can be considered "validation" of the original studies and that the present work is accurately considered a "validation project". The clarification of terminology has strengthened the revised manuscript.

As with any study, the strength and breadth of conclusions that can be drawn from the present work are a function of the strength and resolution of the experiments performed. This may vary across the supplements so readers should interpret the presented results accordingly. In my opinion, all of the supplements in the revised article are sufficiently convincing to warrant inclusion in the final manuscript.

There are several instances where the authors have declined to do additional experiments in response to review. I am comfortable with these decisions, recognizing that the extra work required would be prohibitively much. Instead, the authors have adequately revised their presentation of findings. In some instances where the authors have declined to perform additional experiments, they have provided a compelling logic for their decision in the response to reviews, which unfortunately does not appear in the main manuscript. For example, in response to a comment by Reviewer 1, the authors wrote "In response to the suggestion of scanning multiple blots for quantification, we note that no clear enhancement of ModSP autoactivation was observed across different incubation times. We also explored alternative incubation conditions, but none resulted in a detectable increase in activation. Thus, additional quantification would not change the overall conclusion." This is convincing, but no readers will see it unless it appears in the manuscript or the public response to review. At the same time, I recognize the practical limitations in available space in the main article and supplement. There are a handful of instances where reviewers have made suggestions of additional/alternative experiments that would be more rigorous, accurate, or convincing than what is currently presented but the authors have declined to take up those recommendations. In some instances, I strongly agree with the authors' rationale and in other cases it is less so. However, I did not find any instance where the absence of the suggested additional experiment substantially undermined the conclusions being put forward in this manuscript. Therefore, I believe these decisions can be left to the discretion of the authors in accordance with the eLife publication model and interpretations can be left to the discretion of the readers.

Reviewer 1 and Reviewer 4 both commented on a lack of demonstration of RNAi efficacy in the knockdown experiments and suggested that negative conclusions drawn from RNAi experiments should include controls showing that the RNAi was effective. The authors have declined to include such controls, and I recognize they may be impossible to add now without redoing the entire associated experiment. Instead, the authors point to prior use of the RNA constructs by other authors in the literature. That should be recognized as less than ideal, although it is probably substantially adequate. The authors have added statements to the Discussion that RNAi knockdowns are not equivalent to null mutations and negative results may appear because of hypomorphic effects. This is a positive addition.

The replacement of the sphinx RNAi with a deletion of Sphinx 1 and Sphinx 2 as part of a general strengthening of the section on sphinx and spheroid is an extremely strong addition to the paper. I could not expect that level of effort for each of the validations and I acknowledge and appreciate that they have made that effort here.

Reviewer 1 and Reviewer 4 both commented that some figures in the original submission were ambiguous with respect to sample size and distribution of data points, including undefined error bars and inappropriate graphical representation of experiments with low sample size. These concerns have been largely addressed in the revised manuscript, including with addition of statistical analyses in several cases. The authors note that the figures were generated by multiple individuals across several research teams so it would be a large effort unify their style. I agree that this would be an unreasonable effort at this stage, for minimal return that is primarily aesthetic. The authors have now added illustration of all data points in most figures and have defined the error bars. One response to Reviewer 1 "Data points should be shown in the figure" with respect to a figure in S16 was "Unfortunately, we could not address this point to the difficulty to reach the collaborator who provided the data." That response certainly would not be acceptable in the publication of a typical scientific paper. However, I sense a different standard is being applied to the present work and the lack of these data points in this particular figure does not decrease my confidence in the authors' overall conclusion.

Overall, this is a unique piece of research that is impressive in its scope. Scientific finding is never complete; the process is eternally iterative. Inclusion of the present study in the scientific literature is a testament to collective willingness to engage in that iteration. The authors of this paper are making an exemplar contribution to accuracy in scientific research and should be commended for their effort, dedication, and fairness in what can be a delicate topic.

Author response:

The following is the authors’ response to the original reviews.

eLife Assessment

This fundamental study is part of an impressive, large-scale effort to assess the reproducibility of published findings in the field of Drosophila immunity. In a companion article, the authors analyze 400 papers published between 1959 and 2011, and assess how many of the claims in these papers have been tested in subsequent publications. In this article, the authors report the results of validation experiments to assess a subset of the claims that, according to the literature, have not been corroborated. While the evidence reported for some of these validation studies is convincing, it remains incomplete for others.

We thank the reviewers for their careful assessment of our manuscript. We are fully aware that this review process required a substantial investment of time and effort, and we are deeply grateful for their engagement. We also thank the eLife editors for enabling an open and constructive evaluation process of the highest standard.

We fully agree with the eLife assessment, including the concluding statement highlighting the limitations of our analysis. We also concur with most of the reviewers’ comments, which we have addressed in detail in the revised manuscript. Addressing these comments required extensive additional work, which explains the duration of the revision. This period was also valuable in allowing colleagues from the Drosophila community to provide feedback on our study, including identifying potential issues—for instance regarding SR-C1—that we have carefully re-examined.

Major revisions include the addition of new experimental data in the sections on Hemese, SR-C1, Sphinx1/2, and Caspar. These results largely confirm our initial conclusions, with the exception of SR-C1. We have therefore revised our interpretation of SR-C1, confirming earlier findings that it promotes bacterial binding, while maintaining that it does not function as a major phagocytic receptor. A table with new list of new reagents has been added at the end of the supplement (Table Supplement S4). We also add a comment on the fact that mutation in the serine protease MP1 (work from B.L.) does not affect wound melanization as initially proposed (Tang et al., 2006).

One final point: our study assesses replicability using alternative experimental approaches, and we cannot exclude the possibility that our conclusions may be inaccurate. This limitation is inherent to all reproducibility studies. Nonetheless, our work should encourage the community to exercise caution when interpreting certain previous claims, some of which have become widely accepted.

We thank the reviewers and editors for their constructive input, which has significantly improved the manuscript. At this stage, we are ready to proceed toward a version of record together with the eLife assessment (which may still evolve). We continue to believe that our reproducibility project is unique in its scope and depth. It not only provides quantitative insight into replicability (See companion article)—suggesting that replication rates may be higher in certain fields, consistent with the maturation of knowledge over recent decades—but also contributes to clarifying and refining a number of statements within the field. Although aware of the cost of assessing reproducibility (hard work for articles that are not quoted and statements that are not taking in account), I consider it as one of my most important and original works.

Public Reviews:

Reviewer #1 (Public review):

Summary:

This work revisits a substantial part of the published literature in the field of Drosophila innate immunity from 1959 to 2011. The strategy has been to restrain the analysis to some 400 articles and then to extract a main claim, two to four major claims and up to four minor claims totaling some 2000 claims overall. The consistency of these claims with the current state-of the-art has been evaluated and reported on a dedicated Web site known as ReproSci and also in the text as well as in the 28 Supplements that report experimental verification, direct or indirect, e.g., using novel null mutants unavailable at the time, of a selected set of claims made in several articles. Of note, this review is mostly limited to the manuscript and its associated supplements and does not integrally cover the ReproSci website.

Strengths:

One major strength of this article is that it tackles the issue of reproducibility/consistency on a large scale. Indeed, while many investigators have some serious doubts about some results found in the literature, few have the courage, or the means and time, to seriously challenge studies, especially if published by leaders in the field. The Discussion adequately states the major limitations of the ReproSci approach, which should be kept in mind by the reader to form their own opinion.

This study also allows investigators not familiar with the field to have a clearer understanding of the questions at stake and to derive a more coherent global picture that allows them to better frame their own scientific questions. Besides a thorough and up-to-date knowledge of the literature used to assess the consistency of the claims with our current knowledge, a merit of this study is the undertaking of independent experiments to address some puzzling findings and the evidence presented is often convincing, albeit one should keep in mind the inherent limitations as several parameters are difficult to control, especially in the field of infections, as underlined by the authors themselves. Importantly, some work of the lead author has also been re-evaluated (Supplements S2-S4). Thus, while utmost caution should be exerted, and often is, in challenging claims, even if the challenge eventually proves to be not grounded, it is valuable to point out potential controversial issues to the scientific community.

While this is not a point of this review, it should be acknowledged that the possibility to post comments on the ReproSci website will allow further readjustments by the community in the appreciation of the literature and also of the ReproSci assessments themselves and of its complementary additional experiments.

We praise the reviewer for this assessment of our project

Weaknesses:

Challenging the results from articles is, by its very nature, a highly sensitive issue, and utmost care should be taken when challenging claims. While the authors generally acknowledge the limitations of their approach in the main text and Supplements, there are a few instances where their challenges remain questionable and should be reassessed. This is certainly the case for Supplement S18, for which the ReproSci authors make a claim for a point that was not made in the publication under scrutiny. The authors of that study (Ramet et al., Immunity, 2001) never claimed that scavenger receptor SR-CI is a phagocytosis receptor, but that it is required for optimal binding of S2 cells to bacteria. Westlake et al. here have tested for a role of this scavenger receptor in phagocytosis, which had not been tested by Ramet et al. Thus, even though the ReproSci study brings additional knowledge to our understanding of the function of SR-CI by directly testing its involvement in phagocytosis by larval hemocytes, it did not address the major point of the Ramet et al. study, SR-CI binding to bacteria, and thus inappropriately concludes in Supplement S18 that "Contrary to (Ramet et al., 2001, Saleh et al., 2006), we find that SR-CI is unlikely to be a major Drosophila phagocytic receptor for bacteria in vivo." It follows that the results of Ramet et al. cannot be challenged by ReproSci as it did not address this program. Of note, Saleh et al. (2006) also mistakenly stated that SR-CI impaired phagocytosis in S2 cells and could be used as a positive control to monitor phagocytosis in S2 cells. Their assay appears to have actually not monitored phagocytosis but the association of FITC-labeled bacteria to S2 cells by FACS, as they did not mention quenching the fluorescence of bacteria associated with the surface with Trypan blue.

We have revised our evaluation of Sr-C1, as we acknowledge that we did not initially read Rämet et al. carefully. We incorrectly assumed that this study demonstrated a role for SR-C1 in phagocytosis, whereas it actually reports a defect in binding only. We have updated our assessment of SR-C1, validating the results from Rämet et al., 2001. We still mention our results on phagocytosis (See below for more information).

The inference method to assess the consistency of results with current knowledge also has limitations that should be better acknowledged. At times, the argument is made that the gene under scrutiny may not be expressed at the right time according to large-scale data or that the gene product was not detected in the hemolymph by a mass-spectrometry approach. While being in theory strong arguments, some genes, for instance, those encoding proteases at the apex of proteolytic activation cascades, need not necessarily be strongly expressed and might be released by a few cells. In addition, we are often lacking relevant information on the expression of genes of interest upon specific immune challenges such as infections with such and such pathogens.

The pattern of expression is always used as an element supporting our assessment but never as the central point of decision (See below for more information).

As regards mass spectrometry, there is always the issue of sensitivity that limits the force of the argument. Our understanding of melanization remains currently limited, and methods are lacking to accurately measure the killing activity associated with the triggering of the proPO activation cascade. In this study, the authors monitor only the blackening reaction of the wound site based on a semi-quantitative measurement. They are not attempting to use other assays, such as monitoring the cleavage of proPOs into active POs or measuring PO enzymatic activity. These techniques are sometimes difficult to implement, and they suffer at times from variability. Thus, caution should be exerted when drawing conclusions from just monitoring the melanization of wounds.

We employed approaches similar to those used in the original studies under evaluation. However, we acknowledge that our analysis primarily relied on cuticular melanization, which is more straightforward to monitor. We have revised the manuscript to clarify this limitation, particularly in the case of PGRP-LE.

Likewise, the study of phagocytosis is limited by several factors. As most studies in the field focus on adults, the potential role of phagocytosis in controlling Gram-negative bacterial infections is often masked by the efficiency of the strong IMD-mediated systemic immune response mediated by AMPs (Hanson et al, eLife, 2019). This problem can be bypassed in rare instances of intestinal infections by Gram-negative bacteria such as Serratia marcescens (Nehme et al., PLoS Pathogens, 2007) or Pseudomonas aeruginosa (Limmer et al. PNAS, 2011), which escape from the digestive tract into the hemocoel without triggering, at least initially, the systemic immune response. It is technically feasible to monitor bacterial uptake in adults by injecting fluorescently labeled bacteria and subsequently quenching the signal from noningested bacteria. Nonetheless, many investigators prefer to resort to ex vivo assays starting from hemocytes collected from third-instar wandering larvae as they are easier to collect and then to analyze, e.g., by FACS. However, it should be pointed out that these hemocytes have been strongly exposed to a peak of ecdysone, which may alter their properties. Like for S2 cells, it is thus not clear whether third-instar larval hemocytes faithfully reproduce the situation in adults. The phagocytic assays are often performed with killed bacteria. Evidence with live microorganisms is better, especially with pathogens. Assays with live bacteria require however, an antibody used in a differential permeabilization protocol. Furthermore, the killing method alters the surface of the microorganisms, a key property for phagocytic uptake. Bacterial surface changes are minimal when microorganisms are killed by X-ray or UV light. These limitations should be kept in mind when proceeding to inference analysis of the consistency of claims. Eater illustrates this point well. Westlake et al. state that:" [...] subsequent studies showed that a null mutation of eater does not impact phagocytosis". The authors refer here to Bretscher et al., Biology Open, 2015, in which binding to heat-killed E. coli was assessed in an ex vivo assay in third instar larvae. In contrast, Chung and Kocks (JBC, 2011) tested whether the recombinant extracellular N-terminal ligand-binding domain was able to bind to bacteria. They found that this domain binds to live Gram-positive bacteria but not to live Gram-negative bacteria. For the latter, killing bacteria with ethanol or heating, but not by formaldehyde treatment, allowed binding. More importantly, Chung and Kocks documented a complex picture in which AMPs may be needed to permeabilize the Gram-negative bacterial cell wall that would then allow access of at least the recombinant secreted Eater extracellular domain to peptidoglycan or peptidoglycan-associated molecules. Thus, the systemic Imd-dependent immune response would be required in vivo to allow Eater-dependent uptake of Gram-negative bacteria by adult hemocytes. In ex vivo assays, any AMPs may be diluted too much to effectively attack the bacterial membrane. A prediction is then that there should be an altered phagocytosis of Gramnegative bacteria in IMD-pathway mutants, e.g., an imd null mutant but not the hypomorphic imd[1] allele. This could easily be tested by ReproSci using the adult phagocytosis assay used by Kocks et al, Cell, 2005. At the very least, the part on the role of Eater in phagocytosis should take the Chung &Kocks study into account, and the conclusions modulated.

We agree that experimental conditions can always influence the outcome and may partly explain discrepancies in reproducibility. However, our approach is based on conceptual replication, with the aim of testing the generalizability of the original findings rather than reproducing them under identical conditions. We often used approaches quite similar to the initial statements.

In the case of Eater, our results challenge the conclusion from Kocks et al. that phagocytosis of Gram-negative bacteria is impaired in the absence of Eater, as this was not confirmed when using null eater mutants. Nevertheless, our data support the idea that Eater contributes to the phagocytosis of Gram-negative bacteria, likely in cooperation with NimC1.

Importantly, our findings are fully consistent with those of Chung and Knocks, who showed that the extracellular domain of Eater binds Gram-positive but not Gram-negative bacteria. Therefore, our work refines a specific aspect of the original conclusion, without questioning the overall significance of the initial Eater studies.

Another point is that some mutant phenotypes may be highly sensitive to the genetic background, for instance, even after isogenization in two different backgrounds. In the framework of a Reproducibility project, there might be no other option for such cases than direct reproduction of the experiment as relying solely on inference may not be reliable enough.

With respect to the experimental part, some minor weaknesses have been noted. The authors rely on survival to infection experiments, but often do not show any control experiments with mock-challenged or noninfected mutant fly lines. In some cases, monitoring the microbial burden would have strengthened the evidence. For long survival experiments, a check on the health status of the lines (viral microbiota, Wolbachia) would have been welcome. Also, the experimental validation of reagents, RNAi lines, or KO lines is not documented in all cases.

We thank the reviewer for their careful and constructive assessment. We agree with most of the points raised and have revised the manuscript accordingly. In particular, we have softened the wording throughout to make our evaluation less abrupt. We have also added a dedicated section on limitations at the end of the manuscript, highlighting potential sources of bias and the inherent constraints of our reproducibility approach.

In addition, we have expanded the description of the tools used and the validation of the RNAi experiments (see below). We consider it unlikely that factors such as Wolbachia or Nora virus significantly affect our conclusions, as our assays rely on short-term survival experiments, whereas these variables are more likely to influence ageing-related phenotypes. That said, we fully acknowledge that unrecognized parameters—such as environmental conditions (e.g., humidity), which we did not systematically control—could contribute to discrepancies. This limitation, however, applies broadly to most attempts at experimental reproduction.

More generally, many of the claims we challenge have not been revisited for over a decade. Our inability to validate some of them suggests that the underlying biology may be more complex than initially proposed. We recognize that some of our conclusions may be incorrect, and it would be surprising if all were entirely accurate. At the same time, reproducibility studies often face an asymmetry in standards of evidence, whereby contradicting a published claim requires a higher burden of proof than establishing it. We believe it is important to document and share such discrepancies. We also explicitly acknowledge in the revised manuscript that the ReproSci framework itself may contain errors. Even if a significant proportion of our assessments are not correct, we believe this work contributes meaningfully to clarifying ongoing debates in the field for most of them.

Finally, since the launch of the website, we have received three comments from the community—one addressing a major claim (Sr-C1) and two addressing more minor points— which have led us to refine our conclusions. The limited feedback we received suggest a broad agreement of the community with our assessment. It illustrates the value of maintaining an open, community-driven resource. The public and evolving nature of the project is, in our view, a key strength.

Reviewer #2 (Public review):

Summary:

The authors present an ambitious and large-scale reproducibility analysis of 400 articles on Drosophila immunity published before 2011. They extract major and minor claims from each article, assess their verifiability through literature comparison and, when possible, through targeted experimental re-testing, and synthesize their findings in an openly accessible online database. The goal is to provide clarity to the community regarding claims that have been contradicted, incompletely supported, or insufficiently followed up in the literature, and to foster broader community participation in evaluating historical findings. The manuscript summarizes the major insights emerging from this systematic effort.

Strengths:

(1) Novelty and community value: This work represents a rare example of a systematic, transparent, and community-facing reproducibility project in a specific research domain. The creation of a dedicated public platform for disseminating and discussing these assessments is particularly innovative.

(2) Breadth and depth: The authors analyze an impressive number of publications spanning multiple decades, and they couple literature-based assessments with new experimental data where follow-up is missing.

(3) Clarity of purpose: The manuscript carefully distinguishes between assessing evidential support for claims and judging the scientific merit of historical work. This helps frame the project as constructive rather than punitive.

(4) Metascientific relevance: The analysis identifies methodological and contextual factors that commonly underlie irreproducible claims, providing a useful guide for future study design and interpretation.

(5) Transparency: Supplementary datasets and the public website provide an exceptional degree of openness, which should facilitate community engagement and further refinement.

We praise the reviewer for this assessment of our project.

Weaknesses:

(1) Subjectivity in selection: Despite the authors' efforts, the choice of which papers and claims to highlight cannot be entirely objective. This is an inherent limitation of any retrospective curation effort, but it remains important to acknowledge explicitly.

We have added a section at the end of the discussion to discuss the limitation of our study.

(2) Emphasis on irreproducible claims: The manuscript focuses primarily on claims that are challenged or found to be weakly supported. While understandable from the perspective of novelty, this emphasis may risk overshadowing the value of claims that are well supported and reproducible.

We have added two sentences at the end of the introduction and at the beginning of the discussion that most claims are reproducible.

(3) Framing and language: Certain passages could benefit from more neutral phrasing and avoidance of binary terms such as "correct" or "incorrect," in keeping with the open-ended and iterative nature of scientific progress.

We have carefully checked the phrasing of our article to be more nuanced and avoid black-and-white assessments.

(4) Community interaction with the dataset: While the website is an excellent resource, the manuscript could further clarify how the community is expected to contribute, challenge, or refine the annotations, especially given the large volume of supplementary data.

This point is already addressed in the revised sentence: “We hope that the community accessible website will encourage researchers to share their perspectives and contribute data from diverse sources, thereby improving objectivity.” In addition, we have proactively engaged with principal investigators to motivate them to contribute to this effort by sharing unpublished results and information. However, it remains unclear to what extent scientists are genuinely interested in reproducibility, as has been noted in other studies on the topic. For example, articles that challenge previous findings are often under-cited, and the presence of contradictory evidence does not necessarily prevent researchers from continuing to cite the original claims. This is clearly a challenge of reproducibility study. Comments, evidence and corrections will not be integrated in the ReproSci website. We have already updated this database when revising this article and will continue to do it.

(5) Minor inconsistency: The manuscript states that papers from 1959-2011 were included, but the Methods section mentions a range beginning in 1940.

This should be aligned for clarity.

We have solved this discrepancy by correcting the method section. This should be 1959-2011. Thank you for spotting this mistake.

Impact and significance:

This contribution is likely to have a meaningful impact on both the Drosophila immunity community and the broader scientific ecosystem. It highlights methodological pitfalls, encourages transparent post-publication evaluation, and offers a reusable framework that other fields could adopt. The work also has pedagogical value for early-career researchers entering the field, who often struggle to navigate contradictory or outdated claims. By centralizing and contextualizing these discussions, the manuscript should help accelerate more robust and reproducible research.

Reviewer #3 (Public review):

Summary:

In this ambitious study, the authors set out to analyse the validity of a number of claims, both minor and major, from 400 published articles within the field of Drosophila immunity that were published before 2011. The authors were able to determine initially if claims were supported by comparing them to other published literature in the field and, if required, by experimentally testing 'unchallenged' claims that had not been followed up in subsequent published literature. Using this approach, the authors identified a number of claims that had contradictory evidence using new methods or taking into account developments within the field post-initial publication. They put their findings on a publicly available website designed to enable the research community to assess published work within the field with greater clarity.

Strengths:

The work presented is rigorous and methodical, the data presentation is high quality, and importantly, the data presented support the conclusions. The discussion is balanced, and the study is written considerately and respectfully, highlighting that the aim of the study is not to assign merit to individual scientists or publications but rather to improve clarity for scientists across the field. The approach carried out by the researchers focuses on testing the validity of the claims made in the original papers rather than testing whether the original experimental methods produced reproducible results. This is an important point since there are many reasons why the original interpretation of data may have understandably led to the claims made. These potential explanations for irreproducible data or conclusions are discussed in detail by the authors for each claim investigated.

The authors have generated an accompanying website, which provides a valuable tool for the Drosophila Immunity research community that can be used to fact-check key claims and encourages community engagement. This will achieve one important goal of this study - to prevent time loss for scientists who base their research on claims that are irreproducible. The authors rightly point out that it is impossible (and indeed undesirable) to avoid publication of irreproducible results within a field since science is 'an exploratory process where progress is made by constant course correction'. This study is, however, an important piece of work that will make that course correction more efficient.

Weaknesses:

I have little to recommend for the improvement of this manuscript. As outlined in my comments above, I am very supportive of this manuscript and think it is a bold and ambitious body of work that is important for the Drosophila immunity field and beyond.

We thank the reviewer for this positive assessment. This was indeed a huge amount of work with the hope this would be useful.

Reviewer #4 (Public review):

This is an important paper that can do much to set an example for thoughtful and rigorous evaluation of a discipline-wide body of literature. The compiled website of publications in Drosophila immunity is by itself a valuable contribution to the field. There is much to praise in this work, especially including the extensive and careful evaluation of the published literature.

However, there are also cautions.

We praise the reviewer for this assessment of our project.

One notable concern is that the validation experiments are generally done at low sample sizes and low replication rates, and often lack statistical analysis. This is slippery ground for declaring a published study to be untrue. Since the conclusions reported here are nearly all negative, it is essential that the experiments be performed with adequate power to detect the originally described effects. At a minimum, they should be performed with the same sample size and replication structure as the originally reported studies.

Many of the original claims we are assessing were obtained with lower sample size than our replicability experiments. Furthermore, we are often complementing our data with alternative approaches that are more robust (survival with additional bacterial strains, AMP kinetics with several time points, etc.). Although we can never claim for a total absence of an effect, our experimental designs are largely sufficient to demonstrate that we cannot report the major effects reported in the initial publications.

The first section of Results should be an overview of the general accuracy of the literature.

Of all claims made in the 400 evaluated papers, what proportion fell into each category of "verified", "unchallenged", "challenged", "mixed", or "partially verified"? This summary overview would provide a valuable assessment of the field as a whole. A detailed dispute of individual highlighted claims could follow the summary overview.

We have described the statistic of our analysis on claim validity in a companion article that will be published back-to-back with this article. We have preferred to divide assessment of claims and statistics on replicability in two articles. The present one is to provide an overview of the main claims that we considered to be fragile. Following the reviewer’s comment, we have added at the start of the manuscript a sentence stating that most of the claims were found to be validated, highlighting the solidity of the field with a link to the other article.

Section headings are phrased as declarative statements, "Gene X is not involved in process Y", which is more definitive phrasing than we typically use in scientific research. It implies proving a negative, which is difficult and rare, and the evidence provided in the present manuscript generally does not reach that threshold. A more common phrasing would be "We find no evidence that gene X contributes to process Y". A good model for this more qualified phrasing is the "We conclude that while Caspar might affect the Imd pathway in certain tissue-specific contexts, it is unlikely to act as a generic negative regulator of the Imd pathway," concluding the section on the role of Caspar. I am sure the authors feel that the softer, more qualified phrasing would undermine their article's goal of cleansing the literature of inaccuracies, but the hard declarative 'never' statements are difficult to justify unless every validation experiment is done with a high degree of rigor under a variety of experimental conditions. This caveat is acknowledged in the 3rd paragraph of the Discussion, but it is not reflected in the writing of the Results. The caveat should also appear in the Introduction.

We fully agree with the reviewer and we have softened the way we discuss all the claims in the result section.

The article is clear that "Claims were assessed as verified, unchallenged, challenged, mixed, or partially verified," but the project is called "reproducibility project" in the 7th line of the abstract, and the website is "ReproSci". The fourth line of the abstract and the introduction call some published research "irreproducible". Most of the present manuscript does not describe reproduction or replication. It describes validation, or independent experimental tests for consistency. Published work is considered validated if subsequent studies using distinct approaches yielded consistent results. For work that the authors consider suspicious, or that has not been subsequently tested, the new experiments provided here do not necessarily recreate the published experiment. Instead, the published result is evaluated with experiments that use different tools or methods, again testing for consistency of results. This is an important form of validation, but it is not reproduction, and it should not be referred to as such. I strongly suggest that variations of the words "reproducible" or "replication" be removed from the manuscript and replaced with "validation". This will be more scientifically accurate and will have the additional benefit of reducing the emotional charge that can be associated with declaring published research to be irreproducible.

We agree on this point. In response to the reviewer’s comment, we have clarified our terminology. Specifically, we now use the term “conceptual replicability” in the abstract and consistently refer to “replicability” and “irreplicability” throughout the manuscript. We hope this revised terminology improves clarity.

The manuscript includes an explanatory passage in the Results section, "Our project focuses on assessing the strength of the claims themselves (inferential/indirect reproducibility) rather than testing whether the original methods produce repeatable results (results/direct reproducibility). Thus, our conclusions do not directly challenge the initial results leading to a claim, but rather the general applicability of the claim itself." Rather than first appearing in Results, this statement should appear prominently in the abstract and introduction because it is a core element of the premise of the study. This can be combined with the content of the present Disclaimer section into a single paragraph in the Introduction instead of appearing in two redundant passages. I would again encourage the authors to substitute the word validation for reproduction, which would eliminate the need for the invented distinction between indirect versus direct reproduction. It is notable that the authors have chosen to title the relevant Methods section "Experimental Validation" and not "Replication".

The disclaimer has been moved at the end of the introduction and one sentence has been added in the abstract.

Experimental data "from various laboratories" in the last paragraph of the Introduction and the first paragraph of the Results are ambiguous. Since these new experiments are part of the central core of the manuscript, the specific laboratories contributing them should be named in the two paragraphs. If experiments are being contributed by all authors on the manuscript, it would suffice to say "the authors' laboratories". The attribution to "various labs" appears to be contradicted by the Discussion paragraph 2, which states "the host laboratory has expertise in" antibacterial and antifungal defense, implying a single lab. The claim of expertise by the lead author's laboratory is unnecessary and can be deleted if the Lemaitre lab is the ultimate source of all validation experiments.

The authors and laboratories that did the experiments are listed in the author list and are easy to identify for experts. At the same time, we did not want to highlight the specific authors to avoid a negative impact on their career. We were very happy that a small number of labs helped in our project. Since all the supplement were reviewed by Hannah Westlake (who has left the lab and the project) and myself (BL), our expertise was important as we did not want to test articles far from what we are doing in the lab. I rephrase the sentence to make it clear.

The passage on the controversial role of Duox in the gut is balanced and scholarly, and stands out for its discussion of multiple alternative lines of evidence in the published literature and supplement. This passage may benefit from research by multiple groups following up on the original claims that are not available for other claims, but the tone of the Duox section can be a model for the other sections.

Comments on other sections and supplements:

I understand the desire to explain how original results may have been obtained when they are not substantiated by subsequent experiments. However, statements such as "The initial results may have been obtained due to residual impurities in preparations of recombinant GNBP1" and "Non-replicable results on the roles of Spirit, Sphinx and Spheroide in Toll pathway activation may be due to off-target effects common to first-generation RNAi tools" are speculation. No experimental data are presented to support these assertions, so these statements and others like them (currently at the end of most "insights" sections) should not appear in Results. I recognize that the authors are trying to soften their criticism of prior studies by providing explanations for how errors may have occurred innocently. If they wish to do so, the speculative hypotheses should appear in the Discussion.

The statement in Results that "The initial claim concerning wntD may be explained by a genetic background effect independent of wntD" similarly appears to be a speculation based on the reading of the main text Results. However, the Discussion clarifies that "Here, we obtained the same results as the authors of the claim when using the same mutant lines, but the result does not stand when using an independent mutant of the same gene, indicating the result was likely due to genetic background." That additional explanation in the Discussion greatly increases reader confidence in the Result and should be explained with reference to S5 in the Results. Such complete explanations should be provided everywhere possible without requiring the reader to check the Supplement in each instance.

In some cases, such as "The results of the initial papers are likely due to the use of ubiquitous overexpression of PGRP-LE, resulting in melanization due to overactivation of the Imd pathway and resulting tissue damage", the claim to explain the original finding would be easy to test. The authors should perform those tests where they can, if they wish to retain the statements in the manuscript. Similarly, the claim "The published data are most consistent with a scenario in which RNAi generated off-target knockdown of a protein related to retinophilin/undertaker, while Undertaker itself is unlikely to have a role in phagocytosis" would be stronger if the authors searched the Drosophila genome for a plausible homolog that might have been impacted by the RNAi construct, and then put forth an argument as to why the off-target gene is more likely to have generated the original phenotype than the nominally targeted gene. There is a brief mention in S19 that junctophilin is the authors' preferred off-target candidate, but no evidence or rationale is presented to support that assertion. If the original RNAi line is still available, it would be easy enough to test whether junctophilin is knocked down as an offtarget, and ideally then to use an independent knockdown of junctophilin to recapitulate the original phenotype. Otherwise, the off-target knockdown hypothesis is idle speculation.

A good model is the passage on extracellular DNA, which states, "experiments performed for ReproSci using the original DNAse IIlo hypomorph show that elevated Diptericin expression in the hypomorph is eliminated by outcrossing of chromosome II, and does not occur in an independent DNAse II null mutant, indicating that this effect is due to genetic background (Supplementary S11)." In this case, the authors have performed a clear experiment that explains the original finding, and inclusion of that explanation is warranted. Similar background replacement experiments in other validations are equally compelling.

We believe that offering possible explanations for discrepancies between results is valuable for readers. For this reason, we have generally retained these elements in the Results section, with the exception of PGRP-SD. In that specific case, we removed the sentence stating that we could not identify the expected mutation in the original PGRP-SD mutants (see below). Although these points could have been moved to the Discussion, doing so would have made it difficult to directly associate each explanation with the corresponding claim. We therefore consider that keeping them in the Results section provides greater clarity. Our hypotheses are grounded in well-established considerations. It is known, for example, that early generations of RNAi constructs often exhibited off-target effects, that commercially available LPS preparations (e.g., from Sigma) were sometimes contaminated, and that genetic background can significantly influence immunological outcomes. It is therefore essential to consider and discuss such confounding factors, particularly those that were not fully appreciated at the time the field emerged.

Identifying motivated collaborators and coordinating experimental validation proved to be extremely time-consuming, making it impractical to systematically provide experimental verification for every discrepancy. As a result, most of our explanations remain speculative, except for the cases of WntD and DNase II, where the observed differences can be attributed to genetic background effects.

The statement "Analysis of several fly stocks expected to carry the PGRP-SDdS3 mutation used in the initial study revealed the presence of a wild-type copy PGRP-SD, suggesting that either the stock used in this study did not carry the expected mutation, or that the mutation was lost by contamination prior to sharing the stock with other labs" provides a documentable explanation of a potential error in the original two manuscripts, but the subsequent "analysis of several fly stocks" needs citations to published literature or explanation in the supplement. It is unclear from this passage how the wildtype allele in the purportedly mutant stocks could have led to the misattribution of function to PGRP-SD, so that should be explained more clearly in the manuscript.

We have removed this statement in the revised version from the result section

The originally claimed anorexia of the Gr28b mutation is explained as having been "likely obtained due to comparison to a wild-type line with unusually high feeding rates". This claim would be stronger if the wildtype line in question were named and data showing a high rate of feeding were presented in the supplement or cited from published literature. Otherwise, this appears to be speculation.

The wild-type fly stocks we used are named in the supplement.

In the section "The Toll immune pathway is not negatively regulated by wntD", FlyAtlas is cited as evidence that wntD is not expressed in adult flies. However, the FlyAtlas data is not adequately sensitive to make this claim conclusively. If the present authors wish to state that wntD is not expressed in adults, they should do a thorough test themselves and report it in the Supplement.

Alternatively, the statement "data from FlyAtlas show that wntD is only expressed at the embryonic stage and not at the adult stage at which the experiments were performed by (Gordon et al., 2005a)" could be rephrased to something like "data from FlyAtlas show strong expression of wntD in the embryo but not the adult" and it should be followed by a direct statement that adult expression was also found to be near-undetectable by qPCR in supplement S5. That data is currently "not shown" in the supplement, but it should be shown because this is a central result that is being used to refute the original claim. This manuscript passage should also describe the expression data described in Gordon et al. (2005), for contrast, which was an experimental demonstration of expression in the embryo and a claim "RT-PCR was used to confirm expression of endogenous wntD RNA in adults (data not shown)."

We agree on this but the expression pattern provided from fly atlas and other genomic resource is an additional element that reinforce our conclusion. For instance, the observation that the expression pattern of WntD is limited to early embryo is consistent with a role in the regulation of Toll pathway in D/V patterning but not immunity. The use of the expression pattern is not the key element of our assessments that rely on the use of loss-of-function mutation. We have rephrased the text in the manuscript according to Reviewer’s suggestion.

Inclusion of the section on croquemort is curious because it seems to be focused exclusively on clearance of apoptotic cells in the embryo, not on anything related to immunity. The subsection is titled "Croquemort is not a phagocytic engulfment receptor for apoptotic cells or bacteria", but the text passage contains no mention of phagocytosis of bacteria, and phagocytosis of bacteria is not tested in the S17 supplement. I would suggest deleting this passage entirely if there is not going to be any discussion of the immune-related phenotypes.

The reviewer is correct and we have in the revised version removed ‘bacteria’ from the title. We have kept the section S17 because this statement ‘Crq recognize and uptake apoptotic cells’ (Science, Immunity)’ is still a source of confusion while the role of crq is more downstream. We agree that efferocytosis might not considered as part of innate immunity but we consider these articles were important in the Drosophila immunity community at that time.

The claim "Toll is not activated by overexpression of GNBP3 or Grass: Experiments performed for ReproSci find that contrary to previous reports, overexpression of GNBP3 (Goear et al., 2006) or Grass (El Chamy et al., 2008) in the absence of immune challenge does not effectively activate Toll signaling (Supplementaries S6, S7)" is overly strongly stated unless the authors can directly repeat the original published studies with identical experimental conditions. In the absence of that, the claim in the present manuscript needs to be softened to "we find no evidence that..." or something similar. The definitive claim "does not" presumes that the current experiments are more accurate or correct than the published ones, but no explanation is provided as to why that should be the case. In the absence of a clear and compelling argument as to why the current experiment is more accurate, it appears that there is one study (the original) that obtained a certain result and a second study (the present one) that did not. This can be reported as an inconsistency, but the second experiment does not prove that the first was an error.

We have softened our statement and revised the supplement document. The title of the section is now ‘Toll is not strongly activated by overexpression of GNBP3 or Grass’. However we want to keep these supplements. The comment of the reviewer applies to all our assessment as we could be wrong. We are using experimental validation and not replication, although in this present case, we are using very similar approach to the described in the original study. Of note, the fact that over-expression of GNBP3 does not consistently activate Drs was already stated in Buchon et al 2019 and we spend quite sometimes in attempts to repeat this experiment. Finally, the Ferrandon team that produce this claim can easily repeat the experiment. This does not affect the main conclusion of their landmark article on GNBP3.

The same comment applies to the refutation of the roles for Edin and IRC. Even though the current experiments are done in the context of a broader validation study, this does not automatically make them more correct. The present work should adhere to the same standards of reporting that we expect in any other piece of science.

We fully agree with the reviewer but the comments he/she raised apply to all the assessments made in our article and nearly all the reproducibility experiments. Our assessment does not state that we are right but that we fail to conceptually reproduce the claim. This is clearly mentioned in our article. We hope to see clarification in the future. It may be one day shown that i) IRC indeed regulates intestinal immunity, ii) PGRP-LE will be shown to have an extracellular form, iii) 18W could indeed be a PRR for LPS (maybe in a specific tissues), iv) PGRPSD will be shown to function upstream of Toll, v) that Crq indeed binds apoptotic cell to internalize them… Our study highlights statements that are source of debate, but this is far from being clear. The authors of these challenged claims have the opportunity to contradict us.

The statement "Furthermore, evidence from multiple papers suggests that this result, and other instances where mutations have been found to specifically eliminate Defensin expression, is likely due to segregating polymorphisms within Defensin that disrupt primer binding in some genetic backgrounds and lead to a false negative result (Supplementary S20)" should include citations to the multiple papers being referenced. This passage would benefit from a brief summary of the logic presented in S20 regarding the various means of quantifying Defensin expression.

We appreciate the value of citing some of the papers underlying this claim. Key papers in question are discussed at length in Supplement S20, and the original papers do not realize the issue we are reporting. We therefore modified the text as follows: "Furthermore, this result, and other instances where mutations have been found to specifically eliminate Defensin expression, is likely due to segregating polymorphisms within Defensin that disrupt primer binding in some genetic backgrounds, leading to false negative results (Brennan 2007,Neyen 2014 and see Supplementary S20)”. The hypothesis we raise, namely that defensin function may involve specific polymorphisms that are absent from the reference Drosophila genome, is an important consideration that should be communicated to the community.

In S22 Results, the statement "For general characterization of the IrcMB11278 mutant, including developmental and motor defects and survival to septic injury, see additional information on the ReproSci website" is not acceptable. All necessary information associated with the paper needs to be included in the Supplement. There cannot be supporting data relegated to an independent website with no guaranteed stability or version control. The same comment applies to "Our results show that eiger flies do not have reduced feeding compared to appropriate controls (See ReproSci website)" in S25.

The ReproSci website contains additional data that we could not display in the supplement but may be useful to scientists. We believe that it is worth to mention them. We have however changed the writing stating ‘Experiments reported in the ReproSci website’ rather than ‘See ReproSci website’ to make it clear that the data are not part of the supplement.

For IRC: We have added a new section that addresses the general characterization of the IrcMB11278 mutant in the revised version (Supplement 22 Figure 2 and 3 with associated text). We do not refer to the ReproSci website anymore.

Supplement S21 appears to show a difference between the wildtype and hemese mutants in parasitoid encapsulation, which would support the original finding. However, the validation experiment is performed at a small sample size and is not replicated, so there can be no statistical analysis. There is no reported quantification of lamellocytes or total hemocytes. The validation experiment does not support the conclusion that the original study should be refuted. The S21 evaluation of hemese must either be performed rigorously or removed from the Supplement and the main text.

To address reviewers’ comment, we generated two new Hemese null mutants, Hemese^JP187 and Hemese^JP828, carrying deletions in the central region of the gene. Using these lines, we show that wild-type and Hemese mutant larvae display comparable levels of lamellocytes and similar numbers of melanized capsules following wasp infestation. We further quantified lamellocyte differentiation and included appropriate statistical analyses. We have removed the previous results obtained with the Hemese^SK2 mutant, which yielded similar observations. By reproducing our previous findings with independent mutant lines and by providing quantitative analyses, we strengthen our conclusion that Hemese does not act as a negative regulator of lamellocyte differentiation upon wasp infestation.

In S22, the second sentence of the passage "Due to the fact that IrcMB11278 flies always survived at least 24h prior to death after becoming stuck to the substrate by their wings, we do not attribute the increased mortality in Ecc15-fed IrcMB11278 flies primarily to pathogen ingestion, but rather to locomotor defects. The difference in survival between sucrose-fed and Ecc15-fed IrcMB11278 flies may be explained by the increased viscosity of the Ecc15-containing substrate compared to the sucrose-containing substrate" is quite strange. The first sentence is plausible and a reasonable interpretation of the observations. But to then conclude that the difference between the bacterial treatment versus the control is more plausibly due to substrate viscosity than direct action of the bacteria on the fly is surprising. If the authors wish to put forward that interpretation, they need to test substrate viscosity and demonstrate that fly mortality correlates with viscosity. Otherwise, they must conclude that the validation experiment is consistent with the original study.

The viscosity of a vial can easily be assessed by direct observation. Here, we provide an information that may guide further research but this is by no way a strong point of our article. Thus, we did not do any change to address this comment. We agree that the function of IRC should be entirely re-visited. We have however changed the name viscosity for stickiness to be more neutral.

In S27, the visualization of eiger expression using a GFP reporter is very non-standard as a quantitative assay. The correct assay is qPCR, as is performed in other validation experiments, and which can easily be done on dissected fat body for a tissue-specific analysis. S27 Figure 1 should be replaced with a proper experiment and quantitative analysis. In S27 Figure 2, the authors should add a panel showing that eiger is successfully knocked down with each driver>construct combination. This is important because the data being reported show no effect of knockdown; it is therefore imperative to show that the knockdown is actually occurring. The same comment applies everywhere there is an RNAi to demonstrate a lack of effect.

Eiger expression: Previous RNA-seq (Flyseq, Troha et al., 2018) or Affymetrix array using poly (A)- RNA (De Gregorio et al., 2001) have never identified eiger as an immune-induced gene. We have used a previously characterized GFP reporter gene using the same approach done by the author to attempt to see an induction in the fat body without success. Our conclusions have been nuanced.

Eiger RNAi KD: The two RNAi lines we are using have already been used in many studies assessing the function of eiger ((#108814, reference here, #45253 reference here)

The Drosomycin expression data in S3 Figure 2A look extremely noisy and are presented without error bars or statistical analysis. The S4 claim that sphinx and spheroid are not regulators of the Toll pathway because quantitative expression levels of these genes do not correlate with Toll target expression levels is an extremely weak inference. The RNAi did not work in S4, so no conclusion should be inferred from those experiments. Although the original claims in dispute may be errors in both cases, the validation data used to refute the original claims must be rigorous and of an acceptable scientific standard.

S3: The inducibility of antimicrobial peptide genes is high, albeit very variable. Importantly, we do not observe any inhibition of Drs expression at the two time points (24 and 48 h) following infection with two distinct bacteria, M. luteus and E. faecalis. Although each experiment was performed once, this corresponds to four independent tests challenging the claim from my lab that “Spheroide is required for activation of the Toll pathway.” Furthermore, if Spheroide played a significant role in Toll pathway activation, increased susceptibility to E. faecalis, B. subtilis, and B. bassiana would be expected. However, both survival data (three independent experiments) and Drs expression profiles (four conditions) consistently indicate that Spheroide is not required for Toll pathway activation. For clarity, we have removed the data related to S. aureus.

S4: To address the reviewer’s comments and reinforce our conclusion, we generated a deletion removing both sphinx 1 and sphinx 2 and analyzed Toll pathway activity in the absence of these two serine proteases. We find that flies lacking sphinx 1 and sphinx 2 exhibit wild-type induction of Drosomycin following M. luteus infection, as well as normal survival upon E. faecalis challenge. These new data provide strong evidence supporting our claim that, in contrast to our previous report (Kambris et al., 2006), Sphinx 1 and Sphinx 2 do not regulate the Toll pathway. Accordingly, we have replaced the earlier RNAi-based results with data obtained from loss-of-function mutants.

In S6 Figure 1, it is inappropriate to plot n=2 data points as a histogram with mean and standard errors. If there are fewer than four independent points, all points should be plotted as a dot plot. This comment applies to many qPCR figures throughout the supplement. In S7 Figure 1, "one representative experiment" out of two performed is shown. This strongly suggests that the two replicates are noisy, and a cynical reader might suspect that the authors are trying to hide the variance. This also applies to S5 Fig 3. Particularly in the context of a validation study, it is imperative to present all data clearly and objectively, especially when these are the specific data that are being used to refute the claim.

S6: The figure 1 of S6 shows all the points. Collectively, this represents 6 replications using 3 drivers of the claim ‘over-expression of GNBP3 efficiently activate the Toll pathway’. The observation, we are now reporting in S6, was already mentioned in Buchon et al 2009 (PNAS). To address reviewer’s concern, we have removed the standard deviation. Finally, our conclusion is rather soft ‘Overexpression of GNBP3 either ubiquitously or in the fat body does not effectively activate the Toll pathway’. At most, we find similar to (El Chamy et al., 2008) that overexpression may produce ‘weak but detectable’ activation of Toll’.

S7: We could not address this point. The point stands, there is no condition where overexpression of native Grass corresponds with higher Drosomycin expression

Other comments:

In S26, the authors suggest that much of the observed melanization arises from excessive tissue damage associated with abdominal injection contrasted to the lesser damage associated with thoracic injection. I believe there may be a methodological difference here. The Methods of S27 are not entirely clear, but it appears that the validation experiment was done with a pinprick, whereas the original Mabary and Schneider study was done with injection via a pulled capillary. My lab group (and I personally) have extensive experience with both techniques. In our hands, pinpricks to the abdomen do indeed cause substantial injury, and the physically less pliable thorax is more robust to pinpricks. However, capillary injections to the abdomen do virtually no tissue damage - very probably less than thoracic injections - and result in substantially higher survivals of infection even than thoracic injections. Thus, the present manuscript may infer substantial tissue damage in the original study because they are employing a different technique.

The reviewer is correct that injection and pinprick injury do not induce the same type of injury and there are differences across labs. We still believe that our approach is valuable. The point is that we confirmed a role of eiger in melanization but not due to its expression in the fat body. It is important to underline that our ReproSci is a conceptual replication project and that we never directly challenge the original observation but rather the robustness of the claim. Injection with a pulled capillary was attempted — in our hands, this caused extensive trauma and killed over 80% of flies it was attempted on. The pinprick on the other hand caused very little trauma. The first sentence of the Results/discussion section states this point.

Recommendations for the authors:

Reviewer #1 (Recommendations for the authors):

It is questionable whether all the studies relevant to the field of assessment have been identified, starting with the own publications from the lead authors. Indeed, Buchon et al, Genes & Development, 2009, Buchon et al., Cell Host&Microbe, 2009, Buchon et al., BMC Biology, 2010 have not been included in the 400 articles, even though one of them is cited when discussing the role of Duox in the intestinal host defense. Other missing articles that fulfill the criteria set by the authors include, but may not be limited to, Chung et al. JBC, 2011, Nehme et al., PLoS One, 2011, Cronin et al., Science, 2009, Limmer et al. 2011.

In our selection, we did not choose articles dealing with intestinal homeostasis and stem cells as well as microbiota/symbionts (Buchon 2009a,b and 2010 Cronin 2009). We also did not analyze articles centered on virulence factor (Limmer 2011). I agree that we missed Chung et al. JBC, 2011 and Nehme et al., PLoS One, 2011 that were published in 2011. Table S1 provide all the information on the article we selected and the ones we removed from the first search.

One area of the field that would have benefited from further insights of ReproSci is that of innate immunity memory or priming (Pham et al., PLoS Pathogens, 2007, has been included in the 400 articles). That being said, the scope of this article is already considerable, and this is just a suggestion for future scrutiny.

The reviewer should recognize that this study (Pham et al.) is inherently complex to reproduce. The field of innate immune priming has also matured considerably since the first publications. Importantly, the set of articles containing “non-validated claims” does not fully overlap with the category of studies that might be considered as weak or problematic.

A major issue in science is the exaggeration of claims, which our approach does not capture. As a result, some articles that are strongly promoted yet make relatively trivial claims— often a source of frustration within the community—may still be classified as valid under our framework. This represents an important limitation of our approach.

For the Supplements, it would be better to show all the data points, especially with respect to RTqPCR data. The nature of the error bars (SD, SE) is never indicated except for one exception (Figure 5, S28). When introducing or using novel lines affecting gene expression, such as RNAi lines or KO mutants, there are often no indications as to how these lines have been validated.

This is especially critical for negative results.

Errors bars: The errors bars were standard deviation. Sorry for this lacks of clarity. This is now indicated in the supplement. Experiments were done in multiple laboratories requiring complex coordination. For this reason, changing all the graphs was too complicated and could not be done for all the figures in the revised version. However data points are shown in many figures of the original draft and all the new figures of the revised version.

For RNAi validation: Concerning the RNAi validation:

(a) Sphinx ½: We have extended our work on Sphinx using a newly generated double

mutant that remove both Sphinx1 and 2. Thus our conclusion does not rely anymore only the use of RNAi.

(b) S12: Hep RNAi : We are using standard Hep-RNAi and BskDN fly tools that have been validated in many studies.

Articles validating the HepRNAi (BDSC35210) can be found on this link. This line has also been validated in Figure 1 of the revised version of Supplement S12.

Articles validating the BskDN (BDSC6409) can be found on this link.

Of note, the Bloomington number of Hep RNAi was miss-annotated and has been corrected in the revised version.

(a) S16. Dscam: The Trip RNAi lines has been used in other articles (ex. Bui et al., “Adjacent neuronal fascicle guides motoneuron dendritic branching 2024 (eNeuro)).

(b) S23. Duox. We are using two RNAi lines including the original RNAi construct used in the article we assessed (Ha et al., 2005).

(c) Eiger: we are using two distinct UAS-RNAi constructs that have been validated (#108814, reference here, #45253 reference here).

Loss-of-function mutations:

For previously described mutants: we used some previously described mutants available in stock centers or from other labs. These include Spheroide^D104, wntD mutants Irc^MB11278, NOS^Δall and Gr28b. See references in the supplement. We have always checked the genotypes of these mutants.

For newly generated mutants

spirit^DR32 see Figure 1 in S2

edin^KO: we have added a new figure describing the mutant in S14 D

ΔSphinx1/2 we have added a figure describing the mutant in S4

DNaseII^sk4: the exact frameshift mutation is indicated in S11

SR-CI^SK6 and Δ(Sr-CI,Sr-CIII): we have added a figure describing the mutant in S18

Hemese^JP187 and HemeseJP⁸²⁸ we have added a figure describing the mutant in S21

Listericin^Δ1 and Listericin^Δ2 mutation are described in Figure 1 in S15

We have now included in the revised supplement a table with all the new tools generated in the framework of the ReproSci project (Supplementary Table 4).

The only uncharacterized mutant is PGRP-LE^Δ53, a gift from François Rouyer (Gif-sur-Yvette). But we also did our experiments with an established mutant: PGRP-LE¹¹².

Is the repeated use of "in vitro" to refer to cell culture studies appropriate? In vitro usually refers to experiments performed in the absence of any living organism or cell, e.g., biochemical reactions in a test tube.

Accordingly, we have replaced in vitro by in cell culture when appropriate

(A) MAIN TEXT

(1) Introduction:

PGRP-LC: Ramet et al, Nature 2002 reference is missing.

"[...]epithelial repair through stem cell proliferation" is a typical disease tolerance mechanism and therefore unlikely to "[...]contribute to DETERRING ingested pathogens". In addition, is "detterring" really the appropriate word?

The changes have been made.

(2) Results:

GNBP1: "and find no evidence" would read better as "and found no evidence". In addition, for GNBP3, the authors may also want to cite Mishima et al. (JBC, 2009) and possibly one on silkworm ßGRP (Takehasi et al., PNAS, 2009).

The changes have been made.

Wnt D: "Dif" and not "dif"=diffuse irregular facets

The changes have been made.

cGLR-STING pathway (Cai et al., 2022): subsequent studies indicated that cGLR ligands are RNA molecules, not DNA. Thus, whether such receptors would be able to detect naked DNA remains a remote possibility.

We agree and have amended the text accordingly

DSCAM1: the statement "not amenable to genetic studies" might be modulated into "not easily amenable to genetic studies" as exemplified by Dong et al. PLoS Biol, 2025 (which the authors may now add to their reference list as this article was published after submission of the current work) and all the literature on DSCAM and the Drosophila nervous system.

We agree and have amended the text accordingly

Psidin: "multiple papers suggest...". References are not given.

We have amended the text and reported to the supplement. Some references have been added.

NOS: "this study reports" should be "this study reported"; otherwise, the reader falsely believes that the authors refer to the Reprosci article. NOS expression: what about data from subsequent RNAseq studies and data from FlyCell Atlas?

We agree with the reviewer and used the past time. For simplicity, we did not discuss Nos expression pattern.

Eiger: the Kodra et al. reference appears to be missing in the reference list.

The Kodra reference has been added.

(3) Discussion:

Third source of irreproducible claims: besides off-target effects, a limitation of the RNAi approach is that one can never be sure that a null phenotype is obtained. Thus, when obtaining negative results, it may be caused by a hypomorphic effect.

This additional point has been added with an example that illustrates the point “Besides off-target effects, a limitation of the RNAi approach is that one can never be sure that it mimics a full null phenotype. As an example, the partial silencing of the serine protease gene Grass may explain why its role in the antifungal response was not detected using in vivo RNAi, but was revealed through the use of a null mutation (El Chamy et al., 2008; Kambris et al., 2006b)”.

(4) Reference list:

Cuttell et al, Gordon et al, 2005, and Melcarne et al, 2019 are duplicated.

Thank you for noticing it. Change has been made.

(5) Supplementary materials:

Table S2 and S3 appear to have been interchanged.

This has been corrected.

Ouyang et al: BioRXiv should be indicated.

This has been corrected.

(B) INDIVIDUAL SUPPLEMENTS:

(1) S1: enzymatic activity of GNBP1.

The number of independent experiments does not appear to be mentioned. With respect to the western blot shown in Figure 2A, it seems that the intensity of the band corresponding to activated ModSP is lower in lane 1 than in other lanes. The suggestion is to scan multiple blots to quantify the activation of this protein in different conditions. Do the authors know the nature of the high mobility band that appears solely when S. aureus peptidoglycan is added?

Regarding the lower intensity of the activated ModSP band in lane 1, a similar phenomenon has been reported in An evolutionarily conserved serine protease network mediates melanization and Toll activation in Drosophila (Fig. S2), where ModSP autoactivation was observed following incubation with PGRP-SA, GNBP1, and peptidoglycans. In current experimental design, the primary objective was to determine whether preincubation of peptidoglycans with GNBP1 further enhances ModSP activation. Therefore, our analysis focused on comparisons across different incubation time points rather than direct comparison with lane 1. Within this context, the lower intensity of the activated ModSP band in lane 1 is expected and does not affect the interpretation of the results

In response to the suggestion of scanning multiple blots for quantification, we note that no clear enhancement of ModSP autoactivation was observed across different incubation times. We also explored alternative incubation conditions, but none resulted in a detectable increase in activation. Thus, additional quantification would not change the overall conclusion.

(2) S2: Spirit

It would have been nice to display the spirit mRNA levels in the KO and KD mutants. Is the catalytic domain removed by the deletion and can the authors exclude a potential expression of a truncated Spirit from a cryptic translation initiation site upon leaky Gal4 expression of the transgene, hence the need for RTqPCR data using primers that map 3' to the transposon insertion?

We are using a deletion that remove all the Spirit isoforms that all start with the same ATG which was removed in the mutant, so it's very unlikely. There is an HSP70T = Terminator after the Gal4, which should exclude an initiation of translation after Gal4. However, we have verified the expected mutation by PCR.

Figure 2: Crystal cells appear to be present in spirit larvae. They, however, appear to be smaller, which may be difficult to measure. Quantification of the number of crystal cells is necessary.

We have removed this part in the revised version (Figure 2C and associated text) because it does not concern the assessed claims in Kambris et al et., 2006 that Spirit regulates the Toll pathway.

Figure 3: It is not clear if pooled data are presented for survival experiments.

Yes survivals show pooled data. We have updated figure legend to make this clearer.

(3) S4: Sphinx 1/2

In the absence of testing null mutant flies, the conclusion may be too strong. Can the authors totally exclude the expression of these proteases in a few cells? If they function as apical proteases, a minimal expression may be sufficient to trigger a proteolytic cascade.

To address the reviewer’s comments, we generated a deletion removing both sphinx 1 and sphinx 2 and analyzed Toll pathway activity in the absence of these two serine proteases. We find that flies lacking sphinx 1 and sphinx 2 exhibit wild-type induction of Drosomycin following M. luteus infection, as well as normal survival upon E. faecalis challenge. These new data provide strong evidence supporting our claim that, in contrast to our previous report (Kambris et al., 2006), Sphinx 1 and Sphinx 2 do not regulate the Toll pathway. Accordingly, we have replaced the earlier RNAi-based results with data obtained from loss-of-function mutants.

(4) S5: wntD

Figure 1, if any, appears to be missing.

Thank you for noticing this. This is now corrected Figure 2=>Figure 1 and Figure 3=> Figure

Fig. 3C-C': please, homogenize the color code with other panels (yw in blue and not suddenly in orange)

We have homogenized the color code in the revised version.

(5) S6-S7: Activation of Toll by overexpression of GNBP3 or protease genes The point is not framed appropriately: the overexpression studies were not intended to determine whether the overexpression of these genes activates the Toll pathway to levels of activation encountered during infection, but as tools for epistatic analysis, and in the case of S8, to identify novel protease genes that might be involved in Toll pathway activation. The results of the epistatic analysis are not disputed and, in some cases, have been validated by independent approaches (e.g., by hemolymph transfer experiments), and the hits of the protease screen have been validated by other laboratories, namely grass and SPE. In Figure S5B of Gottar et al, Cell, 2006, a level of only 30% of immunized was achieved, whereas it was about 70% in Figure 3A. Thus, the level of induction by overexpression of GNBP3 is variable but nevertheless sufficient for epistatic analysis both for Gottar et al. and for El Chamy et al., Nat. Immunol, 2008. Furthermore, levels of activation sufficient for epistatic analysis of Toll pathway activation or stimulation of the melanization proteolytic activation cascade upon the overexpression of either GNBP3 or GNBP1 together with PGRP-SA have been reached (Gobert et al., 2003; Gottar et al. 2006; El Chamy et al, 2008; Matskevich et al., Eur. J. Immunol., 2010). Thus, the overexpression of GNBP3 functions for genetic analysis purposes in two independent teams, that however, are working in the same research Unit. While one cannot exclude that enough peptidoglycan metabolites, e.g., TCT, are provided by the microbiota, it cannot be excluded that ß-(1-3)glucans might originate from the food or mycobiota, which may be distinct from that in the Reprosci laboratory. This illustrates the difficulty of controlling all parameters.

We fully agree that the notion that GNBP3 functions upstream of serine protease is validated. The point is that we cannot reproduce the observation that overexpression of GNBP3 trigger a strong Toll activation sufficient for epistatic analysis as shown in Gottar et al., 2006. The point is that ReproSci does not challenged any of the major claims of Gottar et al. 2006 but only a minor claim. The magnitude of gene induction relative to the unchallenged condition is an important consideration in the field of Drosophila immunity and can be misleading. For example, a gene described as being induced fivefold may appear significant, whereas the same gene can be induced up to 1,000-fold following infection. Therefore, absolute expression level compared to challenged level, rather than fold changes alone, should be carefully considered when interpreting immune gene activation.

We have done our studies with two different constructs. The observation that overexpression of a gene activate an immune pathway is by itself a result suggesting a concentration-dependent mode of activation. For instance, overexpression of ModSP is sufficient to activate the Toll pathway (Buchon 2009). This was not the case for GNBP3, even at low level. We cannot exclude that this could be the case in a specific background but at least not all background. Over-expression of GNBP3 is used in 3 panels of Gottar 2006 and this was a significant piece of data. On my side, my lab has lost significant amount of time to reproduce this without success.

Now we could be wrong in our assessment but we let the author of this study to show it.

To take in consideration Reviewer’s comment, we are softened our conclusions : ‘Overexpression of GNBP3 either ubiquitously or in the fat body does not effectively activate the Toll pathway’. At most, we find similar to (El Chamy et al., 2008) that overexpression may produce ‘weak but detectable’ activation of Toll’.

References:

Alphabetical order is not respected (Matskevich before Kim), and the Mishima reference is not optimal: it should be the JBC article in the same year that documents also the biochemistry of binding to ß-glucans.

We have done the changes in the revised version

(6) S8: PGRP-LE not in hemolymph

Fink et al., Mucosal Immunol. 2016 used the NP1-Gal4, which is not expressed in enteroendocrine cells, as stated in the text, but in enterocytes (and possibly the nervous system).

For controls, is the wild-type control the w[iso] Drosdel background?

Please provide the supplier of the GFP antibody.

We have addressed all the points in the revised version.

(7) S9: PGRP-LE and melanization

Figure 1D-E: A quantification would bolster the claim.

We agree that quantification of Figure 1D-E would reinforce the claims but we believe that our data are sufficient to state that PGRP-LE does not block the cuticular melanization upon septic injury (which is also consistent with an intracellular role). We have softened our conclusion.

(8) S10: PGRP-LE and bacillus

Please provide a description of the Bacillus subtilis strain.

This is a strain that derived from the Jean Lambert collection of Strasbourg and used in Lemaitre B PNAS 1997 (J.Millet and A. Klier, Pasteur Institute of Paris). We have checked the strain and confirm that it was indeed a B. subtilis strain that we name B subtillis FS.

It is not clear from the Mat & Meth whether the 2-3 independent replicates per genotype represent biological replicates in one experiment or correspond to several independent experiments. Are the pooled data displayed?

This information was on the methods. It has been copied to the figure caption.

(9) S11: DNAse II and regulation of AMP genes

The exact larval stage is not provided in S11, nor in the original study.

The experiment was done on wandering third-instar larvae as indicated in the material and methods

The number of flies for survival experiments is rather low, but this is not a major issue given the outcome of the experiment.

We agree with the reviewer.

What is the concentration of the injected DNA? As the solution was likely viscous, was the DNA sheared?

We have used a concentration of DNA of: 200 ng/ul, which is now indicated in the revised version of the Supplement. DNA was no sheared.

Figure 1A&D: A positive control of immunization would have been nice. 1D: At which time point was Diptericin expression measured by RTqPCR?

Unchallenged flies were used in Figure 1A while in Figure 1D Diptericin was measured 6 h after challenged. We agree that a bacterial challenge control would have been nice, but clearly the level of Diptericin in 1A corresponds to levels observed in unchallenged flies. Of note, we have tried several different concentrations, of more or less long "sizes" of DNA (short or very long fragments), and also different DNA sources (E. coli or Drosophila) but we did not see any effect on the activation of the Imd pathway.

DNAseII-deficient larvae have wild-type hemocyte numbers. His sub-title is not accurate as Figure 3 actually reports an INCREASED number of hemocytes in the DNase II mutant.

The reviewer is correct and we have adjusted the text.

(10) S12: JNK pathway and AMP gene expression

Hep RNAi lines and possibly also the Bsk-DN line should be validated by monitoring puckered mRNA levels after a stress that activates the JNK pathway. The absence of this control weakens the Reprosci conclusions.

As stated above, we are using standard Hep-RNAi and BskDN fly tools that have been validated in many studies.

Articles validating the HepRNAi (BDSC35210) can be found on this link. We have also validated the RNAi in new Figure 1 of Supplement S12.

Articles validating the BskDN (BDSC6409) can be found on this link

The Bloomington number of Hep RNAi was miss-annotated and has been corrected in the revised version

(11) S13: Caspar

There is clearly a trend for improved survival of the Caspar mutant in Figure 1A. Please, show the pooled data and appropriate statistical analysis. The legend to 1A mentions one overexpression strain that is not displayed and has likely not been performed because loft experiments were performed at 25{degree sign}C, whereas OE experiments should be performed at 29{degree sign}C, hence in separate experiments.

There was a mistake in the original figure legend when we mentioned the impact of overexpression of caspar on survival without showing the data (no effect was observed). The text has been removed in the revised version because it is not related to the claims we assessed from the Caspar article (Kim et al.,2006). To addressed reviewer’s comments, we now present the pooled data in Figure 1 of the revised Supplement S13. The conclusion is that we do not observe any increase host survival in caspar mutant related to wild-type consistent with our observation that the Imd pathway is not strikingly over-activated in absence of infection in caspar loss-of-function mutants.

Why was the c729-Gal4 driver chosen? Has it been validated as a bona fide fat body driver?

The use of c729-Gal4 was suggested by, and borrowed from, the laboratory of Prof. Gaiti Hasan, who had characterized and used it in adult fat body experiments (Subramanian M et al., 2013, Dis. Mod. & Mech). The BDSC number for the corresponding stock is BDSC_6983. Based on Flybase data, it is also expressed in somatic cells of male/female reproductive systems in the adult.

The RTqPCR experiments are not overwhelmingly convincing for a lack of data points: pooled data should be shown, inasmuch as there is a significant difference upon Caspar OE 6h after Ecc15 challenge.

We now show in the revised version of the Supplement S13 the pooled data from different experiments with individual points shown.

In conclusion and although our results are variable, they do not reproduce the results produced in the original article by (Kim et al., 2006) According to our results, we observe:

(1) no increase Dpt expression in absence of challenge both in caspar deficient larvae and adults.

(2) A possible inhibition of Dpt upon over-expression caspar at 6h in adult.

- A possible increased Dpt expression in caspar loss of function mutant larvae upon challenge.

The later may suggest an inhibition role upon overexpression of caspar that required further analysis. The text of the supplement has been amended to take all those points in consideration.

(12) S15: listericin

Introduction: The LLO (also referred to as hly) L. monocytogenes mutant does not prevent the entry of Listeria inside cells. The pore-forming toxin listeriolysin is required for pathogen escape from the phagosome.

There is no reference to LLO in the introduction of the S15 listericin section.

Has the STAT-92E RNAi line been validated? It is difficult to draw a definitive conclusion on the absence of the role of the JAK-STAT pathway in adults using an RNAi transgene because it may only imperfectly silence the expression of the target gene. "Contrary to Goto et al.2010, in our hands reduction of JAK-STAT activity through RNAi [...] in adult flies": Goto et al. apparently tested the JAK-STAT pathway only in S2 cells. This statement should therefore be modulated accordingly.

We have validated the STAT92 RNA in the revised version by confirming that Turandot gene expression is reduced in this mutant (Supplement 15 Figure 2 C-D). We have also validated the PGRP-LE¹¹² null mutant (Supplement 15 Figure 1C).

We have adjusted the conclusion: “Contrary to (Goto et al., 2010), we find that Listericin expression is not specifically dependent on PGRP-LE, and is not dependent on JAK-STAT signaling in flies. Of note, this could be the case in S2 cells where the initial experiments were done.

Antibacterial effects of listericin: It would be cautious not to rely solely on survival experiments, but also to monitor the bacterial titer to draw a definitive conclusion (assay used by Goto et al.). A control overexpressing GFP or a non-relevant peptide would have been welcome to control for potential metabolic effects that would decrease the effect of the overexpression. More importantly, Goto et al. used a Cg-Gal4 driver that is also expressed in hemocytes, which is likely not the case for the Lpp driver. Is it known if the c564 driver is also expressed in hemocytes and to the same level as the Cg-Gal4 driver?

We have used two fat body drivers to overexpress Listerin, Lpp and C564, the latter being expressed in hemocytes indeed. We agree that bacterial counting would provide a sensitive assay but in absence of any protective effect of Listericin overexpression, we do not expect any major titer change in L. monocytogenes growth. To reinforce our conclusion, we have generated Listericin null mutations (Fig 4C) and we have shown that they are not more susceptible than the wild-type to L. monocytogenes.

(13) S16: DSCAM1

The authors use an RNAi construct to knock down DSCAM1 expression. Which part of the transcripts is targeted? Did the authors verify that silencing was effective and would affect all DSCAM transcripts?

The DScam RNAI (BDSC #38945) that we are using has been used in other articles (link) and is validated in Kamiyama et al., Dev cell (Figure S4).

Data points should be shown in the figure.

Unfortunately, we could not address this point to the difficulty to reach the collaborator who provided the data.

Their conclusions would be strengthened by directly testing opsonization using the assay developed by Haller et al., EMBO Reports, 2018. Also, to derive the phagocytic index, the authors must have counted the number of hemocytes retrieved from bled larvae? Was there a decrease when DSCAM1 was silenced in hemocytes? The use of a hml-Gal4-Gal80ts driver would have allowed to bypass the issue of developmental effects.

We expect an opsonin to be abundant in the hemolymph but we did not find any secreted form of dScam in our proteomics analyses (Rommelaere 2024). 20 years after the publication of this landmark article in Science, the absence of any confirmation is puzzling and we are reluctant to do any additional experiment taking into consideration that phagocytosis assays are often variable.

(14) S18: SR-CI

The major issue with this supplement has been dealt with in the Public Review: no phagocytic receptor function was claimed in the Ramet et al, 2001 study.

We fully agree with the reviewer and with Mika Rämet and Monty Krieger (personal communication), and we apologize for this misunderstanding. In fact, Rämet et al. (2001) did not present data demonstrating a role for Sr-CI in phagocytosis per se in his article, but rather showed that this receptor is involved in bacterial binding. Our confusion arose from a subtitle in the Results section of their article (“SR-CI Binds Both Gram-Positive and Gram-Negative Bacteria and Is Necessary for Optimal Phagocytosis by S2 Cells”) and the article title that claim that Sr-CI is a pattern recognition receptor, which does not reflect the content of the article.

To clarify this point, we have revisited the original claim. Consistent with Rämet et al. (2001), we now observe a defect in hemocyte binding to bacteria in Sr-CI mutants. Thus, we validate the claim. Because Sr-CI is often assumed to function as a phagocytic receptor, we have also retained the experiments showing that it is not essential for the phagocytosis of E. coli or S. aureus. All experiments have been repeated during the revision process using a null single mutation and a double mutant removing both Sr-CI and Sr-CIII.

Introduction: Is it really adequate to mention a phagocytic receptor for a ligand, dsRNA, that gets endocytosed?

In the introduction, we summarized what is known on the Sr-CI, including its implication in the uptake of dsRNA. As suggested by the reviewer, we used the term endocytosis to refer to the ability to uptake dsRNA.

Methods: Have the authors excluded the potential off-target effect of CRISPR-Cas9 in other scavenger receptor genes? Monitoring their expression levels would also allow the detection of any compensatory effect in the null mutant by overexpression of other scavenger receptors.

We are now using two mutant lines: one carrying a frameshift mutation (SK6) that specifically affects Sr-CI, and another with a deletion removing both Sr-CI and Sr-CIII. We have carefully validated both mutations by genomic sequencing. As our results are now consistent with the original findings of Rämet et al. (2001), we believe that the analyses presented in the Supplementary Information are genetically well supported.

Results: Figure 1 is really confusing with so much variability in the data, which raises doubts about the reliability of the method. The reduction of the phagocytic index in B is not "mild". Again, one would like to see the individual data points and to know what the error bars are. For 1G, has any statistical analysis been performed? Finally, Panels A to G lack homogeneity in their presentation. Also, why are the absolute values of the y-axis varying so much from panel to panel (A-B vs C-D; E vs. F)?

We have repeated our experiments using a new set of mutants, both generated in the isogenic DrosDel background. In the revised version, we include additional experiments assessing the contribution of Sr-CI to hemocyte binding to bacteria. Consistent with the original report by Rämet et al. (2001), we observe similar results. We have revised the text accordingly and moderated our claims. In addition, we performed further biological replicates and now provide appropriate statistical analyses.

(15) S19: Undertaker

Undertaker has been identified through a deficiency screen that restricted the genomic zone of interest to eight genes. RNAi was then used to identify the candidate gene as being Undertaker. While an off-target effect of RNAi cannot be excluded, it cannot account for the results of the deficiency screen (junctophilin is on the second chromosome, whereas undertaker is on the third). The deficiency phenotype could be rescued by an Undertaker transgene. While overexpression can indeed provide a rescuing activity due to high levels of ectopic expression, why would it do so in the context of an 82F deficiency, unless this deficiency carries a second site mutation, which is a rather remote possibility.

Given that the deficiency line deletes at least 14 genes in a cluster, several of which are involved in growth and development, we assume that another gene(s) within the span of this deficiency is responsible for the effect on efferocytosis, or that some other mutation in the uncontrolled genetic background is responsible. The impact of Df(3R)3-4 on efferocytosis remains to be investigated.

The authors claim that Undertaker is not expressed at the right embryonic stage and hardly in S2 cells, which is correct according to large-scale data. However, Cuttell et al. successfully stained S2 cells with an antibody raised against retinophilin/undertaker by the investigators who studied the role of this gene in the eye (Mecklenburg et al., 2007). Should we trust more largescale datasets based on transcriptomics or a specially developed tool? (Both junctophilin and undertaker are at the limit of detection of the proteomic data available on FlyBase; equivalent signals in hemocytes were measured in FlyCellAtlas). Thus, the argument of the absence of expression is not definitive.

We agree on this but the expression pattern is an additional element that reinforce our conclusion. This is not the key element that relies on the use of loss-of-function mutation. Another element is the absence of follow-up study in the last 18 years.

The new phagocytosis data set generated by Reprosci shows an absence of impact of the undertaker loft mutation in larval hemocytes. Whether these data can be extrapolated to S2 cells, embryonic or adult hemocytes, remains debatable, as are the explanations put forth by Reprosci in their conclusion to explain by a possible off-target effect of RNAi on junctophilin. It would be interesting to determine the phenotype of junctophilin mutants in the ex vivo larval hemocyte assay.

We have analyzed the role of Undertaker using third instar larvae in an ex vivo assay and did not find any role in efferocytosis. I think that all our data and analysis of the literature is consistent with our statement that’ that undertaker is unlikely to have a role in phagocytosis’. Now, we can never exclude that Undertake may play a subtle role in this process. The role of Draper has been observed in larval hemocyte that represent macrophage of flies. We are using a null mutation in undertaker preventing the existence of off-targets. We have nuanced our conclusion in the article and add a sentence in the conclusion of the supplement: ‘We cannot however exclude that Undertaker play a role at a specific stage of development that we did not assess.’

(16) S20: Psidin

This supplement is duplicated

The duplicated supplement has been removed.

Phagocytosis assay: the authors use E. coli-GFP. How can they discriminate between internalization and simple binding at the surface of hemocytes? It would have been more appropriate to use a phalloidin, stained to another color rather than green. The suggestion here is to first use FITC-labeled bacteria, followed by Trypan blue treatment to quench the fluorescence of noninternalized bacteria. Second, pHrodo bacteria could be used to monitor the acidification of the phagosome in wt and psidin hemocytes.

We agree with the reviewer that this would be better. Because our observations are consistent with the data reported in this original article, we did not feel inclined to extend our analysis.

(17) S21: Hemese

The pie charts in Figure 1 would benefit from a statistical analysis. Why do the encapsulated eggs appear to be much larger in the hemese mutant? Could it be that more lamellocytes are recruited and make a thicker capsule? Should Figure 1C be imperatively quantified?

To extent our analysis, we generated two new Hemese null mutants, Hemese^JP187 and Hemese^JP828, carrying deletions in the central region of the gene. Using these lines, we show that wild-type and ^Hemese mutant larvae display comparable levels of lamellocytes and similar numbers of melanized capsules following wasp infestation. We further quantified lamellocyte differentiation and included appropriate statistical analyses. We have removed the previous results obtained with the Hemese^SK2 mutant, which yielded similar observations. By reproducing our previous findings with independent mutant lines and by providing quantitative analyses, we strengthen our conclusion that Hemese does not act as a negative regulator of lamellocyte differentiation upon wasp infestation.

(18) S22 IRC

IRC is actually not a catalase but a heme peroxidase. For the experiments on C. albicans, can the authors exclude that the IRC flies succumb because they are more sensitive to ethanol than wt or spz flies (sucrose is fermented by C. albicans): a control would be to use glycerol (nonfermentable) instead of sucrose solution.

We thank the reviewer and Carolina Barillas (personal communication) to inform us that IRC is not a catalase but an heme peroxidase. This is now included in the article and the supplement. It is clear that the name IRC for immune regulated catalase is misleading. We cannot exclude that the IRC flies succumb because they are more sensitive to ethanol than wt or spz flies but we did not test this interesting idea during the revision.

Do IRC mutants succumb to the ingestion of killed bacteria?

No, they get stuck in fly food and die.

(19) S23 DUOX

Introduction: When citing Kumar et al., the authors might also want to cite Bai et al. on the Bactrocera dorsalis peritrophic matrix.

This reference has been added

Results:

Figure 1: any statistical analysis? Data points are not shown for the lef panel, and error bars are not defined.

We have added in the revised version ‘When using the Mann-Whitney U test, we still observed a statistically significant difference in CFU counts at 6 hpi in Figure 2B, as well as at both 6 hpi and 24 hpi in Figure 2F. However, the magnitude of these differences is relatively small’. Error bars have been defined in the general revision of the supplements (see above).

Figure 2A shows only a limited reduction in Duox expression at the transcriptional level. "successfully" might be a bit of an overstatement.

We have removed the term ‘successfully’.

Figure 2E is too small.

We have increased the size of Figure 2E.

The authors might want to discuss the role of Duox in enterocytes destined to die as described by Amchelavsky et al., Cell Reports, 2020.

Good point, we have discussed this reference in the revised version.

The Sajjadian reference is incomplete.

The reference has been completed

(20) S24: NOS

Maybe the work from the Royet laboratory could be cited along with that of Neyen et al. and Zaidmann-Remy et al. in the Introduction.

Good points. We have added two references related to the work of the Royet lab (Bosco-Drayon et al., 2012; Charroux et al., 2017).

It is not clear from Rabinovich et al. (2016) which line should be the relevant wild-type control. Why use w[1118] in Figure 1 and Ore-R in Figure 2? A more definitive conclusion could be reached upon isogenization or testing a second null mutant in homozygous and transheterozygous conditions?

We agree that ideally, we should have used isogenic wild-type control but in absence of any effect of NOS mutants, we consider that the use of w[1118] and Oregon wild-type control is acceptable, considering that they behave similarly in the assay we have used.

(21) S25 eiger survival

Survival experiments: Are pooled data shown? Have the authors monitored the health status of their fly lines? In some of the survivals with late phenotypes, e.g., egr mutants and E. faecalis, this may be an issue, inasmuch as noninfected or mock-infected controls are not shown (have they been performed?).

The survival analysis showed the pooled of 40 flies and were often done with two sexes. They also include many controls. We cannot never exclude minor effects at late time points, but the critical message is that we cannot reproduced the previously published results.

The survival graphs are difficult to analyze: the authors should definitely avoid light colors such as yellow that do not show up well in a white background.

As stated above, these experiments were done by many scientists in different lab and it was difficult to homogenize the data. Therefore, we did not modify our graphs although we agree with the reviewers. We believe that line patterns and contrast differences between groups guarantee that the readers can still assess the data.

(22) S27: eiger humoral response

Introduction: Have the authors also checked the Flysickseq data from the Buchon lab?

There is no induction of eiger in Flysick. This information along with the reference Troha et al., have been added to S27.

Methods: Have the RNAi lines been validated, and how? Is one more efficient at silencing than the other? This set of data would complement Figure 1 with RTqPCR experiments.

The two RNAi lines we are using have already been used in many studies assessing the function of eiger ((#108814, reference here, #45253 reference here)

Figure 3 colors in survival graphs: see S25.

We did not modify our graphs although we agree with the reviewers. By increasing the magnification, the readers can still assess the data. As stated above, these experiments were done by many scientists in different labs and it was difficult to homogenize the data.

Would it have been worth to also test the transheterozygous Gr28b mutants to avoid any issue of second-site mutations?

We could have extended the genetics approach but, given that both us and another group (Sang) found no effect on feeding in an independent mutant, and extremely high feeding in the claimed 'anorexic' mutant, this would not be relevant to the claim. Future study may tell us if Gr28b has clearly a feeding phenotype.

Reviewer #2 (Recommendations for the authors):

(1) Adjust the discussion to better contextualize subjectivity: Because the choice of claims and papers inevitably involves subjective decisions, consider expanding the discussion to acknowledge this limitation more explicitly and explain how the companion metascience manuscript complements this work.

We have added a section at the end of the discussion to discuss the limitation of our study and we establish more link to the companion article.

(2) Refine language and avoid binary judgments: Terms such as "correct," "incorrect," or similar definitive formulations should be softened. Emphasizing uncertainty and openness to future revision will better reflect the iterative nature of scientific inquiry.

We are softening our language in several of our assessment to avoid binary judgement.

(3) Highlight reproducible findings as well: While the focus on irreproducible or challenged claims is understandable, it may be helpful to balance this by more clearly acknowledging wellsupported claims or areas of strong reproducibility to reinforce the positive impact of robust findings.

We have added two sentences at the end of the introduction and at the beginning of the discussion that most claims are reproducible and that we have focused here on nonreproducible claim.

(4) Clarify how the community can engage with the dataset: Because you provide extensive supplementary material and a large online resource, consider adding more explicit guidance on how researchers can contribute comments, evidence, or corrections, and how these contributions will be integrated.

This point is already addressed in the revised sentence: “We hope that the community-accessible website will encourage researchers to share their perspectives and contribute data from diverse sources, thereby improving objectivity.” In addition, we have proactively engaged with principal investigators to motivate them to contribute to this effort by sharing unpublished results and information. However, it remains unclear to what extent scientists are genuinely interested in reproducibility, as has been noted in other studies on the topic. For example, articles that challenge previous findings are often under-cited, and the presence of contradictory evidence does not necessarily prevent researchers from continuing to cite the original claims. This is clearly a challenge of reproducibility study. Comments, evidence and corrections will be integrated in the ReproSci website. We have already updated this database when revising this article and will continue to do it.

(5) Resolve the date-range inconsistency: The Introduction mentions analysis of papers from 1959-2011, whereas the Methods section cites 1940-2011. This discrepancy should be corrected for consistency.

We have solved this discrepancy by correcting the method section. This should be 1959-2011. Thank you for spotting this mistake.

Reviewer #4 (Recommendations for the authors):

(1) The "disclaimer" subsection should be incorporated into the Introduction, as it is central to the premise of the study.

The disclaimer has been moved at the end of the introduction and one sentence has been added in the abstract.

(2) The inconsistency of graphical representation and statistical analysis of survival and gene expression data across different supplements is distracting and makes the project feel cobbled together. A consistent and unified framework for analysis and presentation would feel more cohesive and rigorous.

The experiments were conducted across multiple laboratories and required substantial coordination. As a result, modifying the graphs at this stage would be technically complex and would require monumental effort for a tiny return. Therefore, we regret that we were unable to implement this change. The most important is that data and analyses are presented transparently, allowing readers to evaluate and interpret the results independently.

(3) The purported roles of eiger and Gr28b in immunity have nothing to do with each other, aside from both having been published by David Schneider's group. I would suggest not combining them into a single section unless there is to be a 'miscellaneous' section that contains work by multiple groups.

We have done two separate sections.

(4) It would be easier to comment on specific passages in the manuscript if line numbers had been included.

Sorry for this. We have included number of lines in the revised version.

(5) The use of the word "elegant" to describe melanization reactions on page 3 is a bit odd and feels out of place.

It has been removed

(6) The sentence "We conclude that PGRP-LE is an exclusively intracellular sensor, with a prominent role regulating the gut immune response (Bosco-Drayon et al., 2012; Neyen et al., 2012)" should be edited to read "as reported in (citations)".

We have edited the sentence.

(7) The sentence "Experiments performed for ReproSci and review of the literature suggest that JNK is unlikely to have a strong direct or systematic role in the regulation of Drosophila AMPs in vivo, and is not required in the adult fat body for AMP expression in response to infection (Supplementary S12)" lacks detail. I realize that the experiments are described in S12, but I would suggest a brief summary in the main text of the article as well.

We have provided more information in the article.

(8) The phrasing of "While many of the challenged claims discussed in this paper are known to be controversial by experts, or have fallen quietly out of current research interests, some are still a source of confusion. This is particularly true for young scientists new to the field of Drosophila immunity or scientists with primary interests outside of Drosophila immunity that use the field as a reference." could be edited for clarity. A more direct phrasing could help, such as "Even in the absence of published refutation, many of the challenged claims were already considered controversial by experts in the field. Others have quietly fallen out of the general current research interest. These may nevertheless still be a source of confusion for young scientists new to the field of Drosophila immunity or scientists with primary interests outside of Drosophila immunity that use the field as a reference."

We have changed the sentence accordingly.

(9) On Discussion page 15, the comma should be removed from "In this article we have not discussed a number of claims, that had no clear follow-up studies and remain unchallenged."

The coma has been removed.

(10) In Methods, the authors "acknowledge that our verification experiments could also be erroneous". It would be appropriate to also include such a disclaimer in the Introduction and/or Discussion.

The disclaimer has been moved at the end of the introduction and one sentence has been added in the abstract.