The ethics of sharing preliminary research findings during public health emergencies : a case study from the 2009 influenza pandemic

During the 2009 A(H1N1) influenza pandemic, a suite of studies conducted in Canada showed an unexpected finding, that patients with medically attended laboratory-confirmed pandemic influenza were more likely to have received seasonal influenza vaccination than test-negative control patients. Different bodies, including scientific journals and government scientific advisory committees, reviewed the evidence simultaneously to determine its scientific validity and implications. Decision-making was complicated when the findings made their way into the media. The normal trajectory of non-urgent research includes peer-review publication after which decision-makers can process the information taking into account other evidence and logistic considerations. In the situation that arose, however, the congruence of an unexpected finding and the simultaneous review of the evidence both within and outside the traditional peer-review sphere raised several interesting issues about how to deal with emerging evidence during a public health emergency. These events are used in this article to aid discussion of the complex interrelationship between researchers, public health decision-makers and scientific journals, the trade-offs between sharing information early and maintaining the peer-review quality assurance process, and to emphasise the need for critical reflection on the practical and ethical norms that govern the way in which research is evaluated, published and communicated in public health emergencies.

During the 2009 A(H1N1) influenza pandemic, a suite of studies conducted in Canada showed an unexpected finding, that patients with medically attended laboratory-confirmed pandemic influenza were more likely to have received seasonal influenza vaccination than test-negative control patients.Different bodies, including scientific journals and government scientific advisory committees, reviewed the evidence simultaneously to determine its scientific validity and implications.Decision-making was complicated when the findings made their way into the media.The normal trajectory of non-urgent research includes peer-review publication after which decision-makers can process the information taking into account other evidence and logistic considerations.In the situation that arose, however, the congruence of an unexpected finding and the simultaneous review of the evidence both within and outside the traditional peer-review sphere raised several interesting issues about how to deal with emerging evidence during a public health emergency.These events are used in this article to aid discussion of the complex interrelationship between researchers, public health decision-makers and scientific journals, the trade-offs between sharing information early and maintaining the peer-review quality assurance process, and to emphasise the need for critical reflection on the practical and ethical norms that govern the way in which research is evaluated, published and communicated in public health emergencies.

Context of research, public health and scientific journals -three solitudes?
Improving population health relies on the generation of knowledge by researchers as well as the communication and translation of knowledge to action by public health decision-makers.During public health emergencies, undertaking rapid high-quality scientific research and communicating research findings is a practical, professional and ethical imperative.In such contexts, the need for evidence is most pressing, but short timelines, limited data and the pressure of competing priorities make acquiring this evidence challenging.The speed at which information is needed by policymakers may be faster than usually possible through traditional mechanisms of research dissemination.As an example, only 7% of studies on severe acute respiratory syndrome (SARS) were published during the 2003 outbreak [1].
There are some public health emergencies in which policies have been based on scientific evidence with levels of uncertainty that may be unacceptable in nonemergencies [2].Some researchers and ethicists have suggested that there is a duty to share preliminary public health investigations and important research findings early during a public health emergency, while others feel this would not be in the public interest [3].
Typically, research is carried out in academic settings, whereas policy decisions are made by politicians or professional staff in government or related agencies.Those who pose and answer research questions often inhabit a world with a different ethos and institutional culture than that of decision-makers, reflecting different purposes.This separation of roles and funding streams ensures that research is shielded, as far as possible, from the competing priorities of the policy environment.However, this can lead to problems with knowledge sharing, translation and integration of evidence into practice [4].Scientific journals, which traditionally control access to the usual means by which health-related scientific evidence is communicated between researchers and policy-makers, represent a third stakeholder group with yet another set of goals and norms.The relationships between the elements of this triad can be tested during a public health emergency when research findings are inconsistent with current knowledge or a priori hypotheses.The following description of events that occurred during the 2009 influenza pandemic will be used to highlight some of the challenges and critical issues that may arise at the interface of three cultures: researchers, public health decision-makers and scientific journals.This analysis is the result of discussion and reflection that took place between those involved with the research and others during and following the events we will describe.While acknowledging that public health agencies may bridge the governmental and academic worlds, sharing some commonalities of organisational culture and constraints with both, for simplicity, we will discuss the issues in relation to the functions of research, decision-making and scientific publication.

An account of the events
In March 2009, a novel influenza strain was identified in Mexico and quickly spread globally [5].Interim analysis of a local outbreak of A(H1N1)pdm09 influenza virus infection in April 2009 in Canada identified an unexpected association between prior seasonal influenza vaccination and increased risk of influenza-like illness (ILI) during that outbreak [6].In order to assess this association based on a laboratory-confirmed outcome, researchers turned to an existing sentinel surveillance network for annual monitoring of influenza vaccine effectiveness [7].Interim analysis of data rapidly assembled from this sentinel system showed an association between prior seasonal trivalent inactivated influenza vaccine and medically attended, laboratory-confirmed A(H1N1)pdm09 virus infection.While the investigators were surprised by the results, they were confident of the robustness of the research methods and processes based on prior seasonal analyses.However, they recognised the need for caution given unique aspects of the pandemic that might have led to bias among those who tested negative, either through a healthy-user bias or confounding by indication.
The researchers recognised that the findings needed to be urgently communicated since, if true, it might not be recommended to receive the 2009/10 seasonal influenza vaccine before the pandemic vaccine was available.The research team rapidly notified and shared preliminary results with public health authorities involved in decision-making and global pandemic planning and response.Throughout the summer of 2009, the findings were shared with several national and international public health bodies through inperson meetings and multisite teleconferences, to ensure they had all the information needed to evaluate the findings and inform their decision-making.During July, two case-control studies (one in Ontario, one in Quebec) and a cohort study (in Quebec) were rapidly developed and conducted to further investigate the findings.In parallel, the initial sentinel study findings were submitted to a peer-reviewed journal on 21 July, with a request for expedited publication.That paper was rejected by the journal in the first week of August, largely on the basis of a lack of biological plausibility for the findings.The journal suggested that further confirmatory studies were required.
The results of the two case-control studies and a cohort study corroborated the findings of the sentinel study, with significant odds ratios of 1.4-2.5 indicating a greater likelihood of cases of A(H1N1)pdm09 having been vaccinated than controls in the case-control studies and a higher risk of infection among those vaccinated in the cohort study.Updated results from the sentinel study and these three additional studies were again presented to decision-makers on 24 August 2009.
A national body then organised an urgent independent peer-review process by international experts.The researchers submitted a confidential detailed report -the first written summary of the full findings from the four confirmatory studies -to the national body in early September and submitted the combined findings to a second journal on 16 September along with the international peer reviews.Both the international panel and the reviewers of the second journal completed their evaluation within three weeks and found no substantial methodological or analytical errors that could explain or dismiss the findings, aside from noting the potential limitations of observational studies.The decision of the second journal was positive, asking for minor revisions.
A few days before the completion of the evaluation by the international and journal reviewers, the results of the study appeared in the media [8,9].The specifics of how this occurred are unknown.Media coverage of the study was extensive and the term 'unpublished' was widely used to question the quality of the evidence [9,10].Pressure was brought to bear on some of the researchers to release the studies publicly through a mechanism other than the traditional peer-review publication process.The authors debated the pros and cons of various avenues for immediate publication.Eventually, the name of the second journal became known to the press.
In the first week of October, an international group convened by WHO reviewed the findings by teleconference.Consistent with the conclusions of all the expert reviews that had taken place thus far, none of the participants of this group identified any specific methodological problems or alternative explanations.The teleconference finished with no consensus on the validity of the results.Despite that, a spokesperson reported the next day that most experts in the teleconference did not seem to believe that the study had found a true link between seasonal vaccination and A(H1N1) influenza [11].
By this time, several other studies showing the opposite but expected outcome of no effect of seasonal influenza vaccination on pandemic influenza risk had been published [10][11][12][13][14][15].
One national immunisation technical advisory committee (NITAG) issued the following statement, despite not having formally received the findings to review, 'The Committee agreed that the available evidence does not indicate that seasonal flu vaccination is a risk factor for H1N1v infection' [16].In contrast, another NITAG reviewed the findings in detail and found merit in them [17].
The second journal subsequently withdrew its interest in publication of the article.The researchers were now in a double bind.Sharing the results outside the peer-review system would give the public and policymakers greater access to evaluate the study and draw conclusions, but would render the studies permanently 'unpublished'.Continuing to attempt to publish in a scientific journal would risk further delay and potentially human lives.In the interest of sharing the details of the studies as soon as possible, they were submitted to a third journal in mid-October 2009, which initially expressed interest, but then made it clear that expedited publication would not occur.By this time, the pandemic vaccine was being rolled out in the country where the study had been undertaken, national policy committees had made and communicated the unequivocal decision to recommend the seasonal vaccine while provincial policy committees issued varying recommendations based on their interpretation of the evidence.The authors decided that publication at that point in time would potentially cause further confusion and potential harm to public confidence in the vaccination.The findings were submitted to a fourth journal through a non-expedited submission process at the end of October 2009.
In total, the results of the four studies were reviewed by at least two NITAGs, two regional or global organisations, three external reviewers for the national public health agency and seven reviewers at journals.The paper was published on 6 April 2010 [18].In the time since the paper was published, no methodological issue that could satisfactorily explain the findings has been identified and other studies have replicated the findings [19,20].

Ethical problématique
These events raised a number of important questions regarding knowledge translation of a controversial finding during public health emergencies from both a practical and ethical perspective (Box 1).While it is routine for public health personnel to make decisions based on unpublished findings during outbreaks, the context is usually very different from the situation we describe.The majority of outbreak investigations are not carried out under the same kinds of pressures, the findings and their implications are rarely as contentious as those described here, and they mostly do not lead to publications in journals.Unexpected or controversial findings can be seen as a rigorous test of the overall system's capacity to evaluate scientific work, deal with uncertainty, rapidly determine the practical and public health implications, translate these into knowledge, communicate risk and maintain transparency throughout the process (in order to ensure that potential conflicts of interest, real or perceived, are in the public domain).Each of the three solitudes represented by academia, public health and scientific journals has different strengths and weaknesses in these regards.If the moral duty to share information is a given, what norms should apply in the complex researcher-policymaker-publisher relationship that would facilitate the appropriate translation of unexpected research findings into practice locally and globally?

Academic motivation
Researchers' careers and credibility are based in large part on their published work.As in this case study, they may feel a moral duty to share their work in confidence but are not willing to sacrifice either their credentials,

Box 1
Proposed changes and outstanding questions regarding the ethics of sharing preliminary research findings during public health emergencies Proposed changes for consideration 1. Open direct lines of communication and minimise barriers between the three distinct cultures of researchers, public health decision-makers and scientific journals.
2. Ensure that emergency planning includes the infrastructure and preparation of all communities for the conduct of research, its evaluation, dissemination and publication.
3. Develop a mechanism to enable rapid peer review and dissemination of controversial or unexpected results during public health emergencies.
4. Elaborate a framework outlining the principles, processes and outcomes to govern the relationship between researchers, funders, publishers, decision-makers and the public during public health emergencies.
Outstanding questions To what degree should advisory committees be transparent about the basis for their decision-making?
2. Should publicly funded researchers be obliged make their findings publicly available in emergencies?
3. What are the mechanisms for social accountability of scientific journals?
4. What is the correct balance between the proprietary rights of researchers and the interests of the public?
5. How should public health decision-makers and scientific journals balance precaution in not accepting unexpected finding that may do harm with fair evaluation of new findings based on their scientific merit during public health emergencies?by allowing their work to remain unpublished and invalidated, or their relationships with scientific journals by questioning the status quo.Academic behaviour in preserving the confidentiality of research findings before publication has been driven by the business requirements of scientific publishing.The Ingelfinger rule denies publication to researchers who release findings to the public before they appear in a journal [21].This approach is followed by most high-impact factor scientific journals, but its utility and feasibility are increasingly being questioned in the Internet age (Box 2) [22].
The limitations placed on release of information before publication may bring researchers into direct conflict with public health imperatives and the public who funds their work.To their credit, several journals have set aside the rule and provided rapid publication mechanisms during the pandemic without jeopardising later publication.The editors at PLoS Medicine stated 'that before the next public health emergency strikes, the scientific publishing establishment needs to ask itself how it can respond in the way the world needs' [23].

Decision-making and transparency in public health and scientific journals
In this example, public health decision-makers dismissed the findings that did not fit with the existing paradigm irrespective of whether they had access to detailed reviews of the research.The decisions were not transparent since the committees deliberated under strict rules of confidentiality.Here the public health decision-makers and scientific journals converged in their processes and outcomes around decision-making.One journal made an explicit judgement to place sustaining confidence in a vaccine at a higher priority than publishing findings about a possible risk that the same vaccine might cause to individuals.In the authors' view, in making this judgement, the journal exceeded their normal role of assessing the desirability of publication of articles that are appropriate for their readership primarily on scientific merit.The public health decision-makers on advisory committees and the scientific journals to which the paper was submitted had access to detailed reviews, and both groups ultimately made their decisions confidentially, based on other considerations.Those other considerations may reflect a motivation in the face of uncertainty to do no harm, but may also have been influenced by financial, institutional and/or political interests.
On the overall observation that new ideas meet with resistance, this is not new; even Nobel prize-winning research has met difficulties in being accepted [22], so it is not surprising to find that unexpected results arising during emergencies receive a lukewarm reception [24].
It is also relevant to public health decision-makers and scientific journals that the events we describe relate to a vaccine.This was not the first vaccine-related public health controversy [23] and it will probably not be the last [25].Previous vaccine scares that have caused great harm started with poor quality, now discredited, research [26].This may have reasonably led to higher standards for quality and certainty for research that questions the safety of immunisation, recognising a different balance in the risk-benefit analysis of publishing poor-quality research that may undermine an essential public health intervention.
How do we define the social responsibility of scientific journals?Most, but not all scientific journals are run as businesses: the choices made by editorial staff maintain the reputation of the journal and ultimately determine its success.Scientific publishing has, however, some unique characteristics.The public funds most of the research and most of the individuals who conduct the peer-review process, as well as the costs of publishing either through library subscriptions or open access author fees.The case could be made for more public accountability and transparency.If an editor feels compelled to ignore the results of peer review for what they perceive to be the public good, could there even be a duty to go beyond the standard peer-review process and to involve others, such as public health authorities and the public, in the decision-making?
The ethics of publication clearly go beyond standard considerations of subject confidentiality, plagiarism and minimising harm, but both incorrect and ethically questionable research have been published in the past [27], and existing guidelines are not transparent about the social responsibilities of scientific journals during public health emergencies [28].

Reflections on the scientific peer review process
The independent peer-review process is considered the gold standard for ensuring research quality in scientific publishing, but it is not without its detractors

Box 2
Quotations on peer review and control of scientific information Ingelfinger rule '… the Ingelfinger rule also serves the "guild" interests of the medical research and public health communities …The distribution of control is at stake here ... a fairly closed fraternity exercises enormous control over what information is known by everyone else, professionals and laypeople alike.They control not only whether the information is considered meritorious, but whether it is known at all ...' (Peter Sandman, personal communication, September 2009) Peer review 'The mistake, of course, is to have thought that peer review was any more than a crude means of discovering the acceptability -not the validity -of a new finding… We portray peer review to the public as a quasi-sacred process that helps to make science our most objective truth teller.But we know that the system of peer review is biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish, and frequently wrong' [26] (Box 2).The normal process for a general interest, non-controversial paper would involve review by two to five peer reviewers before publication.As described above, the studies under consideration were reviewed by at least eight formal committees or review groups (four national, including two NITAGs, one regional, one global) and 11 independent reviewers (three appointed by the Public Health Agency of Canada, five reviewers for Journal 2 and three reviewers for PLoS Medicine).
Scientific journals may in general aim to make scientific merit based on the peer-review process the sole criterion for publication of articles that are suitable for their journal, but a different standard clearly should apply during a crisis and to unexpected results.Scientific journals neither want to spread erroneous results that could cause public harm nor do harm by failing to make important results available.During the public health emergency described here, several studies with negative findings were published in different journals relating to a question that was only of interest because of the leaked but then-unpublished findings.Methodological issues identified with these negative studies included a lack of detail about participants, needed to enable adequate assessment of the potential impact of bias or confounding [29,30].In effect, the peer review process validated and facilitated access to research with results everyone expected to find, but delayed the publication of unexpected research findings.

Information sharing and knowledge translation ethics in public health emergencies
The notion that research is a global public good, coupled with advances in technology, has made data sharing desirable and possible on a scale that had never been feasible before.Many of the largest global funders of research now require data sharing [31], while open access publication has made research findings widely available [32].Fields such as genomics have led, and been well served, by the pre-publication data sharing movements; fields such as public health seem to lag behind [33].This may be an artefact of the professional cultures of these disciplines or it may represent a substantive and justifiable difference.
Advocates of unlimited sample, data and results sharing, as well as those who view research data as the legal and moral property of researchers, recognise that a balance must be struck between the proprietary rights of scientists, the needs of public health and the interests of the public [34].For public health investigations, the process for sharing information for local decision-making (such as during outbreaks of foodborne illness) is well established.While the results of analysis of data assembled during larger emergencies as part of urgent public health investigations and research may be more prone to error, the information needs to be shared, and experience indicates that, with careful tailored processes, the public health benefit can outweigh the risk [35].
While there may be consensus that unpublished scientific findings should be shared with decision-makers during global emergencies, it is not clearly defined how best this should be done or when such findings should be made public.Those who argue for unlimited sharing in public health emergencies based on principles such as reciprocity and solidarity also need to consider how researchers, public health decision-makers, scientific journals and the public relate to the information, and how it is disseminated by the Internet and a 24-hour news cycle.When public health is at stake, information must be shared in a structured and transparent manner that communicates the level of uncertainty and meets the needs of all involved.
The issues outlined here cannot be resolved by merely referring to a theoretical moral duty-to-share or by appealing to professional codes of ethics or legal norms.All of the stakeholders involved need a pathway that accommodates each domain's needs and constraints.The complexity involved demands a carefully thought-out framework outlining the principles, processes and outcomes that would govern a paradigm shift in the relationship of researchers, funders, scientific journals, public health decision-makers and the public during public health emergencies.Conflicts and communication failures may be minimised if emergency planning includes infrastructure and preparation of all these communities for the conduct of research, its evaluation, dissemination and publication.Alternatively, creating a mechanism that allows for exceptional circumstances, similar to that of the United States Food and Drug Administration (FDA) emergency use authorisation mechanism, or the European Medicines Agency emergency procedures, with well-defined criteria and parameters [36,37], may help facilitate more effective communication.Such a mechanism would ideally enable a comprehensive risk-benefit analysis to be carried out during an emergency, taking into account rapidly emerging but conflicting findings, their critical methodological appraisal and the potential good versus harm to be accrued at various decision points.This could serve the dual functions of arriving at thoughtful decisions and also explaining reassuring messages to gain public acceptance.

Conclusions
Despite the existence of several paradigms for pandemic ethics [38], many of these values were challenging to operationalise when it came to knowledge translation in a public health emergency.While all involved were undoubtedly trying to 'do the right thing', public health decision-makers dismissed unexpected research findings, researchers were reluctant to make them publicly available and all but one of the scientific journals approached were reluctant to publish, resulting in confusing messages in the media.As a solution, and as a moral imperative, several authors and groups have suggested unlimited and immediate sharing of information [39,40].However, while this may often be necessary, it is not sufficient to provide a practical or ethical solution.Furthermore, it may not always be in the interests of decision-makers, researchers or the public.Further in-depth sociological, ethical and policy research is warranted to better understand the complex interactions that occur in these situations.Recent controversy regarding attempts to stop publication of gain-of-function research related to influenza A(H5N1) virus highlights the ongoing need for answers to these issues [41].Rapid and extensive publications in highimpact factor journals in response to influenza A(H7N9) virus and Middle East respiratory syndrome coronavirus (MERS-CoV) infections indicate improved communication between the different solitudes, but a situation in which research findings ran counter to the prevailing ethos has not yet recurred to test the system.The events described above underscore the need for a critical review of the way unexpected or controversial research findings that arise during public health emergencies are evaluated by public health decision-makers and scientific journals, and how both the findings and the reviews are communicated transparently with the public.In order to fully understand all the issues and perspectives, we would welcome a debate between researchers, public health personnel, scientific journals and the public, based on a clear set of ethical and professional norms, on how we might better address these issue going forwards and bridge the three solitudes during future public health emergencies.