The following document is the major issues and changes resulting from the JSM'98 Panel Discussion.
You are invited to read it and join the discussion on it.
 

Power Analysis and Other Prerequisites to Valid and Humane Statistical Studies
 

Regarding human subjects protection (HSP), Nancy Berman, Ph.D., GCRC Statistician at the Harbor-UCLA Medical Center, noted that power analyses are required to justify the practical value of the research and the number of sample animals to be used in research on animals. This leads her to recommend power analysis or, where that may not be feasible, some similarly strong justification to justify experimentation on human beings. There should be strong evidence that the research is capable of producing valid results before human beings (or animals) are put at risk. Furthermore, the sample sizes should be large enough to yield valid results, but not so large as to place excessive numbers of subjects at risk unnecessarily. Carolin M. Frey of the Penn State Geisinger Health System recommends use of "prospective power analysis or the planned precision of the study endpoint with consideration given to the feasibility of obtaining subjects and the value of the data elements to be collected."
 

Others have commented on the tendency to protect only against Type I error (false positives) in hypothesis testing generally and have recommended power analysis in all statistical studies in order to protect against Type II error (false negatives) as well. The ethical issues may extend beyond HSP to include avoidance of wasting dollar and talent resources on statistical studies that could be foreseen to be invalid. The originally circulated draft of the guidelines attempted to include such considerations under the relatively simple but broad rubric of seeking practically significant results, not just statistically significant results. A recommendation for power analysis or similar justification could be usefully included both under the heading of Professionalism and under the heading Responsibilities to Research Subjects.
 

One could extend this principle more generally to deal with Tukey's admonition to take some time to understand data sets rather than rushing to run definitive statistical tests against them. John Bailar has cautioned, however, that ad hoc EDA can sometimes represent a "fishing expedition" such that an unorganized researcher simply reports on whatever happens to show some apparent statistical significance after extensive playing around with numerous statistical methods - without disclosing a sometimes much larger set of similar exploratory efforts which yielded no supportable findings. His point is that valid statistical science requires the construction of pre-defined protocols for EDA as well as for definitive statistical testing. The overall protocol (or set of protocols) should be reported clearly, transparently in the publication(s) resulting from the research.
 

How much of this belongs in the Ethical Guidelines? Certainly, we cannot present all of the considerations above in such a document. Can we cover all of the most essential elements in just a few well-chosen placements of just a few words each? Here are some candidate changes:
 
 
 
 

II. ETHICAL GUIDELINES
 

A. Professionalism
 

1. Strive for practical significance, not just statistical significance. Typically, combine normative understanding of the subject matter issues, statistical protocols that are clearly defined before looking at the data, and power analyses or similar justification of both the practical significance of the study and the sample sizes needed for valid results.

2. Considering possible researcher/data provider bias as well as random variation, use data selection processes that will be consistent with clear, transparent treatment of the issues during the research and with accurate understanding of that treatment by readers of the resulting publication(s).
 

3. Use only statistical methodologies suitable to the data and to valid results.
 

<skipping ahead to II.D.>
 

Carolin M. Frey also noted that II.D.6. (protecting special populations) is really a component of II.D.1. (protecting human research subjects generally.) She, along with Mary Grace Kovar suggested including references. As a result, II.D.1. has been rewritten to combine the human subjects protection items, address power analysis, and add federal references. The former II.D.6 has been eliminated.
 
 

D. Responsibilities to Research Subjects
 

1. Know and adhere to appropriate guidelines for human subjects protection, including protection of special populations who may not be fully able to protect their own interests. Assure that normative understanding of the subject matter combines with power analysis or similar justification to support both the practical value of research on human subjects and the sample sizes to be used. [U. S. federal guidelines are administered by the Office for Protection from Research Risks (OPRR) at the National Institutes of Health (NIH.) The regulations are defined in Title 45 of the Code of Federal Regulations, Chapter 46 (45CFR46). State and local rules, private organization guidelines, and regulations or guidelines in other countries may differ.]
 

2. Avoid excessive risk to research subjects and excessive imposition on their time and privacy.
 

<skipping to II.D.7.>
 

7. Know and adhere to appropriate animal welfare guidelines in research involving animals. Assure that normative understanding of the subject matter combines with power analysis or similar justification to support the practical value of the research and the sample sizes to be used.
 

Responsibilities of Those Employing Statistical Practitioners; Responsibilities of "Senior" Statisticians
 

James F. Ward, Ph.D., Solvay Pharmaceuticals, noted that moral behavior has been defined as the result of a moral agent operating in a moral environment. The July 31 and earlier drafts of the Guidelines address only the responsibilities of the individual, the moral agent. If the Guidelines fail to address the issue of a moral environment for the statistical practitioner to work in, then an essential component will have been left out.
 

Others in the audience also noted that senior statisticians have a responsibility to support more junior statisticians in resisting inappropriate pressures of expediency or lack of ethics. That led to a discussion of whether "senior" in this sense referred to age, position, prestige, or to some other indicator of the power to intervene between more subordinate practitioners and those who would impair their moral environment.
 

The audience and the Panel agreed this issue was sufficiently important to require some mention in the Purpose statement, I.A., as well as a separately defined section of Part II.
 

<skipping to new section II.H.>
 

H. Responsibilities of Organizations or Individuals Employing Statistical Practitioners, such as Employers, Attorneys, or other Clients.
 

1. Recognize that statistics is based in objective science. Results of valid statistical studies may turn out to be contrary to the expectations or desires of those commissioning the study or to those of the statistical practitioner.
 

2. Valid results can only result from competent work in a moral environment. Pressure on a statistical practitioner to deviate from the Guidelines above is likely to damage both the validity of study results and the professional credibility of the practitioner.
 

3. Statistical practitioners have ethical obligations to keep methodology in the public domain; proprietary reservation of statistical methodology, if justified at all, should be as limited as possible in time and scope.
 

4. Statistical practitioners and others have a social responsibility to support sound statistical analysis and to expose incompetent or corrupt statistical practice. In cases of conflict, practitioners and those employing them are encouraged to resolve issues of ethical practice privately if possible. If private resolution is not possible, recognize that statistical practitioners have an ethical obligation to expose incompetent or corrupt practice before it can cause avoidable harm to research subjects or to society at large.
 

5. Within organizations and within professions using statistical methods generally, statistical practitioners who have greater prestige, power, or status have a responsibility to protect the professional freedom and responsibility of more subordinate statistical practitioners to comply with these Guidelines.
 

To foreshadow this new section on responsibilities of those employing statisticians, the following is to be added as the last sentence of the Preamble, I.A.:
 

"Employers, attorneys, and other clients of statistical practitioners have a responsibility to provide a moral environment which fosters use of the ethical guidelines."
 

Informed Consent
 

Michael O'Fallon, Ph.D., Mayo Clinic, noted that statistical practitioners cannot personally accept the responsibility in II.D.4. to "obtain" informed consent of human research subjects. Other people typically perform that function. They can be attentive to assure that obtaining informed consent is included in the study protocols of research studies involving humans.
 

Carolin M. Frey also suggests:
 

"Prior to accepting data for analysis or manuscripts for review, ensure that appropriate subject approvals were obtained."
 

To combine these ideas, the revised II.D.4. no longer states "Obtain informed consent . . ."
 

4. When participating in a study involving human beings, analyzing data from such a study, or reviewing manuscripts which report on such studies, consider the conditions under which the human research subjects assented to provide data, including any informed consent statements and assurances of privacy and confidentiality. Respect those conditions as if they constituted a contract between each of the research subjects and yourself.
 

"Acknowledgment"
 

He also noted that there is a problem with "acknowledgments" of statisticians. Various people have disagreed as to the degree of responsibility a statistician or scientist should assume for the competency and ethics of a study under various circumstances. In general, co-authorship implies personal responsibility for the parts of the study performed by or supervised by that individual as well as general agreement with the overall report of the study methods and results. Acknowledgment of a statistical practitioner means, at a minimum, that the statistical practitioner did some professional work related to the study. That professional work may or may not have been actually used in the study. It may or may not have had an impact on the way the report was worded or presented. It may or may not imply that the statistical practitioner actually understood what other work was done or had formed an opinion as to the validity of the conclusions. It may or may not imply that the statistical practitioner had been aware of the contents of the report before publication.
 

Because "acknowledgments" are rife with potential confusion, the Mayo Clinic has adopted a policy of prohibiting any acknowledgment of their statisticians. Others in the audience noted that acknowledgments may be innocent and appropriate, especially when the statistical practitioner has aided the study at a level below that which is appropriate for co-authorship.
 

This suggests an entry in II.A. - Professionalism:
 

4. Do not join a research project unless you can expect to achieve valid results and unless you feel assured that your name will not be in any way associated with the project or resulting publications without your explicit consent.
 

(Subsequent items renumbered.)
 

It can be further addressed by entries in II.E. - Responsibilities to Research Team Colleagues and II.H. Responsibilities of {those employing statisticians}:
 

E. Responsibilities to Research Team Colleagues
 

1. Inform colleagues about relevant aspects of statistical ethics, including your right of refusal of authorship or acknowledgment.
 

H. Responsibilities of Organizations or Individuals Employing Statistical Practitioners, such as Employers, Attorneys, or other Clients.
 

6. Allow statistical practitioners to refuse authorship or even any acknowledgment regarding projects or publications with which they disagree professionally.
 

******************************************************************************
 REVISED DRAFT for General Comment - August 27, 1998