Sidney Siegel--Core Research & Teaching Themes
|Sid pursued his research as a psychologist in a stage in the development of the field when a journal of negative results, however often proposed, was never published for the simple reason that such a journal would have received so many contributed papers that few libraries would have space to shelve the issues. In his own research, however, positive results were the rule rather than the exception. In a typical experiment, he would be plagued with an embarrassment of riches in the form of positive findings.
Some of Sid's core notions about psychological research help to account for the discrepancy between his Own success and that of his contemporaries, although no doubt the explanation lies principally in his unusual talent and his exceptional devotion to his work. The purpose of this section is to review these core notions. They will be stated simply and starkly, with the assumption that the reader is to understand that every sentence here begins implicitly with the phrase "Sid believed that . . . ."
Research should be guided by theory. The purpose of research is to test a theory and to provide data, which will lead to refinement and extension of the theory. To state that research should be designed so as to test an hypothesis is correct but not sufficient. One must state further that the hypothesis should be derived from a theory, not simply dreamed up by an intuitive experimenter. (Confirmation of a man's intuition may contribute to his self-esteem, but it does not advance the science of psychology.) Furthermore, some hypotheses are more interesting than others, and more powerful, precisely because they are surprising. The hypothesis "If I drop this pencil from my hand it will fall to the desk" is testable and is derivable from some classical mechanical notions in physics, but it holds little interest precisely because hardly any other outcome is imaginable or predictable from any theory. An hypothesis is interesting and powerful if it predicts but one outcome among many imaginable outcomes and if the predicted outcome is different from those expected under other theories. If, in fact, the predicted outcome is observed in adequately controlled research, it lends support to the theory from which the hypothesis was generated.
Sid had no patience with so-called exploratory studies, "fishing expeditions," masquerading as research. The fact that a topic might be “interesting," "timely," or "cute," is hardly sufficient to justify collecting a mass of heterogeneous information about it and then dredging around in the information, hunting for "findings." Exploration is a proper purpose of pilot studies, which are valuable and indeed invaluable, but 8, formal experiment ought not to be undertaken until an hypothesis has been derived from an explicit theory about the phenomena to be observed and until the experiment has been designed to provide a clear and unambiguous result relevant to that hypothesis.
With the data of any study in hand, the investigator is bound to find interesting regularities and strong relations if he searches patiently. Such post hoc findings may be useful in suggesting tentative modifications of a theory and then fresh experiments to test the modifications, but they cannot be given the status of findings or results. Sid's reluctance to analyze his data beyond a simple test of a prior hypothesis lends a certain stark austerity to his writing, but it must strengthen confidence in the meaningfulness of any probability statements connected with the reported results. A probability statement attached to a result is meaningful only if the decision to make a statistical test for that result was reached prior to any examination of the data.
Although the statistical argument for limiting data analysis to tests of prior hypotheses is but one reason for austerity and self-imposed restraint in the analysis, it is an important reason, and its logic suggests a related constraint. One must suspect the widespread procedure of testing several hypotheses with the same set of scores, again because of the possibility of capitalizing on chance. A large part of the present confusion in psychological knowledge is attributable to the regrettably common practice of collecting a heterogeneous set of measures from a group of subjects and then correlating every measure with every other and testing the resulting correlation coefficients for "significance."
Sid was adamant about the importance of “remaining close to your data.” This phrase carried several related meanings. One is that a research worker cannot afford routinely to delegate data collection to others. He must remain active in the day-to-day work of his laboratory if he is to understand the data when they are in and if he is to be able to identify the needed next steps in a research program.
The style of measurement one employs determines how close one stays to his data. Some measurement techniques yield numbers which are several steps distant from the actual observations. The best measure is the simplest, the one involving the fewest assumptions and the fewest transformations. Where possible, the subject's response itself should be directly statable as a measure. This was the case in the bargaining studies [17,23,26], where the measure is the monetary amount of the subject's bid or offer. Where a single response is not sufficient, simple counting of responses is a measurement technique which keeps one close to the data, and it is worthwhile to expend considerable ingenuity to develop an assessment technique which will yield responses that may simply be counted to yield a meaningful score. No doubt, one basis of the attractiveness of the two-light situation for the study of choice behavior [12, 14, 20, 25, 28] is the fact that the score of interest is arrived at by enumeration: a subject's strategy is determined by counting the choices he allocates to each alternative.
The design of the research and the statistical analysis associated with the design are also determinants of how close a research worker may stay to his data. Sid was in no sense a statistical psychologist, although his mastery of the field of statistics was equaled by few of his contemporaries among psychologists, and although he was an unusually skillful and enthusiastic teacher of psychological statistics. He averred that the best-designed experiment is one requiring no statistical analysis at all. Where statistics are needed, the simpler the better. A major argument for nonparametric tests is their simplicity: their basis is easily grasped, the computations are straightforward, no distorting transformations are imposed on the raw scores. Preferring clean and simple designs, Sid had little use for the analysis of variance and typically voiced his suspicion by proclaiming his inability to understand the meaning of any interaction. A simple two-group experiment usually sufficed to test hypotheses of interest.
The purpose of science is explanation: A scientific explanation is a theory which asserts the existence of specified relations among stated variables. The appropriate technique for testing whether these relations hold in the empirical world -- for identifying the nature and the extent of the empirical domain to which the theory applies -- is an experiment. That is, the experimental method, in which the investigator manipulate the variable of interest and then observes the effect of his manipulation upon the dependent variable, is the best method for testing an explanatory notion.
Prediction and control, often mentioned jointly with explanation as essential features of science, are to be understood as central features of the experimental method. An experiment is designed to test a prediction; the accuracy of the prediction is an index of the explanatory power of the theory from which the prediction (hypothesis) was derived. Control is essential in an experiment in order to give the prediction a fair test; controls serve to isolate the operations of the variables of interest, to throw the operations of those variables into relief and thus enable a test of the explanatory power of the theory's statement about those variables.
To think that psychologists are working toward a future science in which we shall be able to "predict human behavior" and thus to "control mankind” is to misunderstand the nature of psychological science. We are working toward a science explaining human behavior; prediction and control are features of our method for arriving at adequate explanations.
The experimental method is the only approach in which a prediction may be tested directly. Correlational data must always be equivocal, since controls can never be adequate. Randomization, perhaps the sine qua non of experimentation, is by definition absent in a correlational design, and in its absence the results are inevitably ambiguous.
His belief in the experimental method and his predilection for remaining close to the data combined to make Sid especially suspicious of factor analysis as an approach to psychological research. Of all the correlational approaches, the factor analytic seems to involve the largest number of assumptions, to impose the largest number of transformations on the data, and to remove the investigator the farthest from the observations. Several related arguments are commonly advanced against experimentation by those who prefer observational-correlational approaches. It is said that the laboratory is artificial, removed from "real life." Only by involving ourselves in our subjects' everyday lives in their own milieus can we succeed in studying the important variables, those which really make a difference. The effects an experimenter can produce through his manipulation of some independent variable in an experiment are trivial and insignificant compared to the massive effects produced by the profound manipulations which nature performs. Finally, if social scientists are to succeed in finding answers to the important social questions, they will need to observe the phenomena of interest in their genuine context.
Sid's work exemplifies replies to these arguments. In the first place, the term "experimentation" refers to a design and not to a location. The essential features of experimental design are control and comparison, with randomization an essential part of control. Where randomization can be achieved in the field, experiments can be conducted away from any laboratory, as was done in the study of reference groups and membership groups  in which the living group assignment of subjects was randomly determined.
Second, experiences in a laboratory need not be artificial and removed from putative "real life." With ingenuity, laboratory experiences may be arranged which are meaningful for the subjects and in which they become personally absorbed. At present, the technique for making laboratory experiences meaningful which has vogue among social psychologists is the technique of deception. For example, subjects are deceived about the personal characteristics of their working partners in a laboratory, about their own intelligence scores or the interpretation of their projective test responses, about the purpose of the experiment in which they are serving, etc. Sid disliked and avoided deception, principally on ethical grounds, but also because an experiment involving deception creates a climate of suspicion and distrust toward psychological experimentation. In some laboratories of social psychology, the subject enters wondering what lie he is going to be told this time.
In Sid's, the subject entered wondering how much money he would make. This is put facetiously to introduce an alternative approach for making laboratory experiences meaningful, the one Sid employed. He built into the experimental situation features which enlisted the subject's motivation. He believed in "the payoff." In the studies directed to the measurement of utility in an interval scale [2, 6], the subject gambled for money with the experimenter, and his choices between the alternative gambles offered to him revealed his idiosyncratic utility scale. In the studies of bordered metric measurement, the payoff was a book  or cigarettes . In the level-of-aspiration studies [8, 10, 24], the payoff was the student's grade on a course examination, although in fact the payoff was so arranged that the student's final grade in the course was not differentially affected by his participation in the research. In the bargaining studies [17, 23, 26], the payoff was cash, the monetary amounts of the contracts negotiated by the subjects. And in the choice-behavior studies [12, 20, 25, 28], the payoff was cash or, in the case of young children, trinkets.
The important feature of the payoff is not that the subject is recompensed for giving his time to research. This aspect is incidental. The important feature is that the amount of the payoff to the subject depends directly and differentially on how the subject performs in the experiment; his motivation to gain a large payoff is enlisted as motivation to serve the purposes of the experiment, to highlight the operation of the variables under study.
Offhand, it might seem that payoffs were used in the decision-making research because the topics under study -- utility of money, economic choice behavior-- made it natural to use payoffs. The reverse was more the case. Sid's convictions about decision-making convinced him that meaningful observations of social behavior could be made best where payoffs are involved, and he searched for situations in which a payoff could readily be employed. He was suspicious of any study whose measure depended on the subject's good will or cooperativeness toward the experimenter. In evaluating the validity of any measure, he asked, "What was the subject's motivation for responding?" He voiced his attitude as a suspicion of paper-and-pencil measures, but this expression of it was rather beside the point. In fact, the objection was not to paper and pencil -- in a few of his minor studies of decision making [cf. 10, 16, 21] it was not unknown for subjects to be asked to put pencil to paper -- but rather, the objection was to verbal self-report motivated by good will toward the experiment. As any psychologist would, he always laughed at the "gedanken experiments" of economists in which they try to settle empirical questions by imagining what they would do in the economic situation under analysis. But he parted company with many psychologists in thinking that self-report measures -- adjective checklists, self-rating scales, preference inventories, attitude questionnaires, personality inventories, and the like -- are similarly suspect.
As for the contention that field investigations produce data more relevant to applications of psychology to the solution of practical problems, Sid thought that this is an argument against them. Present-day psychological knowledge is so tenuous and incomplete, and its methods are so crude, that any attempts to apply psychology are greatly premature and foredoomed to failure. Although he loved to talk about the disarmament negotiations in Geneva in the light of his findings concerning economic bargaining, he recognized the chasm between laboratory findings and practical applications and he knew that success in the latter depends much more on the creativity and insight of the person making the practical translation than on the content of the findings themselves. Our generation of psychologists needs to stay in the laboratory, pursuing systematic and theoretically based programs of experimentation, he believed, and leaving the solution of immediate practical problems in the hands of responsible men experienced with them.