A Comparison of Validation
in Somatic Medicine and in
Psychiatry
There are at least three important differences
between the validation methods in somatic medicine and psychiatry:
However some limitations must be taken into account. Firstly, the objective laboratory values can have different meanings in different cultural contexts. Fabrega pointed out that all sorts of physical diseases are considered as such in some cultures and not in others. Such “cultural masking” occurs regarding certain avitaminoses, chronic bronchitis, light to medium anemia, trichuriasis, and other diseases. Even when the same validation methods are used, the line between health and disease, and with that the meaning of the objective values found, differ across cultures. A second significant factor is that knowledge of certain contextual facts is decisive in the assessment of certain validation results. For instance, the presence of acetone in urine can mean that someone has diabetes, or, in the absence of an adequate amount of carbohydrates, that he is starving and therefore his body fat is disintegrating. Whether or not this has pathological significance will have to be derived from the context. The same problem occurs with people who have Munchausen syndrome. These are people who, feigning a serious physical disease, have themselves hospitalized and sometimes even manage to undergo operations or other invasive treatments. Apparently the context of this type of simulation is so extraordinary that the usual validation methods are inadequate for detecting it. Consider also Kety’s example, quoted by Spitzer, “If I were to drink a quart of blood and, concealing what I had done, come to the emergency room of any hospital vomiting blood, the behavior of the staff would be quite predictable…” Finally, there is a classic report by Bakwin about research at the American Child Health Association. 610 of 1000 schoolchildren had undergone tonsillectomy. Physicians who examined the others recommended tonsillectomies in 45% of the children. The remainder were examined by different physicians, who recommended tonsillectomy in 46%. This last group was again examined by other physicians, who recommended tonsillectomies. At the end there remained 65 children who were not further examined because no more physicians were available. There did not seem to be any correlation between the different physicians’ conclusions. It seems to me that in this research, the context had an important role. If the physicians had known that the children had been selected in advance their own selections would have been different.
The objection that validation methods are adversely affected when the context in which the examination took place is manipulated, or when circumstances are artificial, for instance for the sake of research, is even more valid in psychiatry than in somatic medicine, although the phenomena are in principle comparable. Best known in this respect is research by Rosenhan which revealed that healthy people who applied for hospitalization claiming to suffer from hallucinations were unfailingly diagnosed as mentally ill and admitted. Temerlin describes an experiment in which 25 psychiatrists, 25 psychologists, and 45 psychology students were played an audio tape of a psychiatric anamnesis. The interviewee on the tape was in reality an actor who had been instructed by the researchers to impersonate a “normal person.” Before the tape was played the test professionals and students were told by an eminent colleague that the interesting thing about this interviewee was that he “seemed neurotic, but was in fact totally psychotic.” Although the task was to make a diagnosis on the basis of phenomena that were heard or reported, 15 psychiatrists judged the interviewee to be psychotic, 10 thought he was neurotic, and nobody thought he was healthy. The psychologists were in the middle: 7 thought him psychotic, 15 neurotic, and 3 healthy. Among the students, 5 judged him psychotic, 35 neurotic, and 5 healthy. A different group was told beforehand that the person was healthy. They unanimously judged the interviewee healthy. Out of a group of 21 test persons who were told nothing in advance about the interviewee, 9 judged him neurotic, and 12 healthy.
In such situations the paucity of possibilities for validation in psychiatry and lesser objectivity compared to physicochemical methods strikes home. When the circumstances in which the psychiatrist meets his patients are manipulated, the vulnerability and imperfection of the usual assessment methods in psychiatry are exposed.
The most important implication of the fact that the reliability of psychiatric diagnoses is considerably contingent on the context, is that the context in which the examination has taken place and what may be the influence thereof on the diagnostic evaluation must be constantly queried. The majority of diagnostic experiences involves the situation in which the purpose of the diagnose is to determine a treatment, so a situation in which the assessment is in the interest of the patient, who will cooperate. If even in this situation reliability is low, how will it be in a situation where it is in the patient’s interest to present a certain image of himself, or if the patient resists assessment? In such circumstances, research on reliability can be expected to yield differening results. It seems fair to hypothesize that reliability diminishes along with the patient’s willingness to cooperate. Whether the reliability of diagnostic assessment under such circumstances is sufficient to warrant basing decisions on it that may deeply affect the person’s life – which in practice happens regularly – seems dubious to me. I will return to this problem more than once below.
This vulnerability of psychiatric evaluation methods is augmented by the fact that in psychiatry there are many different, in part mutually exclusive frames of reference. The way patients are approached and the way examination results are interpreted differ in respect to the different frames of reference.
Kendell tried to explain the proliferation of the diagnosis schizophrenia in the United States compared to England through the different historical development of psychiatry in these two countries. In the United States psychiatrists attempted to constantly expand the concept of schizophrenia while in England they attempted to circumscribe and define it as narrowly as possible. The period of reduced scientific communication before and during World War II was sufficient to cause the conceptualization to grow apart, according to Kendell. He opines that for the two conceptualizations to grow back towards each other either new treatment possibilities or the finding of physicochemical validation methods will be necessary. In 1982 Spitzer conducted a workshop about the DSM-III in which he related his opinion that the large differences in frequency of the schizophrenia diagnosis have since disappeared. He ascribed the disappearance to the impression which the many publications about this difference made on American psychiatrists and to criticism of the significance of Schneider’s so-called first-rank symptoms, which had been justified by research. Kendell’s prediction was thus discredited…
In summary: Validation is possible to a certain extent in psychiatry, but the possibilities are limited. In principle the methods of validation resemble anamnesis and psychiatric examination. In addition, validation methods are scarcely used in daily practice. The reliability of diagnostic assessment in psychiatry is extraordinarily susceptible to influence by the context.
The conclusion is that reliability and thus also predictive value in psychiatry are meager. At the same time it is shown that validation in psychiatry is not only possible, but too good to conclude that it is purely subjective, as does Szasz. On the other hand validation in psychiatry is too insecure to be considered satisfactory from a scientific viewpoint, and as a rule supplies too little ground for having invasive measures or treatments based on it.
The statement that psychiatric symptoms and disorders can be validated does not imply a statement about the way these phenomena should be understood and interpreted, nor about the significance which should be ascribed to them. These matters will be discussed next.
- In somatic medicine validation is done by physicochemical means. So validation takes place at a different system level and utilizes a different language than that in which the problem is posed. In psychiatry both the posing of the problem and the attempt to verify the hypothesis about it are in terms of behavior and experience, so not different in quality from each other.
- In somatic medicine validation is very carefully and routinely done in research and daily practice. Due to its importance and predictive validity, its valuable role has become essential in the diagnostic process. In psychiatry, in contrast, the above mentioned methods are used mostly for research. In daily practice they are scarcely used although there is a clearly increasing tendency to apply more of such aids in practice. This nonetheless means that until now in daily practice psychiatric diagnoses consist almost entirely of anamneses and psychiatric evaluations, in some cases supplemented by physical exams and the collection of information about the patients’ important relations.
- The number and nature of the methods of validation in somatic medicine are large and varied. There are usually multiple possibilities for further examination and verification of the hypothesis. Reliability can be increased by repeating examinations or involving multiple examiners. In comparison, both the number and nature of instruments of examination available in psychiatry are quite limited.
- Knowledge of the context in which examinations transpire is much more significant in psychiatry than in somatic medicine. I wish to elaborate on this extraordinarily important point.
However some limitations must be taken into account. Firstly, the objective laboratory values can have different meanings in different cultural contexts. Fabrega pointed out that all sorts of physical diseases are considered as such in some cultures and not in others. Such “cultural masking” occurs regarding certain avitaminoses, chronic bronchitis, light to medium anemia, trichuriasis, and other diseases. Even when the same validation methods are used, the line between health and disease, and with that the meaning of the objective values found, differ across cultures. A second significant factor is that knowledge of certain contextual facts is decisive in the assessment of certain validation results. For instance, the presence of acetone in urine can mean that someone has diabetes, or, in the absence of an adequate amount of carbohydrates, that he is starving and therefore his body fat is disintegrating. Whether or not this has pathological significance will have to be derived from the context. The same problem occurs with people who have Munchausen syndrome. These are people who, feigning a serious physical disease, have themselves hospitalized and sometimes even manage to undergo operations or other invasive treatments. Apparently the context of this type of simulation is so extraordinary that the usual validation methods are inadequate for detecting it. Consider also Kety’s example, quoted by Spitzer, “If I were to drink a quart of blood and, concealing what I had done, come to the emergency room of any hospital vomiting blood, the behavior of the staff would be quite predictable…” Finally, there is a classic report by Bakwin about research at the American Child Health Association. 610 of 1000 schoolchildren had undergone tonsillectomy. Physicians who examined the others recommended tonsillectomies in 45% of the children. The remainder were examined by different physicians, who recommended tonsillectomy in 46%. This last group was again examined by other physicians, who recommended tonsillectomies. At the end there remained 65 children who were not further examined because no more physicians were available. There did not seem to be any correlation between the different physicians’ conclusions. It seems to me that in this research, the context had an important role. If the physicians had known that the children had been selected in advance their own selections would have been different.
The objection that validation methods are adversely affected when the context in which the examination took place is manipulated, or when circumstances are artificial, for instance for the sake of research, is even more valid in psychiatry than in somatic medicine, although the phenomena are in principle comparable. Best known in this respect is research by Rosenhan which revealed that healthy people who applied for hospitalization claiming to suffer from hallucinations were unfailingly diagnosed as mentally ill and admitted. Temerlin describes an experiment in which 25 psychiatrists, 25 psychologists, and 45 psychology students were played an audio tape of a psychiatric anamnesis. The interviewee on the tape was in reality an actor who had been instructed by the researchers to impersonate a “normal person.” Before the tape was played the test professionals and students were told by an eminent colleague that the interesting thing about this interviewee was that he “seemed neurotic, but was in fact totally psychotic.” Although the task was to make a diagnosis on the basis of phenomena that were heard or reported, 15 psychiatrists judged the interviewee to be psychotic, 10 thought he was neurotic, and nobody thought he was healthy. The psychologists were in the middle: 7 thought him psychotic, 15 neurotic, and 3 healthy. Among the students, 5 judged him psychotic, 35 neurotic, and 5 healthy. A different group was told beforehand that the person was healthy. They unanimously judged the interviewee healthy. Out of a group of 21 test persons who were told nothing in advance about the interviewee, 9 judged him neurotic, and 12 healthy.
In such situations the paucity of possibilities for validation in psychiatry and lesser objectivity compared to physicochemical methods strikes home. When the circumstances in which the psychiatrist meets his patients are manipulated, the vulnerability and imperfection of the usual assessment methods in psychiatry are exposed.
The most important implication of the fact that the reliability of psychiatric diagnoses is considerably contingent on the context, is that the context in which the examination has taken place and what may be the influence thereof on the diagnostic evaluation must be constantly queried. The majority of diagnostic experiences involves the situation in which the purpose of the diagnose is to determine a treatment, so a situation in which the assessment is in the interest of the patient, who will cooperate. If even in this situation reliability is low, how will it be in a situation where it is in the patient’s interest to present a certain image of himself, or if the patient resists assessment? In such circumstances, research on reliability can be expected to yield differening results. It seems fair to hypothesize that reliability diminishes along with the patient’s willingness to cooperate. Whether the reliability of diagnostic assessment under such circumstances is sufficient to warrant basing decisions on it that may deeply affect the person’s life – which in practice happens regularly – seems dubious to me. I will return to this problem more than once below.
This vulnerability of psychiatric evaluation methods is augmented by the fact that in psychiatry there are many different, in part mutually exclusive frames of reference. The way patients are approached and the way examination results are interpreted differ in respect to the different frames of reference.
Kendell tried to explain the proliferation of the diagnosis schizophrenia in the United States compared to England through the different historical development of psychiatry in these two countries. In the United States psychiatrists attempted to constantly expand the concept of schizophrenia while in England they attempted to circumscribe and define it as narrowly as possible. The period of reduced scientific communication before and during World War II was sufficient to cause the conceptualization to grow apart, according to Kendell. He opines that for the two conceptualizations to grow back towards each other either new treatment possibilities or the finding of physicochemical validation methods will be necessary. In 1982 Spitzer conducted a workshop about the DSM-III in which he related his opinion that the large differences in frequency of the schizophrenia diagnosis have since disappeared. He ascribed the disappearance to the impression which the many publications about this difference made on American psychiatrists and to criticism of the significance of Schneider’s so-called first-rank symptoms, which had been justified by research. Kendell’s prediction was thus discredited…
In summary: Validation is possible to a certain extent in psychiatry, but the possibilities are limited. In principle the methods of validation resemble anamnesis and psychiatric examination. In addition, validation methods are scarcely used in daily practice. The reliability of diagnostic assessment in psychiatry is extraordinarily susceptible to influence by the context.
The conclusion is that reliability and thus also predictive value in psychiatry are meager. At the same time it is shown that validation in psychiatry is not only possible, but too good to conclude that it is purely subjective, as does Szasz. On the other hand validation in psychiatry is too insecure to be considered satisfactory from a scientific viewpoint, and as a rule supplies too little ground for having invasive measures or treatments based on it.
The statement that psychiatric symptoms and disorders can be validated does not imply a statement about the way these phenomena should be understood and interpreted, nor about the significance which should be ascribed to them. These matters will be discussed next.
Next
Table of Contents