Psychological Assessment Meets Legerdemain

In an article appearing in the December 2008 issue of The Matrimonial Strategist entitled “One Picture May Not Be Worth 1,000 Words,” I opined that “drawings cannot possibly be viewed as a reliable source of information concerning children's perceptions of themselves, their families, the dynamics within their families, or anything else.” My view has not changed in the intervening months. The focus of this article is the attempt by proponents of drawings as a data source to develop and employ means by which attributes observed in children's drawings can be quantified.

'Garbage In, Garbage Out'

Mental health professionals who believe that magical things can be done with drawings if scoring systems are employed should consider a popular slogan from the early computer days: Garbage in, garbage out. Scoring systems perform no magic. Assigning numerical values to various aspects of a drawing does not transform the inherently subjective judgment into an objective measurement any more than does the assignment of points to competitive divers, ice skaters, or show dogs. When multiple judges rate a diver, skater, or show dog, the impact of each judge's subjectivity is blunted to some extent by the ratings of each of the other judges. Where one evaluator assigns scores to features of a drawing, evaluator subjectivity functions without any control.

Attorneys who find themselves cross-examining practitioners who proudly assert that they utilize objective scoring systems may wish to ask: 1) how the cut-off scores were determined; 2) what the inter-judge reliability coefficients are for the identified and scored features; and 3) what the test/re-test reliability coefficients are for the identified and scored features. With respect to Number One, the objective of the inquiry is to ascertain whether any empirical studies have been conducted in which it has been determined that a specific cut-off score enables practitioners reliably to offer the descriptive statements that appear in texts describing scoring systems.

Type I and Type II Errors

If practitioners are audacious enough to assert that such studies have been conducted, attorneys should inquire about the frequency of type I and type II errors. (Type I errors are generally referred to as “false positives.” Simply put, a type I error occurs when a device of any type produces a reading that suggests to the operator of the device that the item of interest has been detected, when, in fact, the item of interest is not there. A type I error can also be conceptualized as a “false alarm.” Type II errors are generally referred to as “false negatives.” A type II error occurs when the item of interest is present but goes undetected.)

Returning to the original inquiry, with respect to Number Two, for any specified feature in the drawing under discussion, the practitioner should know how often “blind” raters agree in the scoring of the specified features (large hands, small eyes, short arms, long necks, accentuated knee caps, etc.). Blind raters are those who examine children's drawings without knowledge of the children who have produced the drawings. With respect to Number Three, practitioners who are using drawings in any endeavor that involves looking beyond the current situation (e.g., treatment planning or communicating parenting plan recommendations to courts) should be prepared to cite test/re-test data ' data that reflect the temporal stability (or lack thereof) of the various features that have been identified. What is the probability that a child who draws big eyes today will draw big eyes six months from today?

Tunnel Vision

What professionals look for is profoundly influenced by their fields of practice, professional interests, and personal experiences. This dynamic has become the source of jokes that take jabs at occupationally induced tunnel vision. Within the field of medicine, surgeons are more likely to identify medical problems for which there are surgical solutions, endocrinologists are more likely to identify problems involving hormone imbalance, and psychiatrists are more likely to identify problems for which there are psychopharmacological solutions. The majority of those who employ drawings as a means by which to detect various emotional difficulties in children, treat children with emotional difficulties. Constructive skepticism is warranted.

Weinberg, writing in the Tenth Mental Measurements Yearbook (1989, J. C. Conoley & J. J. Kramer, J. J. (Eds.)), has called attention to the fact that there are no data to support the notion that normal children and what Weinberg refers to as “clinical” children (i.e., children in need of treatment for emotional difficulties) can be differentiated from one another based upon identified features in their drawings.

The Life Events Rating Scale

Skepticism remains appropriate when practitioners assert that, in examining children's drawings, they are looking for specific features (rather than employing an impressionistic approach), that they utilize a scoring system, and that pre-determined cut-off scores are referenced in the interpretive process.

As we enter our court houses, we pass through devices that detect ferrous metals. No matter what we call these devices, law enforcement's purpose in using them is to detect weapons; not zippers and not metal hooks on brassieres. The sensitivity level of these devices must be set with false alarms (false positives) and detection failures (false negatives) in mind. If the threshold for triggering a signal is too low, small metal items of no interest to law enforcement will yield numerous false alarms. If the threshold setting is too high, someone carrying a weapon may pass undetected. The same principle applies to the setting of cut-off scores in systems used to assign scores to features in drawings.

An early example at attempts to enhance precision through quantification may be instructive. In 1967, Thomas Holmes and Richard Rahe developed a scale named the Social Readjustment Rating Scale by them, but more frequently referred to by others as the Life Events Rating Scale (Holmes, T. H. & Rahe, R. H. 1967. The Social Readjustment Rating Scale. J Psychosomatic Research, 11, 213-318.). The concept underlying the development of the scale was that all significant life events, particularly when clustered (i.e., coming in rapid succession) are an element to be considered in understanding the etiology of illnesses. The reasoning employed by Holmes and Rahe, which was simple and widely accepted by medical researchers, was that significant life events demand adjustments by those who have experienced them, that these adjustments are taxing, that the immune system is compromised to some extent as a result, and, that the compromised immune system leaves individuals more vulnerable to forces (bacteria, viruses) that might be successfully repelled under normal circumstances.

Though the introduction of the concept of social readjustment and its implications for health were enthusiastically welcomed, the scale itself was widely criticized, and with good reason. Operating with an obvious lack of empirical data, Holmes and Rahe endeavored to quantify the stress that would be encountered in adjusting to various life events, both positive and negative. Examples of the Holmes and Rahe assignment of numerical values to the stress associated with adjusting to various life events follow: 1) The death of a spouse is assigned 100 points and this is the highest numerical value on the scale; 2) The next event, in descending order, is divorce, which is assigned a numerical value of 73; 3) Marriage is assigned a value of 50; 4) Pregnancy is assigned a value of 40; 5) Also assigned a value of 40 are “sex difficulties”; 6) Foreclosure is assigned a value of 30; 7) The item with the lowest assigned value is “minor violation of the law,” which is assigned a value of 11.

The foregoing examples are presented in the hope that an examination of the items and the accompanying assigned values will bring the reader to recognize that if different people were to be asked to perform the task of assigning adjustment-stress values to these life events, significant disagreement would be likely.

In the course of explaining the perceived advantages of test data (as contrasted with other forms of data gathered in custody evaluations), Dianna Gould-Saltman, an experienced family law attorney, stated that attorneys “understand numbers,” in an article written for the Journal of Child Custody, in 2005. If Gould-Saltman's statement is taken as I believe it was intended, as a general statement about attorneys, I must respectfully disagree. As a rule, attorneys do not understand concepts such the standard error of measurement, and, as a result, are unlikely to know when the difference between two scores on a test is significant and when it is likely to be insignificant. It is the belief, held by attorneys and others, that numbers are understandable that leads people to trust numbers. By extension, when numbers are cited in support of various propositions, their persuasive power is often significant.

The Sign Approach

Mental health professionals who employ scoring systems in their efforts to interpret drawings are employing what is known as the sign approach to figure drawing interpretation. When this approach is being employed, certain features in the drawings are viewed as signs (indicators) of various personality characteristics, life experiences, or perceptions of others. Lally has written: “The data on the [inter-judge] reliability of the specific sign approach are not good ' [They] have reliabilities that are probably too low for making reasonably reliable clinical judgments” (Lally, S. J. (2001). Should Human Figure Drawings Be Admitted Into Court? J Personality Assessment, 76(1), 135-149, p. 140]. In the writing of advocates of the sign method, there is evidence that they do not always agree among themselves concerning what a particular feature indicates. Peterson and her colleagues have opined that large eyes may be drawn by children who have seen or experienced physical abuse (Peterson, L. W. & Hardin, M. E. (1997): Children in Distress: A Guide for Screening Children's Art. NY: Norton; Peterson, L. W., Hardin, M. E., & Nitsch, M. J. (1995). The use of children's drawings in the evaluation and treatment of child sexual, emotional, and physical abuse. Arch Fam Med, 4, 445-452.). Cantlay, on the other hand, opines that distress and trauma may be reflected in atypically small eyes (Cantlay, L. (1996). Detecting Child Abuse: Recognizing Children at Risk Through Drawings. Santa Barbara: Holly Press).

Some of those who develop a dependence upon computers voluntarily relinquish their exercise of common sense when it appears to conflict with a computer-generated solution. Similar dynamics can be seen in the work of mental health professionals who employ scoring systems in interpreting children's drawings. An example follows. An evaluator employs a system in which points are assigned to various features that are suggestive of various emotional problems, in the views of the developers of the system. In this system, one point is assigned for “Writing words on picture.” A child returns from Disneyland on a Monday and enters the evaluator's office on the following Wednesday wearing a t-shirt with “Disneyland” written on it. In responding to the instruction that she draw a person, the child draws a female with a strong resemblance to the child, wearing a t-shirt with “Disneyland” written across the front. The evaluator, adhering to the 'rules' of the scoring system that she is employing, assigns a point to the drawing. Even if we presume, and I do not, that interpretations of children's drawings constitute a useful data source, is the broader task of understanding the psychology of the child being evaluated facilitated by viewing this feature of the child's drawing as being suggestive of abnormality? Would it not be more helpful to chat with the child about the drawing, her t-shirt, and her visit to Disneyland and to use the child's statements as the data source to be drawn upon?

Quantitative Measurement

Quantitative measurement is rendered useless when people who believe themselves to be measuring the same thing are, in fact, measuring different things. Consider a situation in which two individuals are asked to measure the dimensions of a picture frame and, when their results are compared, there is a discrepancy of roughly two inches in the measure of width and two inches in the measure of length. Inquiry reveals that one of the individuals measured the frame from its outer edges and that the other individual measured the frame from its inner edges. Even in situations in which measures can be taken with reasonable precision, different individuals may generate different results if there is lack of clarity with regard to what it is that is being measured. In the example that I have provided, it is likely that discrepant results could have been prevented if each participant had been provided with the same instructions and if those instructions had been precise. Mental health professionals, and attorneys who cross-examine them, would be wise to consider the applicability of this example for psychological assessment.

Anecdotal evidence (primarily in the form of testimony offered by mental health professionals concerning their assessment methods) suggests that evaluators are alarmingly inconsistent in directions provided to children from whom they wish to obtain drawings. It is not uncommon to encounter, in testimony, assertions from evaluators that they do not recall exactly what was said by them to children prior to the production of drawings by the children. What a child draws can be dramatically affected by the child's interpretation of the examiner's wishes. In some projective tasks, ambiguity in instructions is deliberate and the evaluee's interpretation of the instructions (as inferred based upon the evaluee's response) is an element of the projective assessment. It must be emphasized, however, that ambiguity and inconsistency are not synonymous. Where ambiguity is desired, the wished-for level of ambiguity is achieved by means of consistent instructions.

In the Draw A Person test, the instructions are: “Draw a person.” Providing that instruction is quite different from: 1) “Draw something”; 2) “Draw a man”; 3) “Draw a woman”; 4) “Draw a boy”; or “Draw a girl.” Similarly, the instruction “Draw a house” is not the same as “Draw your house.” The accuracy of this statement is not undercut by the reality that users of the H-T-P (House-Tree-Person test) would undoubtedly explain that when children are asked to draw a house, that which they produce is likely to be linked in some manner (realistically or symbolically) to their own houses.

Directions may be of critical importance when drawing tasks are used in the context of custody evaluations in assessing children who may be members of two families. Consider Joey who spends part of his time with his mother and part of his time with his father (who is residing with a woman and her daughter). Evaluators who are skilled in investigating allegations of child sexual abuse take great care to insure that there is no miscommunication concerning the meaning of various words. We must consider the possibility that evaluators employing drawing tasks in routine custody cases do not employ similar care.

Consider the possibility that, as Joey understands the word, a “family” must include a male parent figure and a female parent figure. When asked to draw his family, Joey draws himself, his father, his father's paramour, and the paramour's daughter. Consider, further, the possibility that the evaluator does not discuss Joey's drawing with him, but, instead, files the drawing with the intention of interpreting it at some future time. When the evaluator interprets the drawing, she concludes that Joey feels closer with his father (and those now residing with his father) than with his mother. Is this a valid interpretation?

In offering general commentary on the Draw A Person (DAP) task, Willis, writing in the Eleventh Mental Measurements Yearbook (1992 ' J. J. Kramer & J. C. Conoley (Eds.)), opined that “the DAP was neither intended nor validated as a personality measure, and it clearly should not be used for this purpose.”

Knoff, writing in the Sixth Mental Measurements Yearbook (1965 ' O. K. Buros (Ed.)) on the subject of scoring systems for the House-Tree-Person test, called attention to the fact that the author of the manual being reviewed asserts that the manual “is designed to provide the diagnostician with an objective, quantifiable scoring system that has a well-grounded theoretical and research base.” Knoff opined: “Unfortunately, neither of these assertions is true. This scale makes numerous, faulty assumptions as to the theoretical and research base that forms the foundation of this system, and there are many significant shortcomings in the system's norming process, its normative data, and in its psychometric properties.”

Knoff also called attention to the fact that “the author [of the manual] fails to note that none of [the currently used scoring] systems have been independently validated ' ” He added that “ projective drawing research is largely nonempirical and case study oriented, of questionable methodological and technical integrity, and lacking in convergent and discriminant validity ' .” He opined that “the failure to document or demonstrate the discriminant validity for each of the projective indicators calls the entire system into question.”

Dowd, also writing in the Sixth Mental Measurements Yearbook, pointed out that there is no empirical support for the very underpinnings of drawing interpretation ' the notion (often expressed as a given) that feature X in a drawing symbolically represents Y in the personality of or life of the person who has produced the drawing. Offering an example, Dowd observed that there are no data to support the notion that a “door on the side of a building” is suggestive of withdrawal.

Cosden, writing in the Eleventh Mental Measurements Yearbook, called attention to the fact that no particular scoring system has been agreed upon, that several different systems are in use, and that the “advantage of one set of scoring criteria over another has not been demonstrated empirically.”

In a review appearing in the Tenth Mental Measurements Yearbook, Weinberg, summarizing the available data on quantitative analyses, stated: “In a nutshell, the case (for psychometric use of KFDs) continues to be weak (citations omitted).” “Available data do not support the notion that criteria utilized in four different scoring systems can be conceptualized as objective criteria.” He pointed out that “Poor test-retest stability is reported” and reminded readers of the review that even proponents of drawing tasks acknowledge that certain features “'are sensitive to children's transitory personality states.'” Weinberg observed that “the accumulated psychometric underpinnings for projective drawings ' are lean” and opined that “the age of accountability ' demands that greater psychometric credibility be demonstrated for kinetic drawings.” He concludes, and I agree, that drawing tasks are useful as an “ice-breaker.”

In the Tenth Mental Measurements Yearbook, Cundick points out that the authors of the manual being reviewed do not purport to have a test; rather they indicate that their approach “primarily involves a hypothesis-generating and hypothesis-testing model ' .” Cundick states that “the clinician chooses hypotheses to entertain and test further based on his or her knowledge and experience with the child.” No harm results when treating practitioners utilize drawings to generate hypotheses and, subsequently, explore those hypotheses in the course of a therapeutic relationship. Great harm may be done when forensic practitioners utilize drawings not to generate hypotheses but to formulate opinions, and when those opinions are shared with judges who will be making decisions in custody/access disputes.

David Martindale, a member of this newsletter's Board of Editors, board certified in forensic psychology by the American Board of Professional Psychology, served as the Reporter for the Association of Family and Conciliation Courts' Model Standards for Child Custody Evaluation and is the co-author, with Jon Gould, of The Art and Science of Child Custody Evaluations. His practice is limited to consulting with attorneys, psychologists, and psychology licensing boards in the areas of child custody and professional ethics and standards. He can be contacted by e-mail at [email protected].

In an article appearing in the December 2008 issue of The Matrimonial Strategist entitled “One Picture May Not Be Worth 1,000 Words,” I opined that “drawings cannot possibly be viewed as a reliable source of information concerning children's perceptions of themselves, their families, the dynamics within their families, or anything else.” My view has not changed in the intervening months. The focus of this article is the attempt by proponents of drawings as a data source to develop and employ means by which attributes observed in children's drawings can be quantified.

'Garbage In, Garbage Out'

Mental health professionals who believe that magical things can be done with drawings if scoring systems are employed should consider a popular slogan from the early computer days: Garbage in, garbage out. Scoring systems perform no magic. Assigning numerical values to various aspects of a drawing does not transform the inherently subjective judgment into an objective measurement any more than does the assignment of points to competitive divers, ice skaters, or show dogs. When multiple judges rate a diver, skater, or show dog, the impact of each judge's subjectivity is blunted to some extent by the ratings of each of the other judges. Where one evaluator assigns scores to features of a drawing, evaluator subjectivity functions without any control.

Attorneys who find themselves cross-examining practitioners who proudly assert that they utilize objective scoring systems may wish to ask: 1) how the cut-off scores were determined; 2) what the inter-judge reliability coefficients are for the identified and scored features; and 3) what the test/re-test reliability coefficients are for the identified and scored features. With respect to Number One, the objective of the inquiry is to ascertain whether any empirical studies have been conducted in which it has been determined that a specific cut-off score enables practitioners reliably to offer the descriptive statements that appear in texts describing scoring systems.

Type I and Type II Errors

If practitioners are audacious enough to assert that such studies have been conducted, attorneys should inquire about the frequency of type I and type II errors. (Type I errors are generally referred to as “false positives.” Simply put, a type I error occurs when a device of any type produces a reading that suggests to the operator of the device that the item of interest has been detected, when, in fact, the item of interest is not there. A type I error can also be conceptualized as a “false alarm.” Type II errors are generally referred to as “false negatives.” A type II error occurs when the item of interest is present but goes undetected.)

Returning to the original inquiry, with respect to Number Two, for any specified feature in the drawing under discussion, the practitioner should know how often “blind” raters agree in the scoring of the specified features (large hands, small eyes, short arms, long necks, accentuated knee caps, etc.). Blind raters are those who examine children's drawings without knowledge of the children who have produced the drawings. With respect to Number Three, practitioners who are using drawings in any endeavor that involves looking beyond the current situation (e.g., treatment planning or communicating parenting plan recommendations to courts) should be prepared to cite test/re-test data ' data that reflect the temporal stability (or lack thereof) of the various features that have been identified. What is the probability that a child who draws big eyes today will draw big eyes six months from today?

Tunnel Vision

What professionals look for is profoundly influenced by their fields of practice, professional interests, and personal experiences. This dynamic has become the source of jokes that take jabs at occupationally induced tunnel vision. Within the field of medicine, surgeons are more likely to identify medical problems for which there are surgical solutions, endocrinologists are more likely to identify problems involving hormone imbalance, and psychiatrists are more likely to identify problems for which there are psychopharmacological solutions. The majority of those who employ drawings as a means by which to detect various emotional difficulties in children, treat children with emotional difficulties. Constructive skepticism is warranted.

Weinberg, writing in the Tenth Mental Measurements Yearbook (1989, J. C. Conoley & J. J. Kramer, J. J. (Eds.)), has called attention to the fact that there are no data to support the notion that normal children and what Weinberg refers to as “clinical” children (i.e., children in need of treatment for emotional difficulties) can be differentiated from one another based upon identified features in their drawings.

The Life Events Rating Scale

Skepticism remains appropriate when practitioners assert that, in examining children's drawings, they are looking for specific features (rather than employing an impressionistic approach), that they utilize a scoring system, and that pre-determined cut-off scores are referenced in the interpretive process.

As we enter our court houses, we pass through devices that detect ferrous metals. No matter what we call these devices, law enforcement's purpose in using them is to detect weapons; not zippers and not metal hooks on brassieres. The sensitivity level of these devices must be set with false alarms (false positives) and detection failures (false negatives) in mind. If the threshold for triggering a signal is too low, small metal items of no interest to law enforcement will yield numerous false alarms. If the threshold setting is too high, someone carrying a weapon may pass undetected. The same principle applies to the setting of cut-off scores in systems used to assign scores to features in drawings.

An early example at attempts to enhance precision through quantification may be instructive. In 1967, Thomas Holmes and Richard Rahe developed a scale named the Social Readjustment Rating Scale by them, but more frequently referred to by others as the Life Events Rating Scale (Holmes, T. H. & Rahe, R. H. 1967. The Social Readjustment Rating Scale. J Psychosomatic Research, 11, 213-318.). The concept underlying the development of the scale was that all significant life events, particularly when clustered (i.e., coming in rapid succession) are an element to be considered in understanding the etiology of illnesses. The reasoning employed by Holmes and Rahe, which was simple and widely accepted by medical researchers, was that significant life events demand adjustments by those who have experienced them, that these adjustments are taxing, that the immune system is compromised to some extent as a result, and, that the compromised immune system leaves individuals more vulnerable to forces (bacteria, viruses) that might be successfully repelled under normal circumstances.

Though the introduction of the concept of social readjustment and its implications for health were enthusiastically welcomed, the scale itself was widely criticized, and with good reason. Operating with an obvious lack of empirical data, Holmes and Rahe endeavored to quantify the stress that would be encountered in adjusting to various life events, both positive and negative. Examples of the Holmes and Rahe assignment of numerical values to the stress associated with adjusting to various life events follow: 1) The death of a spouse is assigned 100 points and this is the highest numerical value on the scale; 2) The next event, in descending order, is divorce, which is assigned a numerical value of 73; 3) Marriage is assigned a value of 50; 4) Pregnancy is assigned a value of 40; 5) Also assigned a value of 40 are “sex difficulties”; 6) Foreclosure is assigned a value of 30; 7) The item with the lowest assigned value is “minor violation of the law,” which is assigned a value of 11.

The foregoing examples are presented in the hope that an examination of the items and the accompanying assigned values will bring the reader to recognize that if different people were to be asked to perform the task of assigning adjustment-stress values to these life events, significant disagreement would be likely.

In the course of explaining the perceived advantages of test data (as contrasted with other forms of data gathered in custody evaluations), Dianna Gould-Saltman, an experienced family law attorney, stated that attorneys “understand numbers,” in an article written for the Journal of Child Custody, in 2005. If Gould-Saltman's statement is taken as I believe it was intended, as a general statement about attorneys, I must respectfully disagree. As a rule, attorneys do not understand concepts such the standard error of measurement, and, as a result, are unlikely to know when the difference between two scores on a test is significant and when it is likely to be insignificant. It is the belief, held by attorneys and others, that numbers are understandable that leads people to trust numbers. By extension, when numbers are cited in support of various propositions, their persuasive power is often significant.

The Sign Approach

Mental health professionals who employ scoring systems in their efforts to interpret drawings are employing what is known as the sign approach to figure drawing interpretation. When this approach is being employed, certain features in the drawings are viewed as signs (indicators) of various personality characteristics, life experiences, or perceptions of others. Lally has written: “The data on the [inter-judge] reliability of the specific sign approach are not good ' [They] have reliabilities that are probably too low for making reasonably reliable clinical judgments” (Lally, S. J. (2001). Should Human Figure Drawings Be Admitted Into Court? J Personality Assessment, 76(1), 135-149, p. 140]. In the writing of advocates of the sign method, there is evidence that they do not always agree among themselves concerning what a particular feature indicates. Peterson and her colleagues have opined that large eyes may be drawn by children who have seen or experienced physical abuse (Peterson, L. W. & Hardin, M. E. (1997): Children in Distress: A Guide for Screening Children's Art. NY: Norton; Peterson, L. W., Hardin, M. E., & Nitsch, M. J. (1995). The use of children's drawings in the evaluation and treatment of child sexual, emotional, and physical abuse. Arch Fam Med, 4, 445-452.). Cantlay, on the other hand, opines that distress and trauma may be reflected in atypically small eyes (Cantlay, L. (1996). Detecting Child Abuse: Recognizing Children at Risk Through Drawings. Santa Barbara: Holly Press).

Some of those who develop a dependence upon computers voluntarily relinquish their exercise of common sense when it appears to conflict with a computer-generated solution. Similar dynamics can be seen in the work of mental health professionals who employ scoring systems in interpreting children's drawings. An example follows. An evaluator employs a system in which points are assigned to various features that are suggestive of various emotional problems, in the views of the developers of the system. In this system, one point is assigned for “Writing words on picture.” A child returns from Disneyland on a Monday and enters the evaluator's office on the following Wednesday wearing a t-shirt with “Disneyland” written on it. In responding to the instruction that she draw a person, the child draws a female with a strong resemblance to the child, wearing a t-shirt with “Disneyland” written across the front. The evaluator, adhering to the 'rules' of the scoring system that she is employing, assigns a point to the drawing. Even if we presume, and I do not, that interpretations of children's drawings constitute a useful data source, is the broader task of understanding the psychology of the child being evaluated facilitated by viewing this feature of the child's drawing as being suggestive of abnormality? Would it not be more helpful to chat with the child about the drawing, her t-shirt, and her visit to Disneyland and to use the child's statements as the data source to be drawn upon?

Quantitative Measurement

Quantitative measurement is rendered useless when people who believe themselves to be measuring the same thing are, in fact, measuring different things. Consider a situation in which two individuals are asked to measure the dimensions of a picture frame and, when their results are compared, there is a discrepancy of roughly two inches in the measure of width and two inches in the measure of length. Inquiry reveals that one of the individuals measured the frame from its outer edges and that the other individual measured the frame from its inner edges. Even in situations in which measures can be taken with reasonable precision, different individuals may generate different results if there is lack of clarity with regard to what it is that is being measured. In the example that I have provided, it is likely that discrepant results could have been prevented if each participant had been provided with the same instructions and if those instructions had been precise. Mental health professionals, and attorneys who cross-examine them, would be wise to consider the applicability of this example for psychological assessment.

Anecdotal evidence (primarily in the form of testimony offered by mental health professionals concerning their assessment methods) suggests that evaluators are alarmingly inconsistent in directions provided to children from whom they wish to obtain drawings. It is not uncommon to encounter, in testimony, assertions from evaluators that they do not recall exactly what was said by them to children prior to the production of drawings by the children. What a child draws can be dramatically affected by the child's interpretation of the examiner's wishes. In some projective tasks, ambiguity in instructions is deliberate and the evaluee's interpretation of the instructions (as inferred based upon the evaluee's response) is an element of the projective assessment. It must be emphasized, however, that ambiguity and inconsistency are not synonymous. Where ambiguity is desired, the wished-for level of ambiguity is achieved by means of consistent instructions.

In the Draw A Person test, the instructions are: “Draw a person.” Providing that instruction is quite different from: 1) “Draw something”; 2) “Draw a man”; 3) “Draw a woman”; 4) “Draw a boy”; or “Draw a girl.” Similarly, the instruction “Draw a house” is not the same as “Draw your house.” The accuracy of this statement is not undercut by the reality that users of the H-T-P (House-Tree-Person test) would undoubtedly explain that when children are asked to draw a house, that which they produce is likely to be linked in some manner (realistically or symbolically) to their own houses.

Directions may be of critical importance when drawing tasks are used in the context of custody evaluations in assessing children who may be members of two families. Consider Joey who spends part of his time with his mother and part of his time with his father (who is residing with a woman and her daughter). Evaluators who are skilled in investigating allegations of child sexual abuse take great care to insure that there is no miscommunication concerning the meaning of various words. We must consider the possibility that evaluators employing drawing tasks in routine custody cases do not employ similar care.

Consider the possibility that, as Joey understands the word, a “family” must include a male parent figure and a female parent figure. When asked to draw his family, Joey draws himself, his father, his father's paramour, and the paramour's daughter. Consider, further, the possibility that the evaluator does not discuss Joey's drawing with him, but, instead, files the drawing with the intention of interpreting it at some future time. When the evaluator interprets the drawing, she concludes that Joey feels closer with his father (and those now residing with his father) than with his mother. Is this a valid interpretation?

In offering general commentary on the Draw A Person (DAP) task, Willis, writing in the Eleventh Mental Measurements Yearbook (1992 ' J. J. Kramer & J. C. Conoley (Eds.)), opined that “the DAP was neither intended nor validated as a personality measure, and it clearly should not be used for this purpose.”

Knoff, writing in the Sixth Mental Measurements Yearbook (1965 ' O. K. Buros (Ed.)) on the subject of scoring systems for the House-Tree-Person test, called attention to the fact that the author of the manual being reviewed asserts that the manual “is designed to provide the diagnostician with an objective, quantifiable scoring system that has a well-grounded theoretical and research base.” Knoff opined: “Unfortunately, neither of these assertions is true. This scale makes numerous, faulty assumptions as to the theoretical and research base that forms the foundation of this system, and there are many significant shortcomings in the system's norming process, its normative data, and in its psychometric properties.”

Knoff also called attention to the fact that “the author [of the manual] fails to note that none of [the currently used scoring] systems have been independently validated ' ” He added that “ projective drawing research is largely nonempirical and case study oriented, of questionable methodological and technical integrity, and lacking in convergent and discriminant validity ' .” He opined that “the failure to document or demonstrate the discriminant validity for each of the projective indicators calls the entire system into question.”

Dowd, also writing in the Sixth Mental Measurements Yearbook, pointed out that there is no empirical support for the very underpinnings of drawing interpretation ' the notion (often expressed as a given) that feature X in a drawing symbolically represents Y in the personality of or life of the person who has produced the drawing. Offering an example, Dowd observed that there are no data to support the notion that a “door on the side of a building” is suggestive of withdrawal.

Cosden, writing in the Eleventh Mental Measurements Yearbook, called attention to the fact that no particular scoring system has been agreed upon, that several different systems are in use, and that the “advantage of one set of scoring criteria over another has not been demonstrated empirically.”

In a review appearing in the Tenth Mental Measurements Yearbook, Weinberg, summarizing the available data on quantitative analyses, stated: “In a nutshell, the case (for psychometric use of KFDs) continues to be weak (citations omitted).” “Available data do not support the notion that criteria utilized in four different scoring systems can be conceptualized as objective criteria.” He pointed out that “Poor test-retest stability is reported” and reminded readers of the review that even proponents of drawing tasks acknowledge that certain features “'are sensitive to children's transitory personality states.'” Weinberg observed that “the accumulated psychometric underpinnings for projective drawings ' are lean” and opined that “the age of accountability ' demands that greater psychometric credibility be demonstrated for kinetic drawings.” He concludes, and I agree, that drawing tasks are useful as an “ice-breaker.”

In the Tenth Mental Measurements Yearbook, Cundick points out that the authors of the manual being reviewed do not purport to have a test; rather they indicate that their approach “primarily involves a hypothesis-generating and hypothesis-testing model ' .” Cundick states that “the clinician chooses hypotheses to entertain and test further based on his or her knowledge and experience with the child.” No harm results when treating practitioners utilize drawings to generate hypotheses and, subsequently, explore those hypotheses in the course of a therapeutic relationship. Great harm may be done when forensic practitioners utilize drawings not to generate hypotheses but to formulate opinions, and when those opinions are shared with judges who will be making decisions in custody/access disputes.

David Martindale, a member of this newsletter's Board of Editors, board certified in forensic psychology by the American Board of Professional Psychology, served as the Reporter for the Association of Family and Conciliation Courts' Model Standards for Child Custody Evaluation and is the co-author, with Jon Gould, of The Art and Science of Child Custody Evaluations. His practice is limited to consulting with attorneys, psychologists, and psychology licensing boards in the areas of child custody and professional ethics and standards. He can be contacted by e-mail at [email protected].