Test Item Transparency

We hypothesize that if attorneys, judges, and psychologists were asked to rate the usefulness of data from psychological tests administered in the context of family law matters, attorneys and judges would assign higher ratings than would psychologists. Dianna Gould-Saltman, a former family law attorney, now a judge sitting in the Los Angeles Superior Court, has opined that attorneys like tests because they “break things down to numbers, and we understand numbers.” [Gould-Saltman's views were expressed in an article entitled "Testing, One, Two, Three, Testing: An Attorney Perspective," appearing in the Journal of Child Custody, 2(1&2), 2005, pp. 71-81.]

Some of the problems inherent in psychological testing conducted in the context of litigated custody disputes have been addressed previously in The Matrimonial Strategist [Martindale, D. A. (2005). Psychological Assessment: Evaluating the Evaluations. The Matrimonial Strategist, 22:12, 3-5]. Here, we address the problems that are inalterably linked to test-taker motivation and to the susceptibility [and possible ease] of psychological test data to deliberate distortion. Though the topic of test-taker motivation has been addressed at length in many articles and text chapters, the susceptibility of psychological test data to deliberate distortion has received less attention, and methods for addressing the problem are not as effective as non-psychologists are led to believe they are.

Transparency

The term item “transparency” refers to the ease with which test-takers can discern what aspects of their perceptions, emotions, cognitions, or patterns of behavior are likely to be revealed by their responses to specific test items. When test-takers can, with relative ease, discern what will be revealed about them by their responses to certain test items, they can offer the responses that will generate data which, when scored and interpreted, will create the impression that the test-takers wish to create. It is for this reason that some of the test items that are included for the specific purpose of identifying dissimulators are said to measure attempts at impression management.

To the best of our knowledge, the concept of item transparency was introduced within the context of a presentation by Flens entitled “Psychological Testing in Child Custody Evaluations,” offered in Atlanta, in 2006, under the auspices of the Association of Family and Conciliation Courts.

Response Styles

Without any exception of which we are aware, individuals about to take tests commence the test-taking process with a goal, or mind-set. This mind-set is likely to be based on the context in which the test is to be taken. This mind-set reveals itself in response patterns that are referred to as response styles, because it is the mind-set of test takers that determines the manner (style) in which they will respond to the test. There are four basic response styles. Unfortunately, those who have written on the subject have not always agreed on what terminology to use in describing these response styles, so there are some discrepancies in terminology; however, there is general agreement with regard to the underlying concepts.

The response style that test users hope to encounter has been referred to as the “honest” or “reliable” response style. Test takers who commence the task with this response style are motivated by a desire to assist the test-giver in obtaining an accurate picture. Particularly with regard to mental and emotional self-examination, our perceptions of ourselves are not always accurate; nevertheless, test-takers responding honestly do their best to describe their perceptions, thoughts, actions, and motivations candidly and to communicate in a manner that will yield reliable data.

In settings such as those in which family law attorneys encounter psychological test data ' settings such as disputes regarding custody of or access to children ' there is often good reason for concern that test-takers may have engaged in active efforts to present themselves as being less burdened by human shortcomings than they know themselves to be, and more endowed with admirable traits than they know themselves to be. Test-takers who engage in what we often simply refer to as “faking good” are employing a response style that has been referred to as “defensive.” Other terms that have been used in describing this response style are “dissimulation” and “positive impression management.” The defensiveness is an intentional attempt to present a favorable impression, much like “putting your best foot forward” in a job interview.

As this dynamic has become a problem of interest to test designers and to test data interpreters, we have learned that some of the test takers that we thought were trying to mislead test-givers have actually been endeavoring to mislead themselves ' to convince themselves that they are more virtuous than they really are and less flawed than they really are. The term most often used in describing this dynamic is “self-deceptive enhancement.” In such situations, individuals truly believe they are more virtuous; therefore, self-deceptive enhancement is conceptualized as an element of the personality of the test-taker.

Though cases in which test-takers wish to fake bad are encountered far more frequently by attorneys in the field of tort law, mention must be made of the response style most frequently described by the terms “malingering,” “feigning,” or “negative impression management.” As this concept applies to psychological tests, individuals who are malingering endeavor to respond to test items in such a manner as to create the impression that they have a mental/emotional problem that they do not have, or to exaggerate significantly the symptoms associated with a mental/emotional problem that they do have.

Finally, there are test-takers who, for a variety of reasons, are simply not engaged in the process. They may commence the test-taking process with no objective other than to get it over with. On certain tests, individuals displaying this response style are identified with reasonable ease because certain responses are inconsistent with others. Terms that have been used by different authors in discussing this response style include “irrelevant,” “uncooperative,” “suboptimal,” “disengaged,” and “careless.”

Test Item Transparency

When custody litigants wishing to engage in positive impression management encounter test items that are transparent, problems are created for those who are counting on psychological test data to provide reasonably accurate information concerning those being evaluated [Not limited to impression management].

The custody litigant who asserts, in an interview context, that he spends a great deal of time with his child, is likely to be asked to elaborate. A skilled interviewer posing follow-up questions can obtain information of a type that can be investigated. Through the use of documents and collateral source informants, an evaluator can ascertain whether or not the litigant has been engaged in the activities described during the interview.

Consider what occurs if, instead of inquiring about parent-child activities in an interview, the litigant is asked to complete a self-report inventory and one of the items reads: “I spend a great deal of time with my child.” The litigant's response to this item, along with his responses to other items (many of which are equally transparent) will be reflected in a score. We suggest that Gould-Saltman's observation that tests “break things down to numbers, and we understand numbers” be reconsidered in the light of our example. When information such as that used in our example is gathered by means of a self-report inventory, a score is generated. Though Gould-Saltman has opined that individuals trained in law “understand numbers,” it is our position that the risk that numbers will mislead instead of clarify is greatest when those numbers appear in reports that have been prepared for those who believe that test data are qualitatively superior to interview data because test data are objective.

In the example provided here, information obtained within the context of a well-conducted interview and subsequently subjected to verification is far more likely to be of assistance to a trier of fact than is a test score. We urge readers to consider the many situations in which numbers may create an aura of precision that, when examined, turns out to be misleading.

Ethical Considerations

The psychologists' Ethics Code addresses several elements of the psychological assessment process, and Standard 9.02(a) admonishes psychologists to “administer, adapt, score, interpret, or use assessment techniques, interviews, tests, or instruments in a manner and for purposes that are appropriate in light of the research on or evidence of the usefulness and proper application of the techniques.” We submit that the proper administration of a test, proper scoring of the data, and sound interpretation of the data requires that those using a test be familiar with the manual that accompanies the test. A test manual informs those who read it of the manner in which the test was developed. Far more often than is commonly known, tests currently utilized within the context of child custody litigation were developed for purposes entirely unrelated to any form of litigation. Where this is the case, the implications must be considered.

Context Matters

When individuals seek assistance from psychologists and when, in the course of developing a treatment plan, psychologists administer psychological tests, it is reasonable (though never entirely safe) to presume that those seeking assistance will believe it to be in their interests to respond candidly to the items on whatever tests are administered. Thus, in health service contexts, the issue of item transparency may be of minimal importance.

When, on the other hand, psychological tests are administered within the context of litigation, test-takers are likely to believe that their interests will be best served if they can successfully create an image that is consistent with their litigation strategy. When the matter being litigated involves custody of or access to a child, litigants are motivated to endorse statements descriptive of virtues not characteristic of the litigants and reject as not applicable statements that may, in fact, provide accurate descriptions of psychological problems experienced by them.

Conclusion

When test items are transparent, positive impression management is more likely to occur. We submit that the ease with which test-takers can discern what will be revealed about them by their responses to certain test items is amenable to research. The usefulness of many tests now used in forensic settings would be increased significantly if test developers would ascertain and report transparency indices for test items.

David A. Martindale, Ph.D., ABPP, a member of this newsletter's Board of Editors, is board certified in forensic psychology and is the co-author, with Dr. Jon Gould, of The Art & Science of Child Custody Evaluations. He currently serves as a member of the AFCC Task Force on Child Custody Consultation. Additional information is at www.damartindale.com. James R. Flens, Psy.D., ABPP, is board certified in forensic psychology and is a Fellow of the Society for Personality Assessment. His private practice is in Brandon, FL, where he specializes in family law-related evaluations, consultation both with attorneys and custody evaluators, and research regarding the use of psychological testing within the context of family law litigation. He may be reached by e-mail at [email protected].

We hypothesize that if attorneys, judges, and psychologists were asked to rate the usefulness of data from psychological tests administered in the context of family law matters, attorneys and judges would assign higher ratings than would psychologists. Dianna Gould-Saltman, a former family law attorney, now a judge sitting in the Los Angeles Superior Court, has opined that attorneys like tests because they “break things down to numbers, and we understand numbers.” [Gould-Saltman's views were expressed in an article entitled "Testing, One, Two, Three, Testing: An Attorney Perspective," appearing in the Journal of Child Custody, 2(1&2), 2005, pp. 71-81.]

Some of the problems inherent in psychological testing conducted in the context of litigated custody disputes have been addressed previously in The Matrimonial Strategist [Martindale, D. A. (2005). Psychological Assessment: Evaluating the Evaluations. The Matrimonial Strategist, 22:12, 3-5]. Here, we address the problems that are inalterably linked to test-taker motivation and to the susceptibility [and possible ease] of psychological test data to deliberate distortion. Though the topic of test-taker motivation has been addressed at length in many articles and text chapters, the susceptibility of psychological test data to deliberate distortion has received less attention, and methods for addressing the problem are not as effective as non-psychologists are led to believe they are.

Transparency

The term item “transparency” refers to the ease with which test-takers can discern what aspects of their perceptions, emotions, cognitions, or patterns of behavior are likely to be revealed by their responses to specific test items. When test-takers can, with relative ease, discern what will be revealed about them by their responses to certain test items, they can offer the responses that will generate data which, when scored and interpreted, will create the impression that the test-takers wish to create. It is for this reason that some of the test items that are included for the specific purpose of identifying dissimulators are said to measure attempts at impression management.

To the best of our knowledge, the concept of item transparency was introduced within the context of a presentation by Flens entitled “Psychological Testing in Child Custody Evaluations,” offered in Atlanta, in 2006, under the auspices of the Association of Family and Conciliation Courts.

Response Styles

Without any exception of which we are aware, individuals about to take tests commence the test-taking process with a goal, or mind-set. This mind-set is likely to be based on the context in which the test is to be taken. This mind-set reveals itself in response patterns that are referred to as response styles, because it is the mind-set of test takers that determines the manner (style) in which they will respond to the test. There are four basic response styles. Unfortunately, those who have written on the subject have not always agreed on what terminology to use in describing these response styles, so there are some discrepancies in terminology; however, there is general agreement with regard to the underlying concepts.

The response style that test users hope to encounter has been referred to as the “honest” or “reliable” response style. Test takers who commence the task with this response style are motivated by a desire to assist the test-giver in obtaining an accurate picture. Particularly with regard to mental and emotional self-examination, our perceptions of ourselves are not always accurate; nevertheless, test-takers responding honestly do their best to describe their perceptions, thoughts, actions, and motivations candidly and to communicate in a manner that will yield reliable data.

In settings such as those in which family law attorneys encounter psychological test data ' settings such as disputes regarding custody of or access to children ' there is often good reason for concern that test-takers may have engaged in active efforts to present themselves as being less burdened by human shortcomings than they know themselves to be, and more endowed with admirable traits than they know themselves to be. Test-takers who engage in what we often simply refer to as “faking good” are employing a response style that has been referred to as “defensive.” Other terms that have been used in describing this response style are “dissimulation” and “positive impression management.” The defensiveness is an intentional attempt to present a favorable impression, much like “putting your best foot forward” in a job interview.

As this dynamic has become a problem of interest to test designers and to test data interpreters, we have learned that some of the test takers that we thought were trying to mislead test-givers have actually been endeavoring to mislead themselves ' to convince themselves that they are more virtuous than they really are and less flawed than they really are. The term most often used in describing this dynamic is “self-deceptive enhancement.” In such situations, individuals truly believe they are more virtuous; therefore, self-deceptive enhancement is conceptualized as an element of the personality of the test-taker.

Though cases in which test-takers wish to fake bad are encountered far more frequently by attorneys in the field of tort law, mention must be made of the response style most frequently described by the terms “malingering,” “feigning,” or “negative impression management.” As this concept applies to psychological tests, individuals who are malingering endeavor to respond to test items in such a manner as to create the impression that they have a mental/emotional problem that they do not have, or to exaggerate significantly the symptoms associated with a mental/emotional problem that they do have.

Finally, there are test-takers who, for a variety of reasons, are simply not engaged in the process. They may commence the test-taking process with no objective other than to get it over with. On certain tests, individuals displaying this response style are identified with reasonable ease because certain responses are inconsistent with others. Terms that have been used by different authors in discussing this response style include “irrelevant,” “uncooperative,” “suboptimal,” “disengaged,” and “careless.”

Test Item Transparency

When custody litigants wishing to engage in positive impression management encounter test items that are transparent, problems are created for those who are counting on psychological test data to provide reasonably accurate information concerning those being evaluated [Not limited to impression management].

The custody litigant who asserts, in an interview context, that he spends a great deal of time with his child, is likely to be asked to elaborate. A skilled interviewer posing follow-up questions can obtain information of a type that can be investigated. Through the use of documents and collateral source informants, an evaluator can ascertain whether or not the litigant has been engaged in the activities described during the interview.

Consider what occurs if, instead of inquiring about parent-child activities in an interview, the litigant is asked to complete a self-report inventory and one of the items reads: “I spend a great deal of time with my child.” The litigant's response to this item, along with his responses to other items (many of which are equally transparent) will be reflected in a score. We suggest that Gould-Saltman's observation that tests “break things down to numbers, and we understand numbers” be reconsidered in the light of our example. When information such as that used in our example is gathered by means of a self-report inventory, a score is generated. Though Gould-Saltman has opined that individuals trained in law “understand numbers,” it is our position that the risk that numbers will mislead instead of clarify is greatest when those numbers appear in reports that have been prepared for those who believe that test data are qualitatively superior to interview data because test data are objective.

In the example provided here, information obtained within the context of a well-conducted interview and subsequently subjected to verification is far more likely to be of assistance to a trier of fact than is a test score. We urge readers to consider the many situations in which numbers may create an aura of precision that, when examined, turns out to be misleading.

Ethical Considerations

The psychologists' Ethics Code addresses several elements of the psychological assessment process, and Standard 9.02(a) admonishes psychologists to “administer, adapt, score, interpret, or use assessment techniques, interviews, tests, or instruments in a manner and for purposes that are appropriate in light of the research on or evidence of the usefulness and proper application of the techniques.” We submit that the proper administration of a test, proper scoring of the data, and sound interpretation of the data requires that those using a test be familiar with the manual that accompanies the test. A test manual informs those who read it of the manner in which the test was developed. Far more often than is commonly known, tests currently utilized within the context of child custody litigation were developed for purposes entirely unrelated to any form of litigation. Where this is the case, the implications must be considered.

Context Matters

When individuals seek assistance from psychologists and when, in the course of developing a treatment plan, psychologists administer psychological tests, it is reasonable (though never entirely safe) to presume that those seeking assistance will believe it to be in their interests to respond candidly to the items on whatever tests are administered. Thus, in health service contexts, the issue of item transparency may be of minimal importance.

When, on the other hand, psychological tests are administered within the context of litigation, test-takers are likely to believe that their interests will be best served if they can successfully create an image that is consistent with their litigation strategy. When the matter being litigated involves custody of or access to a child, litigants are motivated to endorse statements descriptive of virtues not characteristic of the litigants and reject as not applicable statements that may, in fact, provide accurate descriptions of psychological problems experienced by them.

Conclusion

When test items are transparent, positive impression management is more likely to occur. We submit that the ease with which test-takers can discern what will be revealed about them by their responses to certain test items is amenable to research. The usefulness of many tests now used in forensic settings would be increased significantly if test developers would ascertain and report transparency indices for test items.

David A. Martindale, Ph.D., ABPP, a member of this newsletter's Board of Editors, is board certified in forensic psychology and is the co-author, with Dr. Jon Gould, of The Art & Science of Child Custody Evaluations. He currently serves as a member of the AFCC Task Force on Child Custody Consultation. Additional information is at www.damartindale.com. James R. Flens, Psy.D., ABPP, is board certified in forensic psychology and is a Fellow of the Society for Personality Assessment. His private practice is in Brandon, FL, where he specializes in family law-related evaluations, consultation both with attorneys and custody evaluators, and research regarding the use of psychological testing within the context of family law litigation. He may be reached by e-mail at [email protected].