Psychology 2100: Introduction to Social Psychology

Warren Thorngate, Professor
Psychology Department, Carleton University
1125 Colonel By Drive
Ottawa, Ontario K1S 5B6
Canada
e-mail = warren_thorngate@carleton.ca

copyright 1999-2006 by Warren Thorngate, all rights reserved 

Lecture 4: The Question of Accuracy

How accurate are we?

Anecdotal evidence of inaccuracies:

• Hiring wrong applicant, giving scholarship to a known cheater, marrying the wrong person, etc. 
• Arguments in marriage resulting from misperception of spouse's words or deeds 
• use of marriage counsellors to develop better communication = more valid processing of cues
• inaccuracies of judgments from stereotypes (judgements made on the basis of group membership)
• Many stereotypes are accurate for majority/plurality of a group
• But those who do not fit the stereotype are misjudged
• Example from class demonstraton
• What is wrong with misjudging people? 
• Often nothing. Misjudgments do not result in any bad behaviours or, if they do they are easily rectified with "I am sorry." and a subsequent change in behaviour
• Sometimes a lot. Misjudgements lead to bad behaviour which leads to misjudgments by others which escalates bad behaviour...  Some examples from the newspapers
• Mel Hurtig's hair 
• air-kiss 
• 73-30 or Che?
• child abuse 
• employee tests 
• driving lesson 
• veiled threat 
• Canadian stereotypes of others
• macleans rankings

Research evidence: Two of hundreds of similar studies

Stuart Oskamp, "Overconfidence in case study judgements"(Journal of Consulting Psychology, 1965, 29, 261-265.)

• Oskamp wanted to know if trained psychologists are more accurate at assessing verbal descriptions of someone than are untrained students. 
• He asked 8 clinical psychologists (5 with PhDs, and all with "several" years of clinical experience), 18 psychology graduate students, and 6 advanced undergraduates in a personality class to participate in his experiment. 
• Each participant was given excerpts from "The Case of Joseph Kidd" -- a chapter in Robert White's Lives in Progress giving a detailed account of the life and problems of a 29 year old male. The chapter is about 50 pages long. Each judge read (1) 35 words from the first page, then (2) 750 words about Kidd's childhood through age 12, then (3) 1,000 words about Kidd's high school and university years, and (4) 600 words about his army service and life to age 29. 

• After reading each of excerpts 1, 2, 3 and 4, each participant was asked to complete a 25-item multiple choice quiz about Joseph Kidd. The questions and correct answers were constructed from the 45+ pages of material that participants did not read. The same questions were given after reading each of the four excerpts; thus the same quiz was taken by each participant four times. 
• For your information, here are two of the 25 multiple-choice questions on the quiz: 
• During College, when Kidd was in a familiar and congenial social situation, he often 
  a. Tried to direct the group and impose his wishes on it
  b. Stayed aloof and withdrawn from the group
  c. Was quite unconcerned about how people reacted to him 
  d. Took and active part in the group but in a quite and modest way
  e. Acted the clown and showed off 
• Kidd's present attitude towards his mother is one of 
  a. Love and respect for her ideals
  b. Affectionate tolerance for her foibles
  c. Combined respect and resentment
  d. Rejection of her and all her beliefs 
  e. Dutiful but perfunctory affection
  
  
• Oskamp's Results: 

• Were the professional psychologists better than the others on the test? No. On average, the clinical psychologists, graduate students and undergraduates did not differ significantly in their quiz scores across the four times they took the quiz. 
• Did the judges improve their test score after reading more information? No. The average score on the first quiz was 26%, not far above the 20% we would expect by chance. The average score on the remaining three quizzes was 23%, 28% and 28%, respectively. Of the 32 judges, 14 received higher scores in quiz 4 than in quiz 1, 6 remained the same, and 12 went down. 
• Did the judges answers change after receiving more information? A little. On average, judges changes 13 of their 25 answers from quiz 1 to 2, 11 of their answers from 2 to 3, and 8 of their answers from 3 to 4. Thus, the judges became more fixed in their judgements with more information, even though they did not improve their accuracy! 
• Did the judges believe they were doing well? Yes! Oskamp asked each judge to estimate his/her score on each quiz. The average estimated score on Quiz 1 was 33%, on Quiz 2 it rose to 39%, on Quiz 3 to 46% and on Quiz 4 to 53%. Thus, even though the judges were not, in fact, increasing their accuracy as they read more information about Joseph Kidd, they believed they were! In the end, this made them disturbingly overconfident. 

Lew Goldberg (1969). "The effectiveness of clinicians' judgments: The diagnosis of organic brain damage from the Bender-Gestalt test. "Journal of Consulting Psychology, 23, 25-33.

• Asked 4 psychology staff (4-9 years experience with Bender-Gestalt), 10 psychology trainees (1-4 years experience and 8 secretaries (1 hour experience), to assess 30 Bender-Gestalt profiles, half of which were taken from files of patients at a Veteran's Administration hospital in the USA who had "organic brain damage" (mostly vets with parts of their brain shot out), and half of which showed no organic brain damage (mostly schizophrenic patients). 
• The 30 profiles were thoroughly shuffled/randomized for each of the 4+10+8 = 22 judges. Each judge was individually asked to sort the 30 profiles into two piles, Organics and Nonorganics, according to their assessments of the Bender-Gestalt test profiles. 
• Relation to Brunswik's Lens Model: 
• Brain state = distal stimulus 
• Bender Gestalt test result = proximal stimulus 
• "Clinical Wisdom" = rules of assessment = "The Lens" 
• Organic/Nonorganic diagnosis = the percept or impression 

• Results: (remember, we expect 50% correct judgement just by flipping a coin) 
• Psychology staff average = 65% correct, range = 60-70% 
• Psychology trainee average =70% correct, range = 60-77% 
• Secretary average = 67% correct, range = 57-73% 
• World expert (Max Hutt) = 83% correct 
• Conclusion: even when the test is potentially accurate, and even when "experts" are judging, there are still many errors of judgement. 

Four reasons for inaccuracy in clinical diagnosis and in real life of social judgement:

1. Ignorance 

• We do not have reliable or valid "cues" to judge people (we have not found proximal stimuli that can reliably and validly distinguish different distal stimuli) 
• Especialy true to the extent that people can "manage" their cues to create the social judgements they seek 
• What to do? Seek better tests or diagnostic instruments to improve our judgements 
• The Tests and Measurements industry is based on this assumption (Educational Testing Service, ETS, makes millions from the GRE. GMAT, LSAT, etc. on the claim that their tests can accurately predict success in graduate school. The claim has repeatedly been shown to be false.) 

2. Naivete 

• There are valid cues, but we do not yet have good rules in our head to process correctly the important information 
• What to do? Provide better feedback about our good and bad judgements to allow better training of judges 
• Graduate programmes in diagnostic professions (medical, psychological, legal, etc.) are justified by the belief that the training the provide can reduce naivete 

3. Stupidity 

• Some or all people may lack the intelligence to handle and processes correctly all the information tht is available to them 
• What to do? (1) find "smart" people and train them, leave the "dumb" ones to study gardening. (2) Replace people with computers programmed to exhibit lots of "artificial intelligence" 
• Graduate school competition are typically based on these ideas: only "smart" people (as judged by Grade Point Average, letters of recommendation, GRE scores, etc.) should be admitted to professional graduate schools. The idea of artificial intelligence continues to inspire programmers to develop programmes that can replace humans in diagnosis; some good ones now diagnose blood diseases and common medical problems better than most trained doctors can. 

4. Motivation 

• Judges may simply not care about accuracy. People with lots of power and security may have no strong motivation to be correct. Most employers, for example, don't care if you will suffer by rejecting your application. "Power means never having to say you're sorry." 
• What to do? Consider the rewards and punishments for correct and incorrect judgements: 

If your motivation is to reduce the number of false alarms (errors of commission, errors of inclusion) you make, then you should rarely, perhaps never, judge someone to be good, or you should raise your standards extremely high hoping to judge someone as good if he/she "jumps all the hurdles." Reducing false alarms is the classic goal of political conservatives; they greatly dislike "bad people living in my neighbourhood, bad employees working in my company, bad students studying in my university, etc." However, this strategy will maximize the number of misses (errors of omission, errors of exclusion) you make. In short, you will incorrectly reject a lot of good people. 

If your motivation is to reduce the number of misses (errors of omission, errors of exclusion), then you should rarely, perhaps never, judge someone to be bad, or you should lower your standards of goodness hoping to judge someone as good if he/she "jumps one hurdle." Reducing misses is the classic goal of political liberals; they greatly dislike "good people excluded from my neighbourhood, good employees not hired in my company, bright students left out of my university, etc." Carleton's "Last Chance U." l abel reflects this classic liberal education philosophy. However, this strategy will maximize the number of false alarms (errors of commission, errors of inclusion) you make. In short, you will incorrectly accept a lot of bad people. 

• What to do? Revolutions are begun by the "Misses"" talented and good people overlooked or rejected by society. Many of their followers are correct rejections. When revolutionaries win, they often lower the criteria for judging goodness. Many errors of commission are made. The revolution fails. This cycle may always repeat itself. The ultimate solution to all social problems may be to take turns being oppressed.