ORGANIZATIONAL BEHAVIOR AND HUMAN DECISION PROCESSES, Vol. 75 Issue 03

ORGANIZATIONAL BEHAVIOR AND HUMAN DECISION PROCESSES

Vol. 75, No. 3, September, pp. 207–221, 1998

ARTICLE NO.

OB982789

Affective Disposition and the Letter of Reference

Timothy A. Judge and Chad A. Higgins

Department of Management and Organizations, College of Business Administration,

University of Iowa

The present study attempted to determine the relationship be-

tween affective disposition and the favorability of letters of refer-

ence. We hypothesized that individuals with positive dispositions

would write more favorable letters than would individuals with

less positive or neutral dispositions. In addition, we also hypothe-

sized that length of letter would partly mediate the relationship

between affective disposition and letter favorability. To test these

hypotheses, two studies were conducted. In order to present let-

ter writers with a controlled stimulus, Study 1 entailed having

110 undergraduate students write letters of reference for two

hypothetical job candidates in a laboratory setting. In order to

test the generalizability of the laboratory study, Study 2 involved

95 faculty members who submitted three recent letters of refer-

ence they had written. Results showed that affective disposition

was related to the favorability of letters of reference in both

studies.Results alsorevealedthatlengthofletterpartlymediated

the relationship between affective disposition and letter favor-

ability.

q 1998 Academic Press

The reference report is somewhat of an enigma. On the one hand, it is one

of the most widely used methods of personnel selection. Muchinsky’s (1979)

review of the literature indicated that between 82 and 99% of employers use

reference reports in their selection decisions. Furthermore, most employers

consider references an essential part of the selection process, particularly for

high-skill or professional jobs (Sleight & Bell, 1954). On the other hand, despite

widespread use, reference reports also appear to rank among the least valid

The authors thank Murray Barrick and Frank Schmidt for comments on an earlier version of

this paper, and Liz Higgins, Brian McNatt, Greg Patton, and Carl Thoresen for their assistance

with the studies.

Address correspondence and reprint requests to Timothy A. Judge, Department of Management

and Organizations, College of Business Administration, University of Iowa, Iowa City, IA 52242.

Electronic mail may be sent via Internet at [email protected].

207

0749-5978/98 $25.00

All rights of reproduction in any form reserved.

208

JUDGE AND HIGGINS

selection measures (Muchinsky, 1979). Unlike other selection methods, very

little research has investigated factors affecting the validity of reference re-

ports. As Muchinsky (1979) noted nearly 20 years ago, “Of all the more com-

monly used personnel selection devices, reference reports are the most under-

researched. A thorough search of psychology, management, and personnel liter-

ature reveals a paucity of studies on reference reports with empirical studies

being particularly scarce” (p. 287). Despite repeated calls for more studies on

references (Smith & George, 1992), it remains a neglected area of research.

There have been a few exceptions (Browning, 1968; Knouse, 1983; Mosel &

Goheen, 1958, 1959; Peres & Garcia, 1962). One general conclusion from this

small body of research is that most reference reports are highly favorable to

the subject of the reports. Indeed, this lack of variation is one factor that is

commonly argued to account for the poor validity of reference reports (Muchin-

sky, 1979). Another factor argued to lead to poor validity is the low inter-rater

reliability that exists between references (Muchinsky, 1979). In terms of one

of the principal forms of reference reports, letters of recommendation, Baxter,

Brock,Hill, andRozelle (1981) found that letterwriters provide highly personal-

ized information that is not shared by other writers even if the subject of the

letters is the same. In interpreting their results, Baxter et al. (1981) noted,

“The results are more directly a function of letter writers’ idiosyncrasies than

of the students’ qualities” (p. 300). Baxter et al.’s study provides an interesting

suggestion: If the letter of reference is, in part, a reflection of the writer, it is

relevant to ask what characteristics of the letter writer might be related to

the written product.

The purpose of the present study is to investigate one aspect of the reference

provider, affective disposition, which represents the relatively stable tendency

to evaluate the environment based on one’s emotional make-up (Judge, 1992).

Although reference reports can take many forms, our focus in this study is on

one of the most common: the letter of reference. Drawing from theory in the

area of person perception, and related research in the areas of performance

evaluation and the employment interview, we hypothesize a link between af-

fective disposition and the favorability of letters of reference. Literature sup-

porting this hypothesis is reviewed next.

Because writing letters of reference is the product of an evaluative process,

similarin thisrespect to performanceappraisal or interviewerdecision-making,

research in these areas may shed light on the possible dispositional basis of

letters of reference. Like the reference letter, performance appraisals are sub-

ject to the cognitiveprocesses the rater uses in reaching decisions about subjects

(Landy & Farr, 1980). As Wherry and Bartlett (1982) proposed in their theory

of rating, evaluations about others can be a function of bias in how original

perceptions about others are evaluated when they are retrieved from memory.

In introducing his model of cognitive processes in the performance appraisal

process, Feldman (1981) noted that dispositional factors in the rater can influ-

ence these cognitive processes. It seems likely that one of these dispositional

AFFECTIVE DISPOSITION

209

factors is whether the rater tends to evaluate objects, events, or people in a

generally positive or negative manner.

In fact, basic person perception research has revealed that affect plays a

central role in the processing of information about others. Research by Fiske

(1982) shows that affective labels are attached to factual information about

persons which is stored in memory. When thoughts about a particular person

are accessed from memory, the perceivers’ affective orientation toward the

target may be the first piece ofinformation retrieved. Accordingto Longenecker,

Jaccoud, Sims, and Gioia (1992), two types of affect may influence the cognitive

processing of information about employees or job applicants. One type of affect

is differentiated affect. This type of affect is dependent upon who (i.e., employee

or applicant) constitutes the target and is typically operationalized as liking.

As the target of observation changes, so too does the differentiated affect of

the perceiver. Research indicates that liking is an important correlate of perfor-

mance (Tsui & Barry, 1986) and interview (Dipboye, 1992) ratings.

The other type of affect discussed by Longenecker et al. (1992) is undifferenti-

ated affect. Undifferentiated affect represents general feelings not specifically

directed at a single person such as mood (state affect) or affective disposition

(trait affect). Although much less research has been done on undifferentiated

affect, several studies have linked mood of the evaluator to decisions about

subordinates (Longenecker et al., 1992) and job applicants (Baron, 1993), as

well as to evaluations of job satisfaction (Brief, Butcher, & Roberson, 1995).

These areas of research suggest a possible dispositional basis of letters of

reference. If undifferentiated affect influences performance ratings and inter-

viewer evaluations, it seems reasonable to expect that similar tendencies will

be displayed on the part of letter writers. In short, the same affect-laden

cognitive processes that influence supervisors and interviewers should also

generalize to letter writers. Writing letters of reference forces writers to retrieve

information previously acquired and encoded into memory and it is likely that

affective information is acquired, encoded, and retrieved as a result of this

person perception process (Fiske, 1982). Because affective disposition is re-

flected in the tendency to be accepting versus critical of others, it seems likely

it will influence the evaluation of information about others (Judge, 1992),

including how letter writers evaluate the subject of their letters. In fact, a

recent study supports this argument by finding that affective disposition was

correlated with faculty member’s evaluation of job candidates and their ratings

of co-workers (Merritt & Davis, 1994).

Thus, there is some indirect evidence that suggests a positive relationship

between affective disposition and the favorability of letters of reference. In

short, if variation in letters of reference is directly a function of individual

differences among writers of the letters (Baxter et al., 1981), it seems likely

that one of those individual differences is the affective disposition of the

writer. Accordingly,

H1: Affective disposition will be related to the favorability of letters of reference such that

positively disposed writers will write more favorable letters than negatively disposed writers.

210

JUDGE AND HIGGINS

Influence of Length of the Reference Letter

One issue that has not been discussed to this point is what a favorable letter

of reference means. Typically, one would consider favorable letters to be those

that contained many positive descriptions and few negative ones. In fact, Peres

and Garcia (1962) found that the favorability of letters could be described in

terms of the adjective descriptions contained in the letters. However, given the

positive tone of most letters, others have suggested that length of the letter

may be more indicative of the true attitude of the letter writer. As Wiens,

Jackson, Manaugh, and Matarazzo (1969) argued, “. . . number of words written

in a letter of recommendation might be more explicit as an index of a writer’s

true attitude toward the person about whom he was writing than was the

content of the letter per se” (p. 264). Their results, as well as those of an earlier

study (Mehrabian, 1965), support this argument. On the basis of this evidence,

it seems reasonable to expect that length of the letter of reference will be

related to its perceived favorability. Furthermore, if affective temperament is

related to the favorability of the reference letter, and if length of the letter is

indicative of its favorability, it seems likely that positively disposed individuals

will write longer letters than those less positively disposed. In such a situation,

length of the reference letter could be expected to mediate, at least in part, the

relationship between affective disposition and the favorability of the reference

letter. Accordingly,

H2: Length of the letter of reference will: (a) be positively related to its favorability; and (b)

partially mediate the relationship between affective disposition and letter favorability.

METHOD OVERVIEW

The hypotheses were tested using two separate studies. The first study

involved a laboratory experiment in which subjects wrote letters of reference

for two hypothetical bank employees. The second study was a field study in

which participants were asked to contribute actual letters of reference they

had written. The laboratory study offers the advantage of presentation of a

controlled stimulus such that letter writers were evaluating the same person

in the same context. However, because the laboratory study may lack realism,

a field study was conducted as it has the advantage of studying actual letters

of reference.

In estimating the relationship between affective disposition and the favor-

ability of letters of reference, several control variables were taken into account.

Past research suggests work experience of the rater positively influences perfor-

mance ratings (Landy & Farr, 1980). Therefore, work experience of the letter

writer was controlled in both studies. Given the possibility of contrast effects

that have been reported in the interview literature (Wexley, Sanders, & Yukl,

1973), in the laboratory study we controlled for the order in which applicants

were presented to the subjects. Finally, Carlson (1967) pointed out that individ-

uals’ evaluations might be influenced by the amount of evaluating experience

they have. Thus, we controlled for letter writing experience of the subjects.

AFFECTIVE DISPOSITION

211

METHOD: STUDY 1

Setting and Subjects

Subjects were undergraduatestudents at a large Midwest university enrolled

in an introductory management course. Participation in the study was volun-

tary. Students choosing to take part were awarded extra credit towards their

course grade. The sample consisted of 120 subjects of whom 52% were male

and88% wereWhite. (Listwise deletionsubsequently reduced the sample sizeto

110.) As was expected with a student sample, only 9% had previous experience

writing letters of recommendation. Average age of subjects was approximately

22 years and average total work experience was 5.2 years.

Procedure

The study consisted of 16 individual sessions. Participants each attended

one 45-min session. Each session included the showing of two brief videos

depicting two hypothetical bank employees performing a routine transaction.

The transaction involved the bank employee accepting a check deposit from a

customer as well as giving the customer a small amount of cash. Employee

quality was controlled by one bank employee being a good performer and one

being an average performer. The actor portraying the good employee welcomed

the customer to the bank, made small talk while performing the transaction,

and counted out the bills as they were given to the customer. In contrast, the

average employee simply called for the next person in line, made no attempt

to converse with the customer, and presented the cash to the customer in a

lump sum. The order in which the videos were shown was alternated in the

sessions in order to assess any impact the order might have on subject’s evalua-

tion of the bank employees. Also, to avoid confounding quality of the employees

with sex differences, both employees were male.

Prior to watching the respective videos, subjects were provided with back-

ground information on each employee (a resume and performance appraisal

that had been completed by the employee’s former supervisor). This information

was tailored to match the performance level of the employee. For example, the

good employee had two years of banking experience while the average employee

had only one year. The good employee had a slightly higher GPA and his

performance appraisal score was slightly higher than that of the average em-

ployee. For the purposes of this experiment, subjects were asked to assume

they had worked with each of the employees during the summer and were now

being asked to write a letter of recommendation for each employee. In order

to simulate the selection process, subjects were told that both employees were

applying for teller positions in other banks. Also, subjects were told that the

employees were applying at different banks so they were not competing for

the same job. No time limit was placed on writing the letters and, likewise,

no guidelines were given as to how long the letters should be.

In addition to watching the videos and writing the letters of reference, sub-

jects filled out a brief survey that was used to assess affective disposition. As

212

JUDGE AND HIGGINS

with the video, the administration of the survey was alternated. Half of the

subjects completed the survey prior to participating in the experiment and half

completed the survey after the experiment. Again, this was done to determine

if the order in which subjects completed the survey had any effect on their

evaluations. There was no significant difference in average ratings between

those subjects completing the affect survey before the experiment and those

completing the survey after the experiment.

Finally, a manipulation check was conducted to ensure that the subjects

felt the two employees were of different quality. The manipulation check was

performed by having subjects rate each employee on a scale of 1 5 unacceptable;

not recommended to 5 5 exceptionally qualified: highest recommendation. The

mean rating for the good employee was 4.47 and the mean rating for the

average employee was 3.38. This difference was highly significant (t

117

5 11.27,

p , .001), indicating that subjects did find differences between the employees

and the manipulation was successful.

Measures

Affective disposition. Affective disposition of subjects was measured with

the Neutral Objects Satisfaction Questionnaire (NOSQ), which is based on a

measure originally developed by Weitz (1952). The NOSQ assesses affective

disposition by asking respondents to rate their satisfaction with a list of 25

neutral objects common to everyday life (e.g., the way people drive, local news-

papers, movies being produced today). Affective disposition was defined as the

average response to the 25 items on the NOSQ, using a 1 5 dissatisfied,25

neutral,35 satisfied scale. Judge and Bretz (1993), among a sample of indus-

trial workers, found that the NOSQ displayed favorable psychometric proper-

ties (all items significantly loaded on a single factor) and was quite stable over

time (r 5 .75 over a 6-month period). In comparison, measures of positive

affectivity (PA) and negative affectivity (NA) were somewhat less stable (r 5

.63 in both cases). Judge and Carter (1995), among a sample of college students,

also found that all NOSQ items loaded on a single factor. They also reported

that, over a 6-week period, the NOSQ was significantly more stable (r 5 .79)

than measures of trait-PA (r 5 .59) or trait-NA (r 5 .61). When the NOSQ and

measures of PA and NA were collected at the same point in time, Judge and

Carter found moderate correlations between the NOSQ and PA (average cor-

rected r 5 .31) and NA (average corrected r 52.29). When the measures were

separated by four weeks, the correlations were slightly lower (average r 5 .28

for PA and average r 52.25 for NA). In the present study, the coefficient a

reliability estimate of the NOSQ was a 5 .76.

Favorability of letter. Favorability of letters was assessed by three raters.

Letters were rated on a scale of 1 5 extremely negative to 10 5 extremely

positive. Prior to actually evaluating the letters, raters were trained and read

a random sample of five letters before they began the rating process. Ratings

were made independently with no discussion between raters regarding the

AFFECTIVE DISPOSITION

213

assignment of ratings. Because dispositional tendencies are most clearly re-

vealed by individuals’ decisions over time or across situations (Epstein, 1977),

and to simplify presentation of the results, favorability of the letter represented

the average favorability of the two letters. In order to determine whether raters

were consistent in their evaluations of the letters, we estimated interrater

reliability using several methods. The coefficient alpha reliability estimate of

the three ratings was .82 for the good employee letters and .91 for the average

employee letters. Using the Spearman–Brown formula, we found a reliability

coefficient of .83 for the good employee and .93 for the average employee.

Other measures. Length of letter was measured by counting the average

number of words in the first three rows and multiplying this figure by the

number of rows in the letter.

Previous experience writing letters of reference

was assessed with the question “Have you written a letter of recommendation

before?” Answers were coded on a 1 5 yes,05 no scale. Total work experience

was determined by the question “How many years of total work experience do

you have?” Work experience was reported in years and months.

METHOD: STUDY 2

Setting, Subjects, and Procedure

Subjects were full-time faculty members at a large Midwest university.

Seventy-six percent of the sample was male and 90% were White. All had

previous experience writing letters of reference with the mean numberof letters

written being estimated at 110. Average age of subjects was 48.1 years and

mean total work experience was 23.3 years.

Surveys were mailed to a random sample of 1,000 full-time faculty members.

A cover letter included with the survey asked participants to fill out the brief

survey and return it along with up to three recent letters of reference they

had written. Participants were assured in the cover letter that individual

responses would be completely confidential. The response rate was 14.6% for

a total of 146 participants. Of the 146 faculty who returned at least one letter

and survey, 132 returned at least two letters, and 108 returned three letters.

Of the 108 faculty who returned at least one letter, 13 surveys contained

incomplete data. Thus, there were 95 observations containing full data (three

letters and a completed survey) that could be included in the analyses.

The low response rate may be attributed to the private and personal nature

of letters of reference. Accordingly, the results may have been biased by whether

those who returned surveys generally write good reference letters. Although

we cannot directly answer this question, a partial answer can be revealed by

In order to determine the accuracy of this method, using a random sample of letters, we

correlated our method of measuring letter length with the actual (exact) number of words in each

letter. Because the number of words measured with our method was very highly correlated with

the actual number of words (average r 5 .98 across the two studies), we considered our method

acceptable.

214

JUDGE AND HIGGINS

comparing the number of letters returned by a participant and the mean

favorability of all the subject’s letters. If letter favorability is related to the

number of letters a participant supplies, we would expect those returning three

letters to have a higher mean favorability rating on their letters than would

those who returned one or two letters. Likewise, participants providing two

letters would be expected to have a higher mean favorability rating than those

providing just one letter. However, results indicated that the average favorabil-

ity ratings did not vary according to how many letters the subjects returned.

This indicates that number of letters returned was unlikely tohave been related

to letter favorability.

Despite the low response rate, participants appear to be representative of

the sample as a whole. A random sample of non-respondents revealed that

76% were male, which exactly matches the percentage of respondents who

were male. ANOVA results comparing the representation of respondents and

non-respondents across the nine colleges in the university indicated no signifi-

cant differences. These results suggest that the sample is representative of the

full population, at least with respect to these attributes. The purpose of the

reference letters was as follows: nonacademic job 5 29%; academic job 5 27%;

graduate school 5 20%; grant, scholarship, or award 5 16%; promotion or

advancement 5 8%. ANOVA results for a test of letter favorability by purpose of

letter indicate letter purpose had no significant influence on letter favorability.

Measures

Affective disposition. Affective disposition was assessed using the same

measure as in Study 1, the NOSQ. In this study, the internal consistency of

the NOSQ was a 5 .76.

Favorability of letter. As in Study 1, favorability of letters was assessed by

three independent raters. Each letter was rated on the same 1 5 extremely

negative to 10 5 extremely positive scale that was used in Study 1. Consistent

with the previous study, favorability of the letters was assessed by computing

the average rating across the three letters. Although using all three letters

meant losing a significant portion of the sample, this did not appear to influence

the results as average ratings of favorability were similar for the three letters.

Interrater reliability was evaluated in the same way as in Study 1. The coeffi-

cient a, based on the correlations between raters, was .81 for the first letter,

.82 for the second letter,and .82 for the third letter. Using the Spearman–Brown

formula to assess interrater reliability, we found reliability coefficients of .82

for the first letter, .84 for the second letter, and .84 for the third letter.

Other measures. Length of letter was again determined by counting the

average number of words in the first three rows and multiplying this figure

by the number of rows in the letter. Previous experience writing letters of

reference was assessed with the question “Approximately how many letters of

recommendation have you written before?” Total work experience was deter-

mined by the question “How many years of total work experience do you have?”

Work experience was reported in years and months.

AFFECTIVE DISPOSITION

215

RESULTS

Table 1 presents the descriptive statistics and intercorrelations of the vari-

ables in both studies. Examination of the means and standard deviations sug-

gests possible departures from normality. Accordingly, we computed skewness

and kurtosis statistics for these variables. For both samples, experience and

number of letters written were significantly skewed and kurtotic. However,

when we normalized thesevariables by computing their natural log, the correla-

tions with other variables were not altered. Therefore, we retained the untrans-

formed measures.

Multiple regression analysis was used to test both hypotheses. In order to

provide unbiased estimates of the true relationships between the independent

and dependent variables, Hunter’s (1992) regression program was used to

correct the correlations between affective disposition and ratings of letter favor-

ability for unreliability. Coefficient alpha reliabilities were used to correct the

affective disposition measure while the Spearman–Brown reliabilities were

used to correct ratings. Once these corrections were made, standardized regres-

sion coefficients were computed. To avoid problemsinherent instatistical signif-

icance testing (Cohen, 1994; Schmidt, 1996), 95% confidence intervals are

drawn around the standardized regression coefficients. In order to determine

the degree to which length of letter mediates the relationship between affective

disposition and favorability of the reference letter (H2), two regressions were

estimated for each study. One equation included length of the letter as an

independent variable while the other equation excluded length of letter. If

there is an association between length of letter and favorability of the reference

letter, and the regression coefficient on affective disposition drops substantially

when length of letter is controlled, H2 is supported. Within each regression,

all variables were entered into each equation simultaneously.

TABLE 1

Descriptive Statistics and Intercorrelations (Uncorrected) of Study Variables

Variable M SD 123456M SD

1. Affective disposition 2.38 0.23 — .10 2.11 — .00 .17 2.44 0.21

2. Total work experience (years) 5.12 3.03 2.04 — .33 — 2.16 .03 23.31 11.66

3. Experience writing letters 0.09 0.29 2.16 2.01 — — .08 .16 109.69 206.57

4. Strong applicant presented first 0.56 0.50 .06 .03 .00 — — — — —

5. Length of letter (words) 153.31 45.18 .33 .01 2.09 2.01 — .44 323.98 174.08

6. Average favorability of letter 7.87 0.89 .29 2.09 2.02 2.22 .45 — 7.85 0.54

Note. Correlations for laboratory study (listwise N 5 110) appear below the diagonal; correlations

for field study (listwise N 5 95) appear above the diagonal. Correlations are uncorrected. For both

studies, the response scale for affective disposition is 1 5 Dissatisfied,25 Neutral, and 3 5

Satisfied. For the laboratory study, the response scale for experience writing letters is 1 5 Yes, I

have written a letter of recommendation,05 No, I have not written a letter of recommendation;

for the field study, this variable reflects the total number of letters written in the past. Strong

applicant presented first is coded 1 5 Ye s,05 No. For both studies, the response scale for average

favorability of letter is 1 5 Extremely negative to 10 5 Extremely positive.

216

JUDGE AND HIGGINS

Table 2 contains the regression estimates predicting favorability of the letter

of reference for the laboratory study. The confidence intervals for work experi-

ence and previous experience writing letters included zero. Conversely, the

confidence intervals for the variable representing the strong applicant being

presented to subjects first, length of letter (for the first equation), and affective

disposition, excluded zero. This latter set of results indicates that these vari-

ables do, in fact, influence letter favorability. Affective disposition shows a

moderate, nonzero relationship with letter favorability, but the relationship is

stronger when length of letter is excluded from the equation. In fact, the b

coefficient for affective disposition is 38% weaker when letter length is con-

trolled. Thus, results in this study support both H1 and H2. The table also

shows that the independent variables explained 36% and 20% of the variance

in letter favorability.

Table 3 contains the regression results for the field study. The results are

relatively consistent with those of the previous study. With the exception of

previous letter writing experience in the second regression, the confidence

intervals for the control variables included zero while length of letter and

affective disposition had nonzero effects on favorabilityof the letterof reference.

Unlike Study 1, however, controlling for length of letter had little effect on the

relationshipbetween affectivedisposition and letter favorability. The coefficient

for affective disposition is only 8% weaker when length of letter is controlled.

Thus, as in Study 1, H1 is supported. However, contrary to the results of Study

1, H2 is only partially supported. Length of letter is related to the judged

favorability of the letter (H2a), but length of letter does not appear to mediate

much of the relationship between affective disposition and letter favorability

TABLE 2

Regression Estimates Predicting Favorability of Letters of Reference:

Laboratory Study

Controlling for Not controlling for

length of letter length of letter

b 95% CI b 95% CI

Total work experience 2.09 (2.27, .09) 2.08 (2.27, .11)

Previous experience writing letters .06 (2.12, .24) .05 (2.15, .25)

Strong applicant presented first 2.25* (2.42, 2.08) 2.26* (2.45, 2.07)

Length of letter .41* (.22, .60) — —

Affective disposition .24* (.02, .46) .38* (.16, .60)

R .60 .45

.36 .20

N 110 110

Note. 95% CI denotes 95% confidence interval around standardized regression coefficients cor-

rected for unreliability.

*Denotes coefficients with confidence intervals excluding zero.

AFFECTIVE DISPOSITION

217

TABLE 3

Regression Estimates Predicting Favorability of Letters of Reference: Field Study

Controlling for Not controlling for

length of letter length of letter

b 95% CI b 95% CI

Total work experience .03 (2.18, .24) 2.07 (2.30, .16)

Previous experience writing letters .16 (2.05, .37) .23* (.01, .46)

Length of letter .48* (.30, .66) — —

Affective disposition .23* (.01, .46) .25* (.01, .50)

R .56 .31

.31 .10

N 95 95

Note: 95% CI denotes 95% confidence interval around standardized regression coefficients cor-

rected for unreliability.

*Denotes coefficients with confidence intervals excluding zero.

(H2b). The regressions also explain a smaller proportion of the variance in

letter favorability compared to Study 1.

DISCUSSION

Past research on reference reports has focused on their validity and on the

value of such reports as descriptors of the person being recommended (Baxter et

al., 1981; Muchinsky, 1979; Peres & Garcia, 1962). This literature has generally

found that letters of recommendation are not particularly reliable or valid. It

has even been suggested that the reference report may tell more about the

reference provider than the person being recommended (Baxter et al., 1981).

The focus of this study was on one type of reference report, the letter of refer-

ence, and one aspect of the reference provider, affective disposition. Results

from two separate studies indicated that affective disposition was related to

the favorability of letters of reference. Thus, positively oriented letter writers

do write more favorable letters.

In addition to investigating a link between affective disposition and favorabil-

ity of the letter of recommendation, we also found that length of the letter was

related to the judged favorability of the letter of recommendation. However,

letter length mediated the relationship between affective disposition and letter

favorability only in the laboratory study. Thus, positively oriented letter writers

do write more favorable letters but it is not clear whether this process occurs

The uncorrected regressions estimating the relationship between affective disposition and

letter favorability provided results similar to those reported in Tables 2–3. For Study 1, the

uncorrected standardized regression coefficients on affective disposition were b 5 .29 (not control-

ling for length of letter) and b 5 .17 (controlling for length of letter). For Study 2, the respective

uncorrected regression coefficients were b 5 .19 and b 5 .18. The confidence intervals for all of

these coefficients excluded zero. Thus, although both corrected and uncorrected regression results

produced nonzero effects, because the uncorrected coefficients are downwardly biased (by failing

to correct for measurement error), the corrected results are presented in Tables 2–3.

218

JUDGE AND HIGGINS

because positive references write longer letters. It would seem likely another

factor explaining the dispositional effect is the increased tendency of letter

writers to use more positive words, and fewer negative words, in describing

the targets. Though these results are interesting, they do not address the causal

relationships among affective disposition, letter length, and letter favorability.

Thus, we have been careful to interpret our results as correlational rather

than causal.

The effect sizes for affective disposition were not large. To some extent, this

is to be expected. Favorability of letters was evaluated in a rigorous manner

that was independent of the letter writer. Furthermore, the process of writing

letters is arguably a “strong” situation with many motives and contextual

dynamicsthat arenot easily observed. In the letter writingprocess, thestrength

of the situation may limit a person’s willingness to discuss negative aspects of

the person they are describing and push them to disclose only the positive or

less negative characteristics of the individual. The impact of situational

strength may work to diminish the observed relationship between affective

disposition and letter favorability. However, the results do indicate that af-

fective disposition is one factor that is related to letters of reference.

The results of these studies would appear to have practical implications for

both employers and applicants. For employers, the results suggest that letters

of reference cannot always be accepted as completely impartial reports about

the applicants’ worthiness for the position. Most employers probably already

know this. However, it is not only that references may have many conscious

motives in writing their letter, such as advancing the interests of the subject

due to friendship or mentorship biases, they may also have subconscious biases

that influence their reports. Thus, a scaling factor may need to be applied

when interpreting reference letters. For example, if the positivity vs. negativity

of the letter writer is known, the employer may implicitly or explicitly discount

letters by positive people and vice versa. For applicants, there is even a clearer

implication. It would appear that applicants who choose letter writers with a

positive disposition stand a better chance of having a favorable letter written

about them. Thus, there would seem to be reasons, all else equal, to find letter

writers with a positive affective temperament and avoid seeking recommenda-

tions from less positive references, particularly if the reader of the letter is

unfamiliar with the letter writer.

Both studies have limitations. The obvious concern with Study 1 is external

validity. Asking college students to write letters about hypothetical job appli-

cants raises questions about the degree to which this is representative of the

actual letter writing process. Significant efforts were undertaken to make the

letter writing process as realistic as possible. Specifically, videos and accompa-

nying materials were carefully produced to give subjects a realistic view of

the target’s job performance. Nevertheless, subjects were inexperienced at the

process of writing letters and subjects did not personally know the targets they

were evaluating.

Study 2 also has limitations. First, the low response rate, and the fact that

the applicants may have selected only positive letter writers (who would write

AFFECTIVE DISPOSITION

219

favorable letters), raises the possibility of nonresponse bias. However, the fact

that the means for affective disposition and letter favorability where very

similar in Study 2 and Study 1 (where response bias was not an issue) argues

against the hypothesis that only positive letter writers responded, or only

positive letters were returned.

A second limitation is that it was not possible

to collect information on the subjects of the letters. Therefore, it is possible

that dispositional tendencies of letter writers are related to characteristics of

the individuals who solicited these letters. To some extent, this potential con-

cern is mitigated by the fact that letter favorability was analyzed across three

letters. Analyzing multiple letters achieves several goals, including reducing

concerns that we did not take characteristics of the subject of the letters into

account. If the letters written were affected by differences among the subjects

of the letters, these differences would have to be systematic across the three

letters. Only future research could determine whether applicant quality is

related to the affective temperament of the individuals from whom letters of

reference are solicited.

A limitation common to both studies is that no cognitive or interpersonal

processes were studied. Both types of processes have been found to be relevant

in similar contexts, such as performance evaluation and interviewer decision-

making (Longenecker et al., 1992). Given that results here suggest that the

tendency to experience undifferentiated affect (affective disposition) is relevant

to the letter writing process, future research should explore the possible cogni-

tive and interpersonal processes that cause writers’ dispositional tendencies

to be expressed in their letters. Another limitation of both studies is the failure

to assess the letter writer’s liking of the subject of the letter. Furthermore,

characteristics of the recipients of the letters, such as their dispositions toward

reading letters, may be quite relevant.

Because the two studies were quite different (one was a laboratory study

with students, the other a field study with actual letter writers), one might

reasonably question the degree to which the results across the two studies

can be compared. Ideally, we would have conducted four studies so that the

independent influences of type of subject (student vs. faculty) and type of

method (laboratory vs. field) would not be confounded. As a way of investigating

the utility of such a design, we pooled the two data sets and computed an

interaction between affective disposition and study (Study 1: laboratory study

with students vs. Study 2: field study with faculty). If the results across the

two studies are incomparable in terms of the relationship between affective

disposition and letter favorability, this interaction should be significant. How-

ever, when we conducted this pooled analysis using hierarchical moderated

regression, affective disposition remained a significant predictor of letter favor-

As was noted by a reviewer on a previous version of this manuscript, if affective disposition

is separated into two dimensions, positive affectivity and negative affectivity, it is possible faculty

responses were biased by negative (as opposed to positive) affectivity. Under this argument, the

measure of affective disposition used in this study reflects positive affectivity and would not

necessarily reflect the possibility that high negative affectivity faculty were less likely to respond.

220

JUDGE AND HIGGINS

ability, but the interaction was not significant. This suggests the results be-

tween the two studies are not incompatible.

We purposely conducted these disparate studies with the idea that the princi-

pal limitations of one study are offset by the other. We sought to offset the lack

of realism in Study 1 with using actual letters of reference in Study 2. Similarly,

we sought to offset the lack of control in Study 2 by controlling the subjects of

the letters in Study 1. That similar results were obtained in both studies

increases confidence that the relationship between affective disposition and

favorability of letters of recommendation is not artifactual (i.e., due to method-

ological artifacts). Thus, affective disposition seems to play an important role

in the favorability of letters of reference.

REFERENCES

Baron, R. A. (1993). Interviewers’ moods and evaluations of job applicants: The role of applicant

qualifications. Journal of Applied Social Psychology, 23, 253–271.

Baxter, J. C., Brock, B., Hill, P. C., & Rozelle, R. M. (1981). Letters of recommendation: A question

of value. Journal of Applied Psychology, 66, 296–301.

Brief, A. P., Butcher. A., & Roberson, L. (1995). Cookies, disposition, and job attitudes: The effects

of positive mood inducing events and negative affectivity on job satisfaction in a field experiment.

Organizational Behavior and Human Decision Processes, 62, 55–62.

Browning, R.C. (1968). Validityof reference ratings from previous employers. Personnel Psychology,

21, 389–393.

Carlson, R. E. (1967). Selection interview decisions: The effect of interviewer experience, relative

quota situation, and applicant sample on interviewer decisions. Personnel Psychology, 20, 259–

280.

Cohen, J. (1994). The earth is round (p , .05). American Psychologist, 49, 997–1003.

Dipboye, R. L. (1992). Selection interviews: Process perspectives. Cincinnati, OH: South-western.

Epstein, S. (1977). Traits are alive and well. In D. Magnusson & N. S. Endler (Eds.), Personality

at the crossroads: Current issues in interactional psychology (pp. 83–98). Hillsdale, NJ: Erlbaum.

Feldman, J. M. (1981). Beyond attribution theory: Cognitive processes in performance appraisal.

Journal of Applied Psychology, 66, 127–148.

Fiske, S. T. (1982). Schema-triggered affect: Applications to social perception. In M. S. Clark &

S. T. Fiske (Eds.), Affect and cognition. Hillsdale, NJ: Erlbaum.

Hunter, J. E. (1992). REGRESS: A multiple regression program in BASICA. User’s manual, Depart-

ment of Psychology, Michigan State University.

Judge, T. A. (1992). The dispositional perspective in human resources research. Research in Person-

nel and Human Resources Management, 10, 31–72.

Judge, T. A., & Bretz, R. D. (1993). Report on an alternative measure of affective disposition.

Educational and Psychological Measurement, 53, 1095–1104.

Judge, T. A., & Carter, S. (1995). A confirmatory investigation of two alternative measures of

affective disposition. Unpublished manuscript, Cornell University.

Knouse, S. B. (1983). The letter of recommendation: Specificity and favorability of information.

Personnel Psychology, 36, 331–341.

Landy, F. J., & Farr, J. L. (1980). Performance rating. Psychological Bulletin, 87, 72–107.

Longenecker, C. O., Jaccoud, A. J., Sims, H. P., Jr., & Gioia, D. A. (1992). Quantitative and

qualitative investigations of affect in executive judgment. Applied Psychology: An International

Review, 41, 21–41.

Mehrabian, A. (1965). Communication length as an index of communicator attitude. Psychological

Reports, 17, 519–522.

AFFECTIVE DISPOSITION

221

Merritt, J. M., & Davis, D. D. (1994). The relationship between negative affectivity, positive

affectivity, and perceptions of performance in the workplace. Paper presented at the Ninth

Annual Conference of the Society for Industrial and Organizational Psychology, Nashville, TN.

Mosel,J. N., &Goheen,H. W.(1958).The validityoftheemployment recommendationquestionnaire

in personnel selection: I. The skilled trades. Personnel Psychology, 11, 481–490.

Mosel, J. N., & Goheen, H. W. (1959). The employment recommendation questionnaire: III. Validity

of different types of references. Personnel Psychology, 12, 469–477.

Muchinsky, P. M. (1979). The use of reference reports in personnel selection: A review and evalua-

tion. Journal of Occupational Psychology, 52, 287–297.

Peres, S. H., & Garcia, J. R. (1962). Validity and dimensions of descriptive adjectives used in

reference letters for engineering applicants. Personnel Psychology, 15, 279–286.

Schmidt, F. L. (1996). Statistical significance testing and cumulative knowledge in psychology:

Implications for training of researchers. Psychological Methods, 1, 115–129.

Sleight, R. B., & Bell, G. D. (1954). Desirable content of letters of recommendation. Personnel

Journal, 32, 421–422.

Smith, M., & George, D. (1992). Selection methods. International Review of Industrial and Organi-

zational Psychology, 7, 55–97.

Tsui, A. S., & Barry, B. (1986). Interpersonal affect and rating errors. Academy of Management

Journal, 29, 586–599.

Weitz, J. (1952). A neglected concept in the study of job satisfaction. Personnel Psychology, 5, 201–

205.

Wexley, K. N., Sanders, R. E., & Yukl, G. A. (1973). Training interviewers to eliminate contrast

effects in employment interviews. Journal of Applied Psychology, 57, 233–236.

Wherry, R. J., Jr., & Bartlett, C. J. (1982). The control of bias in ratings: A theory of ratings.

Personnel Psychology, 35, 521–551.

Wiens, A. N., Jackson, R. H., Manaugh, T. S., & Matarazzo, J. D. (1969). Communication length

as an index of communicator attitude: A replication. Journal of Applied Psychology, 53, 264–266.

Received: March, 12, 1998