Web-Based Personality Assessment

Project Description and Rationale

John A. Johnson

Pennsylvania State University, DuBois

Note: This is an extended, written version of a poster presented at the 71st Annual Meeting of the Eastern Psychological Association, March 24, 2000, Baltimore, MD. Making this essay available on the world wide web does not constitute publication; the author reserves the right to submit a version of this essay as an original manuscript for publication in a print journal.
 

On September 1, 1998, I uploaded to the World-Wide Web (http://cac.psu.edu/~j5j/test/ipipneo1.htm) a 300-item personality inventory constructed by Lewis R. Goldberg (1999) of the Oregon Research Institute. I also uploaded a prototype computer program I designed to (a) score the inventory, (b) provide a narrative feedback report to the respondent, and (c) email the respondent's scores to me. At last count, on August 6, 1999, 4,472 persons had completed the inventory. This report explains the rationale behind setting up a personality inventory on the World-Wide Web, describes the methods used to enable web-based assessment and feedback, presents preliminary findings from the assessment project, and outlines future plans for web-based personality research. Because part of this research was supported by a Research Development Grant (RDG) from Penn State, the final section of this report indicates how RDG funds were used in this project.

Background and Rationale for Web-Based Personality Assessment

In a review of personality assessment via computers (Johnson, 1994), I outlined the advantages of using a computer to collect, score, and interpret responses to personality items. Before the advent of computer administration and scoring, psychologists typically tallied pencil marks on answer sheets and transferred these tallies to profile sheets for visual inspection and interpretation. Blurry marks and incomplete or multiple responses had to be dealt with on an ad hoc basis. Computer administration increases the speed and accuracy of personality data collection. Blurry marks do not exist, multiple responses are disallowed, and incomplete responses can be disallowed or assigned a missing value to be handled by the scoring program. Counting, summing, and norming errors are eliminated.

The clerical advantages of computer administration and scoring represented a liberating breakthrough for research psychologists like myself who work at a small campus with limited human resources. Nonetheless, even at the time I wrote the paper on using computers for data collection (Johnson, 1994a), I was still keyboarding personality item responses into computer data files by hand because our campus did not have sufficient computer technology for automated data input. My limited resources also meant limited samples—in terms of both number and heterogeneity. However, by the latter 1990s, the advent of the World-Wide Web provided a way to overcome these limitations. Personality item responses on the Web can be tallied and scored automatically by a CGI program, saving the researcher enormous amounts of time. Also, if a narrative report writing program is included, the respondent can receive instantaneous feedback for completing the on line inventory. Furthermore, with a Web-based personality inventory the researcher can collect data literally from all over the world rather than from persons who are in physical proximity (Smith & Leigh, 1997).

Despite the liberating potential of the Web, various concerns must be addressed before this potential can be realized. I have described these concerns in a paper on problems associated with using commercial personality inventories for research (Johnson, 1993). One concern is the secrecy surrounding the scoring keys for some of these inventories. This secrecy disallows researchers from conducting even the most fundamental psychometric analyses such as reliability estimation and factor analysis. Furthermore, secrecy surrounding the computer programs used to build narrative reports from commercial personality inventories precludes critical discussion of the merits and drawbacks of different techniques for generating narrative reports. The expense of commercial personality inventories places additional burdens on researchers with limited resources, and copyright restrictions make it virtually impossible to create Web-based versions of these inventories. All of these issues (and additional problems with commercial inventories noted by Goldberg, in press) impede technical progress in personality assessment.

To work around the problems associated with commercial personality tests, Goldberg (1999) has developed, in collaboration with researchers from the Rijksuniversiteit Groningen (The Netherlands) and Universität Bielfeld (Germany) a set of 1,252 items dubbed the International Personality Item Pool (IPIP). By administering the IPIP with a variety of commercial personality inventories to an adult community sample (in stages to prevent fatigue), Goldberg's research team has been able to identify, empirically, sets of IPIP items that measure the same constructs as the commercial inventories. Scales formed from these item sets possess psychometric properties that match or exceed those of the original commercial scales (Goldberg, in press). These scales are in the public domain on the World-Wide Web at http://ipip.ori.org/ipip/ . My contacts with the Bielefeld research group during my 1990-91 sabbatical year (Johnson & Ostendorf, 1993) and the Groningen research group (Johnson, 1994b) led me to Goldberg's IPIP research program at the Oregon Research Institute.

Methods Used to Enable Web-Based Assessment and Feedback

I chose from among the various personality inventories at Goldberg's IPIP Web site his 300-item proxy for the revised NEO Personality Inventory (NEO PI-R; Costa & McCrae, 1992), which I call the IPIP-NEO. I chose to work with the IPIP-NEO because the NEO PI-R is one of the most widely used and well-validated commercial inventories in the world (Johnson, in press). Furthermore, the NEO PI-R is based on today's most significant paradigm for personality research, the five-factor model (FFM; Wiggins, 1996). During my sabbatical year in Bielefeld I had created computer algorithms in FORTRAN for producing FFM-based narrative reports (Johnson, 1993). Between 1995 and 1998 I taught myself the hyper-text markup language (HTML) and practical extraction and report language (PERL) needed to transport these algorithms to the World-Wide Web.

Responses to the IPIP-NEO items are scored on a five-point scale. The HTML method for presenting items and scoring responses employs the relatively simple TABLE and FORM formats. After some introductory instructions, IPIP-NEO items are presented in a column on the left side of a table. Five small circles called radio buttons appear next to each item in the right-hand column; these radio buttons are labeled very inaccurate, moderately inaccurate, neither accurate nor inaccurate, moderately accurate, and very accurate. Numerical values of 1, 2, 3, 4, and 5 are associated with these respective responses (or 5, 4, 3, 2, and 1 for items scored in reverse direction) by the HTML underlying the Web page. Respondents do not see these numbers, although persons familiar with HTML and features of their Web browsers can select "view source" to see the software code underlying the Web page. Respondents are constrained to clicking only one of the five radio boxes for each item. When the respondent reaches the end of the Web page, clicking the "submit button" sends the numerical responses to a common gateway interface (CGI) script written in PERL for further processing.

I pilot tested the first version of the IPIP-NEO with students in my psychology courses. The pilot test indicated that presenting all 300 items on one web page overtaxed many computers with relatively slow processors, limited memory, or slow Internet connections. Respondents reported that their computers "froze up" after they completed only part of the inventory. I corrected this problem by presenting only 60 items on the first Web page and storing the HTML for the remaining items within a series of CGI script programs. Clicking the submit button at the end of 60 items temporarily stored the item responses and triggered the next CGI script to generate, on-the-fly, a Web page presenting the next 60 items. Clicking the submit button after the final item sent all responses to the final CGI script that (a) summed the responses into scale scores, (b)  transformed the scores into T-scores with a mean of 50 and standard deviation of 10, (c) created a Web page containing a narrative report explaining the meanings of the scores, and (d) automatically sent me an email containing these scores.

Other papers (Johnson, 1993, 1994a) describe my program's basic algorithms for producing narrative reports from personality scores, although these earlier programs created text files for printing rather than Web pages. Essentially, the program produces a Web page with a detailed description of each personality dimension from the FFM (Neuroticism, Extraversion, Openness to Experience, Agreeableness, and Conscientiousness). Included within these descriptions is what research has revealed about persons with relatively high or low scores on each of the five factors. At the end of each detailed description of the major factors a paragraph with the following form appears: "Your score on [name of factor] is [high, average, low], indicating that [brief summary of what research has revealed about persons with the score]."

Underneath the detailed descriptions of the five major factors are shorter descriptions of the six facets of dimension. For example, short descriptions of Anxiety, Anger, Depression, Self-Consciousness, Immoderation, and Vulnerability appear under the description of Neuroticism. After each facet description is a sentence of the form: "Your level of [name of facet] is [high, average, low]."

High scores were defined by T-scores greater than 55 (i.e., greater than .5 standard deviation above the mean) and low scores by T-scores less than 45. Preliminary norms for generating the T-scores were based on means of the adults in Goldberg's (1999) community sample, adjusted for the slight differences between this sample's scores on the NEO PI-R and national norms for the NEO-PI-R reported in the professional manual (Costa & McCrae, 1992). Separate norms were used for each sex, graded by age. Norms for persons under 21 were estimated by adjusting the adult norms according to differences between the two age groups reported by Costa & McCrae (1992) for the NEO PI-R. These estimated norms were considered preliminary until sufficient data were collected to establish genuine norms from Internet participation.

Development of the CGI PERL scripts continued throughout my 1999 sabbatical leave. I added to the introductory web page a section where respondents could indicate if they were completing the IPIP-NEO for course extra credit or if they were visiting from outside the university. I included a special checkbox for visitors who linked to the inventory from a set of Web pages on personality that I helped construct for the Annenberg/Corporation for Public Broadcasting Project (http://www.learner.org/exhibits/personality/ ). I added "traps" in the first CGI script to send respondents who failed to indicate their age, sex, or visitor status back to the first page to complete that information. By August 6, 1999, I taught myself enough additional PERL to have the last CGI script save item responses directly to a computer file rather than email the scores. The results reported below included only data collected through August 6th. On August 18-22 I traveled to the Oregon Research Institute to share my findings with Lewis Goldberg and discuss plans for future research.

Results

Information in the headers of emails containing IPIP-NEO scores was used to eliminate duplicate emails from individuals who apparently clicked the final submit button more than once. Scores were also screened to insure that they fell within one point of the minimum score for each scale. This procedure retained scores from individuals who skipped a few items on the IPIP-NEO, but eliminated scores from individuals who left many answers blank. (Blank items are scored 0 by default, whereas the lowest score of a completed item is 1.) After eliminating duplicate and incomplete protocols, 4,472 valid protocols remained. Of these protocols, 639 were identified as links from the Annenberg/CPB site and the remaining from unspecified other sites. Data from Penn State students were not included in these analyses. The sample included 2,026 males and 2,446 females. Reported ages ranged from 11 through 90; the average age for males was 34.1 years (SD=12.3) and for females, 31.8 years (SD=12.0).

The opening of new Web sites can be advertised by submitting the address and nature of the site to a number of companies whose business is to help people find information on the Web. The on line IPIP-NEO was not advertised in this or any other manner except for a single link on the Annenberg/CPB personality page. On the day the Annenberg site opened (September 1, 1998) one person completed the IPIP-NEO; 32 completed it the following day. The number of emails containing non-duplicate, complete protocols averaged 14.6 per day from September 1, 1998 through August 6, 1999. Figure 1 contains a graph showing the frequency of responses over the past year. For unknown reasons, over 200 persons responded on two consecutive days in November, 1998, and around 100 persons responded on each of three days at the beginning of June, 1999. Aside from these peaks, response rate remained steady over the period of data collection.

Figure 1




The means and standard deviations on all IPIP-NEO scales for males and females both under age 21 and age 21 and over are presented in Table 1. The adult means are compared to the adult means for Goldberg's (1999) Eugene-Springfield community sample via t-tests. Alpha was set at .001 to yield an effective alpha of .03 (two-tailed) across the 35 t-tests. The Internet sample scored significantly higher in Neuroticism and Openness and lower in Agreeableness and Conscientiousness. The higher score on Neuroticism is particularly noteworthy, given that the Eugene-Springfield sample itself scored about .1 SD higher than national norms on the commercial NEO PI-R Neuroticism scale. This finding is in accord with a previous suggestion (Kraut, et al., 1998) that Internet use may be linked to depression. The lower score on Conscientiousness is somewhat offset by the fact that the Eugene-Springfield sample scored about .06 SD (males) to .1 SD (females) above national norms for NEO PI-R Agreeableness. However, the Eugene-Springfield sample scored .2 SD (females) to .26 SD (males) below national norms for NEO PI-R Extraversion, so the "no significant differences" between this reference group and the Internet group masks the possibility that the Internet sample is below the national average in Extraversion.

On the commercial NEO PI-R, college-age individuals score notably higher than adults on the Neuroticism and Extraversion scales, somewhat higher on Openness to Experience, and markedly lower on Agreeableness and Conscientiousness. The same differences are found on the IPIP-NEO domain scales—albeit not as large as the differences on the NEO PI-R scales—between the older and younger Internet samples except for Openness to Experience, on which the younger and older participants barely differ. All in all, the means and standard deviations for the Internet sample look reasonable comparable to the existing normative data from the Springfield-Eugene sample.

Because data collected through August 6, 1999 consisted only of scale scores, item-based analyses such as scale homogeneity could not be conducted on the present data set. The format of the data does allow a factor analyses of the facet-level scales to see whether they load on the appropriate factor. Therefore, a principal components factor analysis of the matrix of facet intercorrelations was conducted. Eigenvalues for the first seven factors were 7.1, 4.3, 3.6, 2.4, 2.1, 1.0, and .92. The program was instructed to extract five factors, which accounted for 64.9% of the variance. The varimax-rotated solution is presented in Table 2. The five factors are clearly defined by the five sets of facet scales. In 27 of 30 cases the facet scale showed its primary loading on the factor defined by the other facets in its domain.

Table 1: Descriptive Statistics and T-Tests for Eugene-Springfield Community Sample and Internet Sample on IPIP-NEO Scales
 
Males
Females
Adult Adult
IPIP-NEO Scale Community Int > 20 Int < 21 Community Int > 20 Int < 21 Male Female
N=211
N=1766
N=260
N=289
N=2026
N=482
t(1975) t(2313)
M SD M SD M SD M SD M SD M SD
Neuroticism
144.0
30.6
161.8
38.7
164.2
33.4
150.0
31.8
172.5
36.8
180.0
37.5
-6.43*
-9.88*
Extraversion
191.4
25.8
192.8
31.9
197.6
34.2
192.0
27.0
196.2
31.7
203.9
35.0
-0.61 
-2.14 
Openness to Experience
202.8
27.6
218.4
26.6
217.8
28.1
208.8
28.8
226.3
26.8
228.7
26.8
-7.99*
-10.30*
Agreeableness
217.8
22.8
202.8
27.3
197.1
28.0
234.0
19.2
217.3
25.3
208.6
29.1
7.67*
10.75*
Conscientiousness
232.8
22.2
215.3
30.8
204.2
29.1
234.0
24.6
220.7
29.8
203.5
31.0
8.02*
7.22*
N1 Anxiety
24.7
6.5
27.6
8.0
27.9
7.7
26.9
6.8
30.7
7.8
31.3
7.9
-5.07*
-7.88*
N2 Anger
24.8
7.7
27.0
9.3
26.8
8.3
23.8
7.1
28.8
8.8
29.8
9.1
-3.38*
-9.23*
N3 Depression
20.8
7.3
25.6
9.8
26.2
9.2
21.9
7.9
26.5
9.4
27.9
9.6
-6.92*
-7.89*
N4 Self-Consciousness
25.6
6.0
27.8
7.6
29.4
7.4
26.9
7.0
29.1
7.5
30.6
7.7
-3.98*
-4.72*
N5 Immoderation
27.5
6.3
31.1
7.3
29.9
6.4
27.6
6.7
31.1
7.3
29.9
6.4
-6.88*
-7.72*
N6 Vulnerability
20.8
5.5
22.6
7.4
24.1
6.3
23.1
6.7
25.3
7.5
27.6
7.6
-3.42*
-4.64*
E1 Friendliness
36.0
7.1
33.1
8.1
33.9
8.0
37.9
7.1
35.1
8.1
34.8
8.5
4.94*
5.56*
E2 Gregariousness
28.8
6.8
27.3
8.1
29.8
8.9
29.9
6.9
28.5
8.5
31.7
9.2
2.65 
2.70 
E3 Assertiveness
33.2
6.7
34.6
7.5
34.0
7.4
32.1
6.6
34.2
7.4
34.0
8.1
-2.61 
-4.52*
E4 Activity-level
30.3
5.4
30.6
6.0
29.6
5.6
31.2
5.4
32.0
5.9
30.5
5.6
-0.68 
-2.21 
E5 Excitement-Seeking
27.3
6.6
30.8
7.6
33.9
8.1
24.2
6.1
28.6
7.8
34.0
8.2
-6.41*
-9.18*
E6 Cheerfulness
36.0
5.6
36.4
7.0
36.5
7.1
36.9
6.8
37.8
6.9
38.9
7.0
-0.78 
-2.09 
O1 Imagination
34.9
6.5
39.1
6.6
40.6
6.8
33.0
7.2
38.5
7.4
41.5
6.9
-8.83*
-11.87*
O2 Artistic Interests
38.2
6.5
38.3
6.8
37.3
7.0
42.3
6.3
42.3
5.7
42.5
5.7
-0.27 
-0.05 
O3 Emotionality
33.8
6.4
35.4
6.7
36.1
6.8
37.3
6.0
39.4
6.2
39.4
6.5
-3.34*
-5.47*
O4 Adventurousness
33.5
5.5
35.8
6.6
35.7
6.4
34.1
5.7
35.9
6.7
36.3
6.3
-4.82*
-4.42*
O5 Intellect
36.8
6.9
40.9
6.7
39.6
7.1
35.8
7.8
39.8
7.0
39.0
7.1
-8.41*
-8.95*
O6 Liberalism
25.6
8.4
28.8
8.1
28.5
8.0
26.6
8.4
30.3
7.5
30.0
7.2
-5.35*
-7.77*
A1 Trust
36.7
6.0
33.6
7.6
32.5
7.6
37.8
6.1
34.5
7.4
33.9
7.4
5.77*
7.19*
A2 Morality
39.2
5.3
36.2
6.4
34.7
6.7
42.8
4.6
39.6
5.7
37.4
6.1
6.54*
9.22*
A3 Altruism
39.2
5.2
37.6
6.4
37.5
6.6
42.2
4.6
40.5
5.9
39.5
6.5
3.48*
4.65*
A4 Cooperation
38.2
5.5
33.7
6.9
31.2
6.9
40.7
5.1
35.5
6.8
32.5
7.6
9.14*
12.55*
A5 Modesty
30.7
6.1
28.5
6.9
28.1
7.1
33.2
6.3
31.0
6.7
29.9
7.0
4.39*
5.36*
A6 Sympathy
33.5
5.9
33.2
6.8
33.2
6.5
37.6
4.6
36.3
6.3
35.4
6.7
0.55 
3.47*
C1 Self-efficacy
40.0
4.6
39.7
5.7
38.3
5.6
39.5
5.1
39.9
5.6
37.6
5.8
0.78 
-1.11 
C2 Orderliness
37.2
6.4
33.0
7.8
30.5
7.7
37.7
7.1
34.1
8.5
30.6
8.3
7.62*
6.90*
C3 Dutifulness
43.3
4.3
39.7
5.8
38.1
5.9
45.1
3.9
41.5
5.4
38.9
6.0
8.67*
10.99*
C4 Achievement-striving
39.6
5.1
38.2
6.7
36.2
6.7
39.7
5.3
39.4
6.2
37.2
7.0
2.84 
0.87 
C5 Self-discipline
36.1
6.6
31.3
8.4
29.2
8.0
36.2
7.3
32.6
8.3
29.0
8.0
8.08*
6.92*
C6 Cautiousness
36.7
5.3
33.4
7.3
31.9
7.2
36.0
5.9
33.3
7.3
30.2
7.7
6.29*
6.05*

*p < .03, two-tailed, adjusted for the 35 repetitions of the t-test

The first exception was E4 (Activity-level) with only a .26 loading on Extraversion but a .68 loading on Conscientiousness. In the original NEO PI-R, E4 shows a considerable secondary loading (.42 vs. a .54 primary loading—Costa & McCrae, 1992) on Conscientiousness. Thus, both E4s are actually located between the Extraversion and Conscientiousness dimensions. The second exception was O3 (Emotionality), with a .49 loading on Openness to Experience and a .53 loading on Neuroticism. This indicates that the experimental IPIP O3 facet scale is oriented too much toward negative emotionality. (O3 in the original NEO PI-R has a primary loading of .50 but secondary loadings of .37 on Neuroticism and .41 on Extraversion.) Finally, C6 (Cautiousness) loaded .45 on Conscientiousness and -.46 on Extraversion. (C6 in the original NEO PI-R shows only a -.28 secondary loading on Extraversion). One might tentatively conclude from this analysis that the facets are generally loading where we might expect them, but several IPIP-NEO facet scales could use further work to align them better with their intended domains.

Future Directions

A Research Development Grant from Penn State supported my travelling to the Oregon Research Institute between August 18-22 to discuss the findings presented in this paper with Goldberg's research team and to plan future research with Goldberg's IPIP. Much of these discussions concerned the development of improved programming and automated procedures for screening data. It appears that in some cases, individuals are not getting feedback instantly (as they should) after clicking the final submit button, which leads them to click and send their data several times. Although duplicate entries are easily detected after the fact, much time could be saved if the mechanism for storing data could prevent multiple submissions. Also, traps will be developed for persons who are leaving too many items blank or using the same response option too often (e.g., mostly answering with 3s). I also may incorporate reliability checks by having the scoring routine correlate scores with an additional set of responses to content-relevant synonyms and antonyms. Or, I may have the scoring program conduct intra-person reliability checks (Jackson, 1976).

We also discussed the logistical advantages and disadvantages of maintaining a collaboratory (a computer-supported system that allows scientists to work with each other, facilities, and data bases without regard to geographical location—Finholt & Olson, 1997) on computers at Penn State versus at the Oregon Research Institute. I inquired about relocating my CGI scripts and data archive to ORI because I have limited Web server space at Penn State and no technical support dedicated to my project. Goldberg thought that the physical location of the computer facilities is less important that the adequacy of the facilities and underlying technical support. To enable us to create a proper collaboratory, he suggested that we write a joint grant to allow us to establish computer facilities dedicated entirely to on-line personality testing and to hire a staff person with

Table 2: Factor Loadings of IPIP-NEO Facets
 
FACTOR 1 FACTOR 2 FACTOR 3 FACTOR 4 FACTOR 5
N1 Anxiety
.87
-.22
-.05
.02
.00
N2 Anger
.75
-.08
.09
-.36
-.03
N3 Depression
.73
-.30
-.31
-.08
.07
N4 Self-Consciousness
.55
-.47
-.32
.22
-.09
N5 Immoderation
.50
.23
-.33
-.23
.10
N6 Vulnerability
.83
-.11
-.31
.06
-.10
E1 Friendliness
-.19
.81
.21
.24
.00
E2 Gregariousness
-.05
.83
.08
.02
-.06
E3 Assertiveness
-.17
.55
.48
-.33
.21
E4 Activity-level
-.03
.26
.68
-.17
.03
E5 Excitement-Seeking
.02
.63
-.12
-.37
.14
E6 Cheerfulness
-.23
.69
.06
.15
.20
O1 Imagination
.16
.07
-.15
-.08
.71
O2 Artistic Interests
.10
.27
.10
.31
.58
O3 Emotionality
.53
.28
.15
.23
.49
O4 Adventurousness
-.30
.39
.07
-.08
.54
O5 Intellect
-.22
-.05
.21
-.08
.78
O6 Liberalism
.01
-.07
-.23
.06
.60
A1 Trust
-.30
.44
-.03
.49
.08
A2 Morality
-.02
-.11
.20
.78
-.03
A3 Altruism
-.01
.50
.19
.67
.18
A4 Cooperation
-.23
-.03
-.09
.79
-.01
A5 Modesty
.25
-.32
-.14
.50
-.26
A6 Sympathy
.15
.21
-.06
.67
.37
C1 Self-efficacy
-.47
.14
.66
.06
.26
C2 Orderliness
.13
-.10
.62
.14
-.36
C3 Dutifulness
-.13
-.08
.54
.60
-.09
C4 Achievement-striving
-.17
.13
.79
.06
.13
C5 Self-discipline
-.23
.04
.79
.13
-.18
C6 Cautiousness
-.30
-.46
.45
.38
-.05

 

specialized programming skills and knowledge of database management. The grant will be written as a follow up on an NIMH grant awarded to Goldberg (5R37MH49227-06) to study the structure of personality traits.

Establishing dedicated computer equipment with staff support would certainly facilitate the type of data sharing and personality scale development represented by the collaboratory concept. Nonetheless, I am moving forward with some next steps in this project even as we are planning our joint grant proposal. Now that my revised CGI scripts are collecting complete item responses I will soon be able to compute inter-item correlations on scales, calculate alpha reliability estimates for scales, and conduct factor analyses at the item level. These analyses may suggest revisions to the existing experimental 300-item IPIP-NEO.

I am also planning comparisons between IPIP responses collected on line and other information about the respondents, including demographic background, scores on other psychological measures, and significant life events (Buchanan, Goldberg, & Johnson, 1999). One of the reasons the IPIP was established was to create public-domain versions of commercial instruments so that research could be conducted unimpeded by prohibitive costs and copyright restrictions. The procedure for developing these public domain proxies involves comparing scores on the commercial inventories to responses on to the IPIP. To this end, I have targeted two inventories for which I think IPIP proxies would be useful: Kolb's (1999) Learning Style Inventory (LSI) and Riso's (1999) Riso-Hudson Enneagram Type Indicator (RHETI). These expensive inventories were purchased with Penn State RDG funds. Some data on a previous version of the LSI were collected from Psychology 002 student last fall; this fall I will collect responses to the current version of the LSI as well as the RHETI and a set of items from the IPIP.

Another future project I discussed with Goldberg is expanding options for answering different sets of IPIP items and enlisting knowledgeable acquaintances to rate the personalities of the IPIP respondents. We talked about developing a menu system that would allow participants to choose among sets of IPIP items varying in size (depending on how much time they wanted to spend on responding to items). We also talked about ways for respondents to engage acquaintances in rating procedures. Enlisting acquaintances in the research is vital for assessing and improving the validity of the narrative reports based on responses to the IPIP. Research aimed at expanding response options, gathering acquaintance ratings, and assessing the validity of narrative reports may require help from programming specialists.

I anticipate that much of my future research will involve further collaboration already begun with Lewis R. Goldberg at ORI and Tom Buchanan of the University of Sunderland, United Kingdom. I am also currently engaged in a collaborative international analysis of part of the IPIP with the Groningen research group in The Netherlands. I hope one day to return to Groningen and also Bielefeld, Germany to continue my work on on-line personality that began in these cities during my 1990-91 sabbatical.

References

Buchanan, T., Goldberg, L. R., & Johnson, J. A. (1999, November). WWW personality assessment: Evaluation of an on-line five factor inventory. Paper to be presented at the meeting of the Society for Computers in Psychology, Los Angeles, CA.

Buchanan, T., & Smith, J. L. (in press). Using the Internet for psychological research: Personality testing on the World-Wide Web. British Journal of Psychology.

Costa, P. T., Jr., & McCrae, R. R (1992). Revised NEO Personality Inventory (NEO PI-Rtm) and NEO Five-Factor Inventory (NEO-FFI): Professional manual. Odessa, FL: Psychological Assessment Resources.

Goldberg, L. R. (in press). The comparative validity of adult personality inventories: Applications of a consumer-testing framework. To appear in S. R. Briggs, J. M Cheek, & E. M. Donahue (Eds.), Handbook of adult personality inventories. New York: Plenum.

Goldberg, L. R. (1999). A broad-bandwidth, public-domain, personality inventory measuring the lower-level facets of several five-factor models. In I. Mervielde, I. J. Deary, F. De Fruyt, & F. Ostendorf (Eds.), Personality Psychology in Europe (Vol. 7, pp. 7-28). Tilburg, The Netherlands: Tilburg University Press, pp. 7-28.

Jackson, D. N. (1976). The appraisal of personal reliability. Paper presented at the meetings of the Society of Multivariate Experimental Psychology, University Park, PA.

Johnson, J. A. (1993). Generating computer narrative reports for the CPI, HPI, and other omnibus personality inventories with the AB5C model. Unpublished paper available from the author at Penn State DuBois, DuBois, PA 15801

Johnson, J. A. (1994a). Computer narrative interpretations of individual profiles. Chapter originally prepared for R. Hogan, J. Johnson, and S. Briggs (Eds.), Handbook of personality psychology, but withdrawn due to space limitations. Available from the author at Penn State DuBois, DuBois, PA 15801.

Johnson, J. A. (1994b). Multimethod replication of the AB5C model of personality traits. In B. de Raad, W. K. B. Hofstee, & G. L. van Heck (Eds.), Personality psychology in Europe. Volume 5: Selected papers from the sixth European conference on personality held in Groningen, The Netherlands, June 1992. Tilburg, The Netherlands: Tilburg University Press.

Johnson, J. A. (in press). Predicting observers' ratings of the Big Five from the CPI, HPI and NEO-PI-R: A comparative validity study. European Journal of Personality, 13.

Johnson, J. A., & Ostendorf, F. (1993). Clarification of the Five Factor Model with the Abridged Big Five-Dimensional Circumplex. Journal of Personality and Social Psychology, 65, 563-576.

Finholt, T. A., & Olson, G. M. (1997). From laboratories to collaboratories: A new organizational form for scientific collaboration. Psychological Science, 8, 28-36.

Kolb, D. A. (1999). Learning Style Inventory. Boston: Hay/McBer Training Resource Group.

Kraut, R., Lundmark, V., Patterson, M., Kiesler, S., Mukopadhyay, T., & Scherlis, W. (1998). Internet paradox: A social technology that reduces social involvement and psychological well-being? American Psychologist, 53, 1017-1031.

Riso, D. R. (1999). The Riso-Hudson Enneagram Type Indicator (Version 2.5). New York: The Enneagram Institute.

Smith, M. A., & Leigh, B. (1997). Virtual subjects: Using the Internet as an alternative source of subjects and research environment. Behavior Research Methods, Instruments, & Computers, 29, 496-505.

Wiggins, J. S. (Ed.) The five-factor model of personality: Theoretical perspectives. New York: Guilford Press.