Insuring Psychological Science
BY ROGERS TAYLOR
I supervise a group of applied researchers in two areas of activity: one is reasonably well described by the term "human resources research" and the other by the term, "usability." In the area of human resources research, we use the methods of psychological science, measurement and experimental statistics to develop, collect, analyze and summarize empirical data in a form that can be understood and used by executives. The process of developing such information has afforded those of us who make up the Human Resources Research unit many opportunities to contribute to psychological science over the years, but that is a secondary purpose of the group; our primary purpose is to assist executives with business decision-making.
My staff and I have presented papers at professional meetings, participated in workshops, panels and symposia at annual meetings of The Society for Industrial and Organizational Psychology, have published articles in professional journals, and have contributed a chapter to a book. At State Farm, as is the case with most business organizations I have been acquainted with, professional activity is encouraged to the extent that it does not interfere with work activities seen as more directly related to the purposes of the organization.
HAVING AN IMPACT
Working as psychological scientists in an organization like State Farm presents both opportunities and challenges. Among the opportunities, working internally allows a person to have an important impact on an organization. Early in my career I was able to complete one of the first studies in the industry dealing with validity generalization. Later, I suggested to an insurance trade association that they sponsor a consortium validation study of employment tests for the entire insurance industry, to pursue the topic of validity generalization on a larger scale. This research ultimately became one of the first industry-wide studies completed in the United States.
The Human Resources Research unit spent several years helping an association of private hospitals complete a longitudinal study of treatment effectiveness for chemical dependency. More recently, the unit became heavily involved in the use of computers for administering employment tests, and has published in that area. As part of that research, members of the unit developed, validated and implemented a computer adaptive test for use in the selection of computer programmers.
On numerous occasions, research performed by the unit to evaluate the effectiveness of programs has resulted in the programs being dropped, saving the company substantial amounts of money that would otherwise have gone to those ineffective programs. Alternatively, much of our program evaluation research has suggested ways to improve already-effective programs or to guide decisions about which among competing programs should receive additional funding.
The Human Resources Research unit also is responsible for developing and evaluating the company's employee opinion survey program, an important method of employee temperature-taking used by the organization. Questionnaires assessing work related attitudes and opinions are administered to all employees approximately every three years. New norms for this survey are developed annually using sample data. Working with an outside vendor, State Farm will soon have an almost fully automated survey capability utilizing the company intranet for questionnaire administration. A final area of work activity in which members of the unit staff are engaged involves consulting with other areas of the company, such as Auditing, on matters relating to sampling, experimental statistics and mathematical modeling.
I should note parenthetically that only a fraction of the survey work performed by the Research Division is conducted by members of my unit. In addition to the attitude survey program managed by my unit, the Research Division includes a different unit devoted to qualitative research and to surveying. This group works closely with the company business units of the company to conduct focus groups and surveys both within and outside the State Farm organizations, e.g. with claimants and with policyholders. This group also hires psychological scientists, with many members holding bachelors or masters degrees in psychology.
As I noted above, the other major area for which I am responsible rubric is Usability. Usability is a sub-area of Ergonomics, is closely related to Human Factors, and deals with "the making of devices and programs convenient and practical for a user doing work" (Bennett, 1979, p. 5). Bennett suggested the term "usability" as a replacement for the previously used expression "user-friendly." The Usability area of the unit operates two labs at State Farm corporate headquarters, consists of 6 employees at the present time, one of whom is a PhD, and is actively seeking to hire another PhD-level Ergonomics, Human Factors or Cognitive Psychologist. The Usability function is looked upon increasingly by the organization as a valuable way of ensuring that products and materials work the way they were intended to work once they get in the hands of end users.
VARIETY OF PROJECTS
I mentioned previously that working as a psychological scientist within an organization like State Farm is ripe with opportunities and with challenges. The variety of the projects we have become involved with is a major opportunity. Over the course of my years with the company, I have taken on projects covering most of the topics included in the table of contents of textbooks in the area of industrial and organizational psychology. Another advantage, I believe, is being able to work on projects that are important to the organization. Yet another advantage to working in a large company is access to sample sizes that are generally quite adequate for our purposes and do not consist of undergraduate students.
Another advantage is that we rarely have to actively sell our services. We usually are sought out by managers or executives who have questions that need empirical answers. While being sought out to provide a service is an advantage when it comes to filling one's work day with meaningful work, it is a disadvantage when it comes to conducting programmatic research. Because of the wide variety of topics we research, there tends to be little time to acquire expert knowledge in any single area or research field. I have always found it personally satisfying, however, to be able to concentrate on performing research in my field without having to fit that research around other activities such as course preparation and teaching.
VEXING, YET GRATIFYING
The challenges in working in a business organization can sometimes be as vexing as the opportunities for doing meaningful work can be gratifying. Once I had to wait 10 years after completion of one project before being allowed to publish the research. The importance of sharing of information with colleagues, on which science depends, is foreign to many business people. It is more common to find executives in business organizations who treat research findings the way an inebriate treats a lamp post, more for support than enlightenment!
Research in business organizations also tends to be used for purposes of advocacy, with pressures to not find "truth," but to obtain results that strengthen one particular point of view. In such instances, research findings sometimes become clubs which one executive uses to figuratively beat another executive as they espouse different sides. Being in the middle of feuding executives is a very uncomfortable position. On more than one occasion I've been asked to "bet my job," that research conclusions were correct (which is especially disquieting when reporting results having a .05 alpha level!).
It is not unusual to develop and validate test batteries and interview protocols, implement them and, a year later, see their use halted because the job market has become tight and openings more difficult to fill. I have also come to believe at times, however erroneously, that all I needed to do to put a halt to hiring at State Farm was to initiate an in-use study of the predictive validity of one of our selection tests!
Please don't think that the above challenges and frustrations are unique to State Farm or that State Farm is worse than any other business organization in its treatment of research-based information (or researchers). In fact, State Farm, because it is a mutual company and not subject to the short-term demands and expectations of stock owners, has always impressed me as one of the saner companies a researcher could work for.
I would like to make another point by way of advice-giving, and that is to encourage individuals anticipating either a move into industry or a career as an industrial-organizational psychologist to develop strengths in the areas of psychological measurement, experimental design, quasi-experimental design, and multivariate statistical analysis. Not only will such skills be invaluable as you are presented with a wide array of topics to research; they will serve to differentiate you from people trained in other disciplines against whom you may be competing, either for a job or once actually on the job. Business schools are offering business and industry individuals trained in organizational behavior as alternatives to persons trained as psychological scientists.
In my opinion, psychology departments offer better training in the design of experiments and in methods of data analysis than is available to most business school students. I would urge you to take full advantage of the depth and breadth of training in research methods you can receive through psychology departments.
PREOCCUPATION WITH QUALITY
My next comment may come across as drum beating and I apologize if it does, but my group and I have endeavored where possible to do work of sufficiently high quality that we would not be embarrassed to have it reviewed by a jury of our peers. We do this for two reasons. First, because it has made it much easier to get up in the morning and to come to the office over the years knowing that we are attempting to do good work. Ideally, I would like our work to be replicable, to have ties where possible to theory, to involve reliable measures of constructs, and to possess internal validity. I also urge the use of statistical analyses that have been thought through in a advance as contrasted with having been applied because there is a computer program that will run them. "Quick and dirty" research can be done by almost anyone. The value we add to the research process as psychological scientists is to provide the organization with research of the highest quality possible.
A second reason for my preoccupation with quality is that we have in fact found our work challenged by others after the fact, when no challenge was anticipated before the research was completed. This tends to occur when our findings were not those desired by managers or executives espousing a different viewpoint in an organizational disagreement.
As my time as a graduate student at Purdue came to a close, I shopped around for employment and found opportunities readily available. I decided to accept State Farm's offer because I sensed a chance to get in on the ground floor of an applied research endeavor which had the potential both for career growth and for having an impact on a work organization. I have now been with State Farm almost 33 years and the opportunities for professional growth and for having an impact have exceeded even my optimistic expectations. I have also been able to have a family and, with my wife, to raise two fine sons and to live comfortably on my income. While no work career spanning more than 30 years is entirely a bed of roses, I feel that my choice of vocation and of employer were good ones and were ones that I would recommend to any scientific psychologist having similar interests and ambitions.
(A list of selected papers and publications of staff members from the State Farm Human Resources Research unit is available from the author at rogers.taylor.av02@StateFarm.com.)
Bennett J.L. (1979). The Commercial Impact of Usability in Interactive Systems. In:
American Psychological Society
Vol. 14, No. 1