Upcoming Events
September 1, 2004New Programs and Initiatives
September 1, 2004By George Fitzsimmons and Donald Macnab, Psychometrics Canada Ltd.
Are you considering switching from paper and pencil assessment of your clients to an on-line Internet based system? The traditional methods most of us were trained to use are rapidly being updated with new technology. Is it safe, secure, respectable? Are there precautions, processes, or policies to implement before taking the plunge? How could using on-line testing affect my quality of service and my client’s progress?
Counsellors who use tests in their work have several objectives in mind. They would like an efficient way to describe aspects of their client such as interests, work values, or personality. They would like the results of the test to facilitate the counselling process. It may bring new information, affirmation of attitudes and beliefs, possible alternatives to consider in the decision making process, and a sense of integrity from participating in this activity.
Clients appreciate the formal structure of questions and answers related to their quest. The face validity of the chosen instruments affirms the counsellor’s understanding of what information will contribute to the working relationship. Often the results, including the printed report, are highly prized because of the personal nature of the information. The meaning of the report becomes better understood with the coaching provided by the counsellor.
Many of the on-line systems remove clerical work from office staff. You do not need to keep question booklets and answer keys on hand, or set up procedures for their secure storage. Instead of hand scoring with templates or mail-in scoring, Internet based systems score the clients test immediately and provide you with instant feedback. This allows you to enhance your relationship with your client by providing timely reports and feedback.
The decision making and control of the assessment process remains with the counsellor. The activity of the client is slightly different, using a keyboard, mouse and monitor. Where to test still requires a decision. Some instruments can be completed at the client’s convenience in any setting; others need proctoring in a secure location just as they do now. “Many issues about Internet testing practices are similar to those faced by the profession in the past.” (Naglieri et al 2004 p 150.)
Just because the test is available on-line rather than from your filing cabinet does not absolve you from the ethical considerations that you have always been responsible for. We still need to satisfy ourselves that the rigour of research, test development, sampling, and meaningfulness of results are present in the product employed. (See: Standards for Educational and Psychological Testing, published by the American Educational Research Association et al in 1999).
Recent publications may help you to understand some of the issues surrounding changing your assessment practices. In the April, 2004 issue of the American Psychologist, Naglieri et al, conclude that on-line assessment has a promising future for the practitioner. They suggest that, “Internet testing should be subject to the same defensible standards. . . as paper and pencil tests. . .” for validity and reliability. A very complete listing of standards for the client, counsellor and publisher are presented in an on-line article, “Rights and Responsibilities of Test Takers: Guidelines and Expectation” at www.apa.org/science/ttrr.html. This article will help sensitize you to the many factors to consider in assessment practices.
A recent publication of the Public Service Commission of Canada “on-line tests en ligne; Phase II Implementation: Report to Stakeholders July, 2003”, details answers to many of the questions that counsellors have when considering moving to an on-line system. The report examines the user experience, satisfaction, and utility. The report concludes that candidates and test administrators are very satisfied with the experience and the utility of on-line tests. Their research found a time savings for the professional of eighteen minutes per test administered for a candidate. They also present a careful analysis, tracking input and operating costs suggesting a cost savings of “2 to 3 times lower” than using traditional paper and pencil tests.
References
“on-line Tests en ligne”: Phase II Implementation: Report to Stakeholders July 2003. Francois Chiocchio, Pascal Desganes, Bastian Druidenier, Suzanne Lalonde, Diane Thibeault.
Psychological Testing on the Internet. Jack A. Naglieri, Fritz Drasgow, Mark Schmit, Len Handler, Aurelio Prifitera, Amy Margolis, and Roberto Velasquez. American Psychologist vol. 59, No. 3, 150-162, 2004.
Rights and Responsibilities of Test Takers: Guidelines and Expectation
www.apa.org/science/ttrr.html
Standards for Educational and Psychological Testing
American Educational Research Association,1999.
Dr. George Fitzsimmons is professor emeritus from Dept. of Educational Psychology, University of Alberta; where he taught courses in career development and using tests in counselling. He is President of Psychometrics Canada Ltd.
Dr. Donald Macnab is Director of Research, at Psychometrics Canada Ltd. He has been developing web based assessment procedures for the past five years.