Assessing Assessments: The Inequality of Electronic Testing

Introduction

Computer and Internet based tests are used for a variety of purposes. From entering education or employment, to improving basic learning, people everywhere are taking electronically formatted tests. Examples of both high-stakes and low-stakes tests include:

• Exit examinations for grade-level or graduation
• SAT's, ACT's or GRE's for college entrance
• National or International board examinations across disciplines
• Microsoft, Novell, IBM, and other technical certifications
• Management certifications used throughout the business world
• Professional development and hiring examinations
• Ongoing examinations used in public schools and higher education for continuous student improvement and curricular-outcome testing

With the advancement of testing from traditional paper-based tests to technologically advanced electronic tests, people reap the benefits of easier access to tests, faster response times, and greater reliability and validity of tests. However, persons with disabilities are being left out of the picture and out of many typically-administered tests.

Problem

Over 2.7 million individuals rely on web-based and computer-based tests each year (ETS - external link). Current estimates indicate that 8.5% of the U.S. population or over 24 million Americans, over the age of 5, have at least one disability that limits their use of the computer and Internet (e.g., vision, hearing, or dexterity skills) Disability Status: U.S. Census - external link (pdf document). Given current population estimates in the U.S. of over 293 million people (U.S. Census - external link), and the current estimates of 143 million Americans using the Internet (NTIA - external link) there are millions of U.S. citizens that could be disadvantaged by inaccessible online testing. This does not even begin to take into consideration the millions of individuals around the world that are disadvantaged by inaccessible testing practices. Indications are that online testing will only increase with time. The issue is internationally significant, not only because of the numbers, but also because electronic testing permeates all age groups and fields. Moreover, it has important implications for the self-determined and independent lives of individuals around the world.

For example, in the U.S., the No Child Left Behind Act requires educational assessment of students as part of a school's obligation to report on annual yearly progress. All school-aged students must participate including those with disabilities. Currently, many can only participate in tests given special testing accommodations, as they would be unable to directly access and complete these tests independently. This is often not because of the skills of the students, rather because the tests are not designed with accessibility in mind.

To examine the extent to which individuals with disabilities could access online tests, WebAIM evaluated a broad sample of online tests. Results indicated that not one test of the ten sampled could be accessed by many individuals with disabilities (Pilot Data). Moreover, not a single test was designed to implement the current electronic accessibility standard that applies to U.S. federal entities (i.e., Section 508 of the Rehab Act) or the international accessibility guidelines of the World Wide Web Consortium (i.e., the Web Content Accessibility Guidelines -WCAG 1.0).

The inability to access online tests can negatively impact an individual's ability to gain education, employment, and career advancement. Currently, an individual with disabilities can only access online testing by requesting an "accommodation." The three accommodations most often available to "eligible" individuals with disabilities are to have (1) an individual "read" the test to them, (2) an individual "write" the responses for them, or (3) extra time. For some with disabilities these accommodations are viewed as demeaning, especially in light of their independence with the use of technology (e.g., screen readers, refreshable Braille displays). It is also important to note that while some testing environments offer these types of accommodations for users with disabilities, others do not.

Testing companies limit accommodation possibilities to guarantee the reliability, validity, and standardization of their tests. When asked why current test environments did not maximize the possibility of natively accessible practices in their design, the responses of a sample of staff from testing companies centered on issues of test standardization. Many of the respondents did not demonstrate basic knowledge of accessibility principles.

Though WebAIM is less familiar with the full range of testing issues (e.g., high-stakes secure testing), we are certain that fewer accommodations would be necessary if tests were natively accessible. This would increase confidence of standardization principles, and more individuals with disabilities could avail themselves of direct testing opportunities without accommodation.

Scenario

The following scenario, though fictitious, illustrates the real-life needs and frustrations of people with disabilities with regard to online testing. The scenario shows how inaccessible tests can have a negative impact on the lives of people with disabilities in terms of career opportunities as well as general feelings of self-worth. Creating accessible testing environments for individuals with disabilities can have a positive impact by helping to satisfy the needs and concerns raised by this scenario.

Scenario: Testing Environments Impact the Lives of Americans

Solution

WebAIM's strategy to dissipate the accessibility barrier of electronic testing is to capitalize on existing resources, develop new partnerships with those in the testing industry, and help develop content germane to those who create online tests. One important piece of WebAIM's work will be to develop an accessible testing model that will showcase accessibility practice and display the limits of accessibility in test environments.

Conclusion

WebAIM's approach is just one of many possible ways to address the inaccessibility of electronic testing. However, it is important to keep in mind that something needs to be done soon to begin electronic testing reform. As our society moves to digital and networked solutions within education, business, and government, we must remember those individuals with disabilities and allow their strengths to be utilized. Technology can become and important ally to individuals with disabilities rather than their greatest foe.