Evaluating accessibility practice

These notes are part of a series for the book.

Seale, J. (2014) ‘Ch. 6, Evaluating accessibility practice’ in E-Learning and Disability in Higher Education: Accessibility research and practice, 2nd ed., New York, Routledge.

Outline

Introduction
1. Evaluating artifacts of practice
2. Evaluating learning resources
3. Evaluating web accessibility
4. Evaluating accessibility policies
Evaluating service provision
1. A service auditing tool
Conclusion

Notes

There are several levels at which to evaluate accessibility. You can evaluate learning resources, websites, accessibility policies, and service provision.

Learning resources

Different studies have approached the evaluation of learning resources differently. This chapter discusses a dozen or more studies that focused on evaluating the accessibility of resources as varied as Google Docs, e-book platforms, and LMSs. The studies approached their evaluations in different manners, such as:

Evaluating the technical and functional aspects
Testing users on how they perceived the “ease of use” of a resource as a way to identify ‘perceptual, usability, and technological barriers’ (Seale, 2014, p. 94)
Conducting surveys of small groups of users to understand the depth of their knowledge of the accessibility features of a resource

Websites

Most studies that evaluate web accessibility in the higher education arena focus on three areas of a schools’ website: the home page, the main library page, and a sample course page.
Sometimes tools are used to provide an even-handed accessibility report; one such tool is WAVE. Other studies use a combination of these tools and human testing, to add subjective, qualitative assessment.

Policies

One interesting study that evaluated accessibility policies was a large-scale study conducted in 2010 of 50 American universities (one in each state). They needed to evaluate the policies against a standard, and so they started with WebAim’s seven criteria for accessibility policies and then added to that six more it. Here’s what they found:

This criteria...	From...	Was met by...
Reasons for the accessibility policy	Bradbard	Most of the schools
Standards and guidelines	WebAim	Most of the schools
To whom the policy applies	WebAim	Less than 50% of the schools
Definitions	Bradbard	Less than 50% of the schools
Lists of responsibilities	Bradbard	More than 50% of the schools, but not in detail
Training resources	WebAim	Less than 50% of the schools
Tips and examples	Bradbard	Most of the schools
Validation tools	Bradbard	More than 50% of the schools, but not in detail
Timeframe for implementation	WebAim	Less than 50% of the schools
How accessibility efforts are approved	WebAim	Less than 50% of the schools
How the policy will be enforced	WebAim	Less than 50% of the schools
What happens when the standards are not met	WebAim	Less than 50% of the schools
Contact information	Bradbard	More than 50% of the schools, but not in detail

(This is my summary of Bradbard et al. (2010), as told by Seale (2014), pp. 109-110)

Service provision

There are several different levels for evaluation:

Institutional: Our laws are focused on changing practice at this level.
Service: This is the department, unit, school, or faculty level, and there is a gap here.
- I thought this was an interesting comment: ‘In thinking about how useful it is to audit or evaluate accessible e-learning service delivery, it is possible to see that services would be resistant to having to undergo yet another quality assurance exercise on top of all the others that they are required to undertake by their institutions and by external funding agencies. Therefore any audit tool that is developed will probably need to be developed by and for service providers, probably with a view to using it as an informal self-development tool’ (Seale, 2014, p. 118).
Individual: Many guidelines and standards are focused on this level.

Katherine Hinchey