The six experts who had evaluated the first prototype items were engaged two years later to review the final TE items and to provide feedback on the progress that had been made. A seventh expert reviewer, an administrator from a state school for the blind, was added to the group.
Reviewers found that the adapted item types were accessible in all of the formats they evaluated. Reviewer comments largely focused on online tools and accommodations as well as the quality of the hard copy braille test forms. These comments form the basis for continued improvement in both online and external accommodations.
Reviewers commented that the audio delivery and pace were very good. Reviewers noted, however, that audio delivery could be improved by having the audio respond to the student’s choices, such as by having the selected sentence read aloud in the Select Text item type. They requested audio for the online calculator. This suggestion will be implemented by providing a talking calculator to students who need it. . Adjustable reading speed was also suggested. Concerns were voiced about the forward and back audio buttons being too small. One reviewer noted that if audio and screen magnification were used together, the audio play box could cover part of the item.
Expert reviewers noted that screen magnification worked inconsistently. For example, sometimes the text wrapped so that horizontal scrolling was not required. In other situations, text did not wrap, forcing the student to scroll both vertically and horizontally. In some items, the text but not the graphics were enlarged with screen magnification. Reviewers also noted that students with motor impairments may have difficulty navigating with the zoom feature, particularly when both vertical and horizontal scrolling are required.
Like screen magnification, switches were noted to work inconsistently across different item types. Reviewers also noted that the step scanning system can be confusing and suggested that students complete a tutorial before encountering it during a test. Some of the online tools were noted to be unavailable when step scanning was selected, with no obvious way to disable step scanning in order to access other functions. These comments referred to known issues in KITE, ATEA’s testing platform, and may not apply to other testing platforms. Enhancements to audio, screen magnification, and switches in KITE are currently in development.
Experts who reviewed braille test forms remarked that having the hard copy in braille provided complete access to the text, eliminating the need to scroll and navigate with a refreshable braille display or synthesized speech. They also suggested additional descriptions for some graphics, such as tables and bar graphs. These descriptions would be crucial for students who have been blind from birth. In one item, one of the answers to a spelling question was a word that is usually contracted in braille. Reviewers noted that providing the complete spelling, either correct or incorrect, for a contracted word would give away the answer, unless the entire passage is presented in uncontracted braille. These suggestions warrant annual review and consultation with braille readers during the development of items that will be presented in braille.
Off-screen accommodations were recommended for students using braille, including an abacus, talking calculator, and braillewriter. For students utilizing on-screen delivery, reviewers commented, a scribe may be necessary to assist in locating tools and activating features as well as to record responses. This is a reminder that developing accessible alternatives to TE items does not eliminate the need for accommodations when students use those methods for instructional access.