Accelerating math accessibility with the use of AI

A year ago, NWEA, now part of HMH, shared their innovative approach to make math more accessible for students. The aim was to identify the biggest challenges and gaps in mathematics for students who use screen readers and refreshable braille devices, because classroom materials are not always adapted to formats such as braille or large print, and materials are not always suitable for a screen-reader navigation, voice input, or a combination of these designs. NWEA developed prototypes that enabled screen readers to interact with equations in a more intuitive way, based on a method called process driven math (PDM). 

NWEA continued to innovate and build on their previous research to create different ways of presenting complex math, especially for math taught in grades six to nine. They also worked on different ways of outputting math that included screen-reader functionality and refreshable braille devices in both UEB (Unified English Braille) and Nemeth formats. Moreover, they developed a prototype for a voice-activated chatbot.  

To account for the accessibility of math equations, they used two markup languages, HTML and ARIA, to split equations into parts or regions. Each region, as well as the whole equation, had a hidden label that a screen reader would say to users as they explored the equation or expression. When students moved from one region to another, they would hear a word that described the kind of math in that region (for example, “term” or “constant”). Students could then decide to go into the region and hear the exact math, or they could just skip to the next region.
 

The use of generative AI  

By using AI, specifically GPT-4, the team was able to improve both the quality of the math as well as the time required to convert the equations to HTML, and to use code generation to write the code for the first prototype. The model only needed a few examples to learn how to change the initial test set of equations from MathML to the HTML structure that was the most accessible. From there, the model required context to ensure that responses were formatted in the best way for the app.  

Demo of using the equations with a screen reader: