“Customization is Key”: Reconfigurable Textual Tokens for Accessible Data Visualizations
This pre-recorded conference talk introduces how reconfigurable text-to-speech description tokens enable screen reader users to customise their reading experience during data exploration. .
Transcript
Hi, I’m Shuli Jones and today I’m going to show you the work I’ve done with the MIT Visualization Group on the customization of accessible visualizations.
Customization is crucial for digital accessibility. Blind and low-vision people, just like sighted people, are a large, heterogeneous group with a wide variety of needs when it comes to digital media. Different blind and low-vision users have different levels of experience with assistive technologies — they might combine their use of a screen reader with a Braille display, think of data in tables, or interact with it using tactile graphics. These differences lead to different preferences in accessible tooling depending on the user and the context they’re working in. A finance enthusiast might focus closely on stock prices but skim the weather section. An amateur meteorologist might do the opposite. And the same meteorologist might want different things from their screen reader depending on whether they’re coding a climate model or checking song lyrics. Good accessibility tooling should support all of these scenarios.
The current state-of-the-art approach to accessible visualizations is navigable hierarchies. These present text in a tree shape: higher levels provide broader overviews, while lower levels offer granular detail. This lets users start with a general overview and then decide how deeply to focus. However, users can’t change what information is present in the text or how it’s conveyed.
To demonstrate, I’ll use our open-source accessible visualization toolkit, Olli. Olli works with a wide variety of visualizations. Here’s an example with stock prices of five tech companies over a decade. Olli takes the dataset as input and outputs a navigable hierarchy description. Using the keyboard, I can move down or across the levels, and my screen reader reads the text. As I go deeper, I get more details about smaller areas of the graph — for example, focusing on Google’s stock prices, exploring the x-axis, and even reaching a table of individual data points. But sometimes I just want a quick overview of the rise and fall, not all the details. Other times I want even more information than the hierarchy provides. These conflicting desires — for less and for more — are addressed through customization.
Our work makes this customization possible. We identified four design goals for how blind and low-vision people should be able to customize accessible descriptions. We created a model for customization that meets these goals, and we implemented it as an extension to Olli. The extension includes three main features: a settings menu, custom settings, and a command box.
Previously, there was no way to shorten lengthy descriptions. Now, I can open the settings menu and set different lengths, or “verbosity,” for each level of the hierarchy. A novice user might shorten everything overall by setting verbosity to low or medium. An experienced user might create a custom setting for a particular level. In the custom menu, you can control the verbosity of each token and reorder them. For example, if I care about how sections compare to one another, I can prioritize the quantile token, make it shorter, and place it before the dates, while leaving averages in. I can save this as a “comparison” setting and reuse it later. Now, I can zip through the quartiles and quickly sense how prices changed over time. If I also want associated numbers, I can use the command box. For instance, I can navigate to a low quartile and issue the “aggregate” command to hear aggregate statistics. This gives me quick insight without having to create a new setting. That’s customizable Olli — it lets users adjust descriptions both for overall preferences and for immediate tasks.
Users can create custom settings, reuse them across tasks, and adjust descriptions to match their level of experience. Less experienced users can hear longer descriptions, while more advanced users can opt for concise versions. These improvements were guided by four design goals: presence, verbosity, ordering, and duration. Presence is about what information is included. Verbosity is about how concisely it’s expressed. Ordering is about the sequence of information, which matters especially with screen readers since skimming isn’t possible. Duration is about how long customizations last — they should persist across tasks when needed, but be easy to switch on or off.
To identify these goals, we built on work by Zong, Lee, Lundgard, et al. in Rich Screen Reader Experiences for Accessible Data Visualization, which introduced design dimensions of structure, navigation, and description. We re-coded transcripts from their user study to focus on descriptions, then developed our goals iteratively. To support customization, we broke descriptions into tokens. Tokens can be toggled (presence), reordered (ordering), set to concise or verbose (verbosity), and assigned a duration. Tokens are defined by two parameters: affordance (what task they support — wayfinding or consuming) and direction (what they describe relative to position — parent, self, or children). Together, these parameters form a grid of token possibilities that users can customize.
Throughout development, we practiced co-design with our blind coauthor, Daniel Hajas. We met biweekly, sharing models and prototypes. Daniel contributed both as a researcher and as a user, testing the system and reflecting on usability. Once finalized, we evaluated the system with 13 screen reader users through 90-minute Zoom interviews. We discussed their use of accessible visualizations and customization, let them explore Olli and the extension, and asked them to complete tasks. Coding the transcripts revealed three main themes.
First, customization supports autonomy and agency, but opportunities to customize are often missing. Designers sometimes make assumptions about what details users want, leading to serious consequences such as difficulties at work or disengagement with data altogether. Second, the four design goals we identified were each important to participants. Presence and verbosity were especially critical — concise text was generally preferred, though some needed more detail at the start of an analysis or when tackling unfamiliar topics. Third, customization is context-dependent. Less experienced users took longer to get used to Olli and relied more on defaults, while experienced users picked it up quickly and wanted to customize it more extensively. Interest in customization was also tied to personal relevance of the data.
Looking ahead, we see two directions for future work. One is developing a third affordance for tokens, interpreting, which would situate consumed data in a broader context — for example, noting that a 2008 stock price dip was tied to the U.S. recession. While hard to generate automatically, advances in large language models may make this feasible. The other direction is multisensory representation. Our work focused on text, but screen reader users often combine text with sonification, Braille, or tactile graphics, each with unique strengths. Extending customization to multisensory outputs could unlock richer, more flexible accessible visualizations.
Thanks for listening!