Text-based Tactile Graphics Generation for the Visually Impaired

1Carnegie Mellon University 2Seoul National University 3University of Illinois Urbana-Champaign

*Equal contribution.

TL;DR: We develop Text2TactileGraphics, the first generative system to bring AI content creation from screens to fingertips, turning text prompts into 3D-printable touchable graphics for blind and low-vision accessibility.

Abstract

Tactile graphics are a primary medium for blind and low-vision (BLV) individuals to access non-textual information. However, they are difficult to scale or personalize. While recent generative models have revolutionized visual content creation, they are optimized for screen-based visual realism and fail to satisfy the haptic perceptual and physical fabrication constraints required for touch. We present the first integrated generative system that produces fabrication-ready 2.5D tactile graphics directly from natural language prompts, jointly encoding global base geometry, fine-grained tactile surface textures, and standard-compliant braille within a unified 3D-printable representation. Our approach introduces fabrication-aware techniques, including template-guided relief generation, a fast diffusion-based text-to-texture module for high-resolution tileable normal maps, and strict base flattening to ensure tactile readability and printability, while supporting both automatic generation and interactive texture control. Extensive evaluations, together with in-person user studies with BLV participants and blindfolded sighted participants using physically 3D-printed outputs, show that participants consistently prefer our results over baselines. By extending generative graphics beyond screens to touchable reliefs, our work broadens access to generative AI for the BLV community and beyond.

Method

Out method generates tactile graphics via a multistep pipeline:

1

Base Geometry Generation

We first synthesize a base image constrained by a plate template, and lift the image to 2.5D base relief via monocular geometry estimation.

2

Tactile Texture Generation

A tactile texture module produces tileable normal maps from text prompts or tactile sensor captures, which can be assigned to object parts through text or user interaction.

3

Integration & Polishing

The textures are integrated as geometric deformation on the base relief, followed by fabrication-aware base flattening and braille generation.

4

Physical Fabrication

The resulting watertight mesh is fabricated via SLA resin 3D printing.

Click a step above to jump the video to that stage.

Acknowledgment

We thank Amin Mirzaee, Maxwell Jones, and Nupur Kumari for their helpful comments and discussion. We are also grateful to Sheng-Yu Wang for proofreading the draft. We appreciate the generous help from the Library of Accessible Media for Pennsylvanians (LAMP) and VisAbility Pittsburgh for recruiting blind and low-vision users for our studies. The project is partially supported by the Amazon Faculty Research Award, Cisco Research, Sloan Foundation, and the Packard Foundation. Ruihan Gao is supported by the A*STAR National Science Scholarship (Ph.D.). Joonghyuk Shin and Jaesik Park were supported by the IITP grant (No. RS-2021-II211343: Artificial Intelligence Graduate School Program at Seoul National University, 10%) and NRF grant (No. RS-2024-00405857, 90%) funded by the Korea government (MSIT).

BibTex

[Insert citation here]