Future experiences
This page contains recorded examples of audio and touch experiences. These are completely manually generated mockups to demonstrate what we hope to be able to accomplish in the future, as machine learning algorithms improve and more underlying data is available along with graphics on the web.
Example 1: Solar System
Category: Educational diagrams
Diagrams in textbooks and educational websites are often essential to the learning experience. In this example, a map of the solar system, with the planets orbiting the sun, conveys the ordering and relative sizes of the planets. Thermal haptic feedback can demonstrate the temperatures of the various planets in a much more direct way than simply reading about them, providing an experience richer than that delivered to sighted users simply looking at the graphic.
Haptic Rendering
Audio Rendering 1
Objects are presented from left to right, loudness is mapped to object size.
Audio Rendering 2
Objects are presented in order of size, loudness is also mapped to object size.
Example 2: Jewelry Image
Category: Product photographs
Shopping sites rely on high-quality pictures to give customers the best idea possible of the products at hand. Although the text descriptions alongside items for sale often give a description of the product, for items like jewelry, they are unlikely to describe important aspects like the shapes and relative positions of all the set stones. In this example, a friend sent a direct link to just the picture itself, and all the surrounding context is lost. By recognizing color and shape, a much more accurate description can be provided either by audio, or by switching between various aspects on a haptic display, such as the positions of the blue, clear, or pink stones in this example.
Haptic Rendering
Audio Rendering 1
Elements are presented from left to right.
Audio Rendering 2
Groups of elements are presented, with individual elements given as pings.
Example 3: Landscape Painting
Category: Art
Visual artworks such as paintings can be particularly difficult to capture using sound and touch. We are investigating methods of allowing artists and creators to produce their own custom audio-haptic experiences.
Audio Rendering 1
Presentation of objects from left to right with confidence mapped to loudness.