Lane, D.M. Development of sonification design theorgy: Metaphors, mappings, holistic sound design, and data-specific sonification. National Science Foundation, $402,781 (1999-2002).

A major element in the practice of science and engineering, and in the education and training of researchers in those fields, is the analysis and interpretation of data. Although most data exploration tools are exclusively visual in nature, data presentation and exploration systems could benefit greatly from the addition of sonification capacities. Auditory representation of data, or sonification, promises gains in the representation of temporal and high-dimensional data, in data monitoring tasks where the eyes are busy, in tasks requiring recognition of patterns in a data set, in high-stress or critical conditions where cross-modal correlations would be of value, and where the display users are students or researchers with visual disabilities. However, very little work has been done to determine the best ways to map data dimensions, such as temperature or pressure, onto auditory display dimensions, such as pitch or tempo. Further, little work has been done on the question of the metaphorical associations that sounds create, and how to utilize these connotations to design effective, compelling, and pleasant sonifications which are easy to learn and intuitive to use.

The proposed research seeks to discover the optimal data-to-display mappings for use in scientific sonification and investigate whether these optimal mappings vary within and/or across fields of application. To accomplish this, we propose to develop a cross-platform research environment that will incorporate sophisticated sound synthesis and data collection capabilities. The sound synthesis engine will allow for the precise control of many auditory dimensions, so that we can explore a range of promising new ways to map data onto sound.

Experiment 1 will ask undergraduate participants to compare pairs of sounds and indicate which pair member is better for representing various data types. For example, a participant would hear two sounds that differ only in pitch, and be asked which best represents something hotter. This is a very direct approach to determining population stereotypes and listener preferences in data-to-sound mappings. Over 14,000 data points will allow us to discern response patterns that indicate population stereotypes for this group of listeners.

Experiment 2 will extend Experiment 1 to listeners who are researchers in the fields of Chemistry, Physics, Biology, Economics, and Psychology, to examine how mappings differ across various scientific areas. This will be critical for designing effective sonifications that take into account the specific needs of scientists in different fields.

Experiment 3 will measure performance (reaction time and accuracy) on a number of tasks that use data sonification, in order to validate the population-specific mapping preferences that emerge from Experiments 1 and 2, It is critical to determine how the stated preferences of listeners compares to their actual performance using those preferences. Sonification design guidelines should be based on the results of task performance, as well as on the population stereotypes that listeners report directly. Experiment 3A will involve a statistical graph interpretation task, representative of tasks currently performed in laboratories and classrooms using visual graphs. Experiment 3B will involve a data-mining and exploration task, requiring listeners to detect patterns in multivariate data sets representative of those that prove unyielding to visual analysis.

In order to validate the results and conclusions derived from Experiments 1-3, and to continue to explore the practical issues associated with sonification, we will develop a cross-platform sonification software tool for use by researchers and educators. The sonification application will include two novel and important features — a content-analysis wizard and enhanced auditory design — to create optimal displays depending on the characteristics of the user, the domain of study, and the type of data to be explored. The tools will be suitable for widespread deployment and integration into existing scientific and education settings. Tool use will be monitored and analyzed, through usability testing and usage questionnaires, to inform subsequent refinements in both the theory and tool design.

Dissemination of the theory, design guidelines, and our research paradigm will bootstrap further research efforts in this field. Use of the sonification tools will answer a call from the auditory display researcher community for better theory-based tools, provide researchers with a new and powerful data-analysis tool, and provide visually-disabled scientists and students immediate inroads into the world of science that is still largely dominated by visuocentric techniques and technologies.