- What is Sonification
- Sonification on Sonoplanet
- Aims Sonoplanet Sonification method
- Other educational sonification and multisensory projects
What is Sonification
‘Sonification’ is a technology to represent information and data, including images, in non-speech sound. Although Sonification is not a new technology (with the Geiger Counter in 1908 as a first example), world-wide right now sonification is increasingly being applied in science (particularly in astronomy), technology and the arts. There’s growing awareness about the opportunities Sonification offers for accessibility, inclusiveness, and multisensory learning.
Recorded presentation Phia Damsma for 2021 Conference of South Pacific Educators in Vision Impairment: 'Discovering sonification: where to begin'.
Recorded presentation Phia Damsma for 2021 Conference of the Round Table on Information Access for People with Print Disabilities: 'Discovering Sonification – Experience'.
Recorded presentation Phia Damsma for 2021 VISCON Conference, NextSense Institute: 'Sonification in Education - From Outer Space to Spatial Orientation'.
Renowned Universities and Research Centres in Australia and abroad (including NASA), use sonification for the analysis of data and for educational outreach. Due to its inherent accessibility for people who are blind or vision impaired – data is represented in sound - the ability to use this mainstream sonification technology opens opportunities for students who are blind or vision impaired to have equal and independent access to science research and scientific data. This is important, because with its heavy reliance on visualization (graphs, charts, diagrams), students who are blind or vision impaired have traditionally experienced big barriers to access STEM subjects, or to pursue a career in scientific research. If sonification becomes widely available, they may no longer have to depend on others to provide lesson materials or scientific data in an alternate format. Sonification can also be applied for inclusive and multi-sensory projects within the Arts and Music curriculum.
Sonification on Sonoplanet
There currently are no international standards for the way things are sonified. The basic sonification algorithm that Sonokids applies in the app has been tested with the delegates of the 2021 SPEVI Conference and the 2021 Round Table Conference. It proved to be an effective algorithm, allowing delegates, including those who are blind or vision impaired, to comprehend and correctly identify the sonification of shapes after only a very short introduction.
How does it work? In short: a double tone indicates a point on a line. The tempo of the double tone goes up to the right on the line. The further to the left on a line, the lower the tempo of the double tone.
The higher a data point is located, the higher the pitch of the tone. If in the sonification of a line the pitch of the tone stays the same, it is a horizontal line. A ‘ding’ sound in a shape such as a rectangle indicates that a ‘turn’ is imminent.
Aims Sonoplanet Sonification method
The Sonokids sonification method that is used on Sonoplanet does not require the use of a headset (although it will enhance the audio experience), and it is very straightforward. The aim is to test the power and/or limitations of sonification. Purposely, there is only limited visual support in the app, although the Settings allow for customisation.
Students can build generic sonification skills through gamified learning on Sonoplanet. These skills can then be transferred to the use of other sonification tools. This approach is similar to Sonokids’ successful series of ‘Ballyland Code’ apps, which introduce basic, general coding concepts and computational thinking skills that prepare students for the use of any future programming language.
With more than 20 years of experience, Sonokids knows that the earlier students start to (playfully) work with technology and explore its concepts, the more beneficial this will be for their future use of this technology. Sonokids created the Ballyland CosmoBally on Sonoplanet app as a tool for early learning of essential sonification concepts and skills, coined ‘emergent sonification literacy’. Early learning of sonification concepts and basic skills can significantly contribute to students’ future interest in and access to STEM.
Other educational sonification and multisensory STEM projects
Audio Universe: Tour of the Solar System is a planetarium show, but unlike traditional shows the soundtrack takes the lead role. All of the objects in space are represented with rich sounds as well as with incredible visuals. The audience can listen to the stars appear and hear the planets orbit around their heads. Audio Universe: Tour of the Solar System is an immersive experience that can be enjoyed irrespective of level of vision.
NASA’s Universe of Sound – Listen to Space. Listen to sonifications of parts of our Milky Way galaxy and of the greater Universe beyond it, through data captured by NASA's Chandra X-ray Observatory, Hubble Space Telescope, and Spitzer Space Telescope.
“Planetary Bands, Warming World”. Sonification of global warming through a musical composition for string quartet. Created by Daniel Crawford and Scott St. George, graduate fellow and professor at the University of Minnesota, Minneapolis, USA.
Oceanographic symphonic experience. This sonification project was created by mathematically translating hidden layers of “ocean color” reflectance data from NASA’s Earth-observing Aqua-MODIS satellite into musical notes.
Accessible Oceans: Exploring Ocean Data Through Sound. The Accessible Oceans project is using sonification to convey meaningful aspects of ocean science data to increase access to ocean data and ocean literacy. This research is expected to contribute new strategies and resources for representing quantitative information that can be applied across STEM disciplines that use similar data displays in informal learning environments. By Amy Bower, Jon Bellona, Jessica Roberts, Leslie Smith.
Sensemath – Royal Dutch Visio. Free app for iOS, in the Appstore.
Tactile Universe: Tactile models and lesson plans for accessible astronomy education.
Listen to the sounds on Mars (April 2022). Not sonification, but read and listen how sound samples collected by NASA Perseverance Mars Rover lead to new scientific insights about the atmosphere of the Red Planet.
Sonification Report 2019 from The DIAGRAM Center (a Benetech initiative supported by the U.S. Department of Education, Office of Special Education Programs).