SuperSonic - Symphonies Beyond Earth
Our project is an ambitious educational initiative that aims to introduce sonification as a novel concept and strives to establish the interconnectedness between human beings and infinite space. We have developed a system of sonification that translates galaxial 3D and 2D data into a plethora of sounds. Then, we applied them to a web application that presented them in a visually captivating manner. Simulating space travel from the earth to outer space for our users, we start from Earth and as the ‘travelers’ scroll down, they will be able to ‘visit’ the space images of their choices to experience the symphonies the space data can produce by the process of sonification.Data Sonification Methodology
In Python, loop through each column of pixels within the image, then depending on whether the image is RGB/RGBA, or grayscale, we calculate the average R, G, B values of the column:
1. If the mode of the image is RGB or RGBA, we use PIL’s getpixel function to assign r, g, b to the corresponding RGB value
2. If the mode of the image is grayscale, we assign to r, g, b the same grayscale value.
Calculating the average of r, g, and b, then turning those averages into pitch
Using the equation pitch = (red + blue + green)/10, we derive a pitch from the average RGB value of each column.
The reason why we used this equation is that, through various experiments, the resultant pitch would produce the best, most soothing sound to the ears, while still maintaining the effect of the change in RGB on the audible sound.
With the pitch produced, we derive certain notes using: note = base_frequency * (semitone_ratio ** pitch) (Where: Semitone_ratio = 2 ** (1 / 12), Sase_frequency = 130.81 (C3 frequency in Hz)
Using the notes, we plug it into a few sine waves to complicate the waveforms in order to produce a more intriguing pattern of sound. From this, we produce an audio using Pydub.
Performing the same sweeping method from left to right with the animation using moviepy.editor, and syncing the sound up with the sweeping motion through Pydub.
Finally, add the existing audio into the video using AudioSegment of Pydub and export the product.