visualizing sound in processing
This was the final applet in motion. Using the minim library for processing, each waveform is generated in realtime as the two sounds play over eachother creating a pretty chaotic sound, but there are some instances of overlapping patterns where the mashup works pretty well. In the third version of the code, the boolean of the two waveforms is generated, producing a new way to visualize the waveforms. View the youtube video here, but I really need to figure out a way to add sound to the video, silence doesn’t do it justice. Charlie Parker, Iggy Pop and Richard Wagner comparison + code:
import processing.dxf.*;
import ddf.minim.analysis.*;
import ddf.minim.*;
FFT fftLog1;
FFT fftLog2;
Waveform myRects;
Minim minim;
AudioPlayer groove1;
AudioPlayer groove2;
boolean record;
PFont font;
PFont fontoutline;
void setup(){
size(1200,600,P3D);
noStroke();
minim = new Minim(this);
groove1 = minim.loadFile(“groove_iggy.mp3”);
groove2 = minim.loadFile(“groove_wagner.mp3”);
groove1.loop();//repeat each song
groove2.loop();
font = loadFont(“HelveticaNeueLT-Bold-18.vlw”);
fontoutline = loadFont(“HelveticaNeueLT-Bold-18.vlw”);
fftLog1 = new FFT(groove1.bufferSize(),groove1.sampleRate()); //create the FFT logarithmic scale
fftLog2 = new FFT(groove2.bufferSize(),groove2.sampleRate());
fftLog1.logAverages(22,4); //adjust numbers to adjust spacing
fftLog2.logAverages(22,4);
float w1 = float ((fftLog1.avgSize()+fftLog2.avgSize())/2);
float x = w1;
float y = 0;
float z = 50;
myRects = new Waveform(x,y,z);
}
void draw(){
background(15);
directionalLight(126,126,126,sin(radians(frameCount)),cos(radians(frameCount)),1);
ambientLight(152,152,152);
for(int i = 0; i < fftLog1.avgSize(); i++){
int w = int(width/fftLog1.avgSize());
float zoom = 1;
float jitter = (max(fftLog1.getAvg(i)*200,fftLog2.getAvg(i)*200 )); //jitter in camera influenced by waveform
PVector foc = new PVector((myRects.x*.5+jitter*.5), myRects.y+jitter, 0);
PVector cam = new PVector(zoom, zoom, -zoom);
if (frameCount < 260){
camera(foc.x+cam.x,foc.y+(cam.y-1500*(cos(radians(-frameCount+60)))),foc.z+cam.z-400,
foc.x,foc.y,foc.z-100,
0,0,1);
//println(-1500*(cos(radians(-frameCount+60))));
}
else {
camera(foc.x+cam.x,foc.y+(cam.y+1418.278),foc.z+cam.z-400,foc.x,foc.y,foc.z-100,0,0,1);
}
}
fftLog1.forward(groove1.mix); //play each song
fftLog2.forward(groove2.mix);
myRects.update1(); //update each waveform+boolean
myRects.update2();
myRects.update3();
myRects.textdraw1(); //draw z height for song waveforms
myRects.textdraw2();
if(record){
beginRaw(DXF, "output.dxf");
}
// DXF will export the stuff drawn between here.
myRects.plotBoolean(); //create surfaces
myRects.plotTrace1();
myRects.plotTrace2();
if(record){
endRaw();
record = false;
println("Done DXF~!");
}
}
void stop() {
groove1.close(); // always close Minim audio classes when you finish with them
groove2.close();
minim.stop(); // always stop Minim before exiting
super.stop();
}
class Waveform{
float x,y,z;
PVector[] pts1 = new PVector[fftLog1.avgSize()];
PVector[] pts2 = new PVector[fftLog2.avgSize()];
PVector[] pts3 = new PVector[fftLog1.avgSize()]; //needed for boolean waveform
PVector[] trace1 = new PVector[0];
PVector[] trace2 = new PVector[0];
PVector[] trace3 = new PVector[0]; //needed for boolean waveform
Waveform(float incomingX, float incomingY, float incomingZ){
x = incomingX;
y = incomingY;
z = incomingZ;
}
void update1(){ //plot boolean waveform
plotB();
}
void plotB(){
for(int i = 0; i < fftLog1.avgSize(); i++){
int w = int(width/fftLog1.avgSize());
x = i*w-1050; //adjust the x position of the waveform here
y = frameCount*5;
z = height/4-fftLog1.getAvg(i)*10;
stroke(0);
point(x, y, z);
pts1[i] = new PVector(x, y, z);
//increase size of array trace by length+1
trace1 = (PVector[]) expand(trace1, trace1.length+1);
//always get the next to last
trace1[trace1.length-1] = new PVector(pts1[i].x, pts1[i].y, pts1[i].z);
}
}
void plotBoolean(){
stroke(255,80);
int inc = (fftLog1.avgSize()+fftLog2.avgSize())/2;
for(int i=1; i<(trace1.length+trace2.length)/2-inc; i++){
if(i%inc != 0){
beginShape(TRIANGLE_STRIP);
float value = (trace1[i].z*100);
float m = map(value, -500, 20000, 0, 255);
fill(m*2, 125, -m*2, 140);
int threshold = 15;
if (trace1[i].z
textFont(fontoutline, 24);
fill(155);
text(“wagner”,200,500,0);
text(“iggy”,900,500,0);
text(“max(iggy-wagner)”,500,500,0);
}
}
}
void plotTrace1(){
stroke(255,80);
int inc = fftLog1.avgSize();
for(int i=1; i
October 24th, 2010 at 1:59 am
I am interested in studying your code. However a bunch of your for loops are incomplete in what is posted.
for(int i=1; i
also, one of the vertex statements trails off:
void plotBoolean(){
[…]
vertex(trace1[i].x
Would you kindly repost your code?
Thanks much!
May 8th, 2011 at 9:30 am
Hi, I just emailed you about this source code which I could not find previously. Though I am having errors as per Andrew, though I am too much of a beginner to fix them. Is there any chance that you might share the code?
Thank you
Rebecca
May 8th, 2011 at 2:26 pm
This post has a link to the the code live in action: http://gracefulspoon.com/blog/2009/03/06/iggy-wave/
May 9th, 2011 at 11:45 am
Thank you so much! It is beautiful.
December 10th, 2016 at 5:46 pm
Hello every body i used this sample to improve it significally. I have far better performance, better code in terms of best practices and severals bugs are fixed.
you can find my version here (4 classes) :
package minimsoundapp;
import ddf.minim.analysis.*;
import ddf.minim.*;
import java.awt.Color;
import processing.core.*;
import processing.core.PApplet;
import static processing.core.PApplet.map;
import static processing.core.PConstants.TRIANGLE_STRIP;
public class ThreeDimensionSpectrum extends PApplet {
FFT fftLog;
Minim minim;
AudioPlayer audioplayer;
final int TOTAL_TRACE_LENGTH=300;
final int X_AXIS_SCALE=40;
final int Y_AXIS_SCALE=8;
final int Z_AXIS_SCALE=7;
float x,y,z;
PVector[] tempMatrix;
PVector[] fullMatrix;
@Override
public void settings() {
//size(1280, 720, P3D);
size(480, 360, P3D);
//size(500, 300, P3D);
}
@Override
public void setup(){
noStroke();
minim = new Minim(this);
audioplayer = minim.loadFile(RunSpectrum.activeFile, 1024);
audioplayer.play();
background(255);
fftLog = new FFT(audioplayer.bufferSize(),audioplayer.sampleRate());
fftLog.logAverages(22,6); //adjust numbers to adjust spacing
tempMatrix = new PVector[fftLog.avgSize()];
fullMatrix = new PVector[TOTAL_TRACE_LENGTH*fftLog.avgSize()];
//longueur * largeur
//TOTAL_TRACE_LENGTH : longueur
//fftLog.avgSize() : largeur
for(int i=0;i<fullMatrix.length;i++){
fullMatrix[i]=new PVector(0, 0, 0);
}
}
public void beforeDraw(){
background(0);
directionalLight(500,500,500,-100,-100,50);
//directionalLight(150,150,1,sin(radians(frameCount)),cos(radians(frameCount)),1);
ambientLight(80,80,80);
camera((x+1000),(y+0)-(TOTAL_TRACE_LENGTH*4),-1000-200,0,y-(TOTAL_TRACE_LENGTH*4),0,0,0,1);
//play the song
fftLog.forward(audioplayer.mix);
fillTempMatrix();
updateFullMatrix();
}
@Override
public void draw(){
beforeDraw();
int fftLogSpectrumTotalLength = fftLog.avgSize();
float saturation = 1.0f; //saturation
float brightness = 1f; //brightness
for(int i=0; i<(fullMatrix.length-1); i++){
//stroke(255,0,0);
//locator(trace[i].x, trace[i].y, trace[i].z, 1);
// float value = (trace[i].z*100);
float color_input = (fullMatrix[i].x);
float color_rescale = map(color_input, 0,fftLogSpectrumTotalLength*X_AXIS_SCALE, 0, 1);
Color myRGBColor = Color.getHSBColor(color_rescale, saturation, brightness);
//fill(myRGBColor.getRed(),myRGBColor.getGreen(), myRGBColor.getBlue());
//point(trace[i].x, trace[i].y, trace[i].z);
if( (i+1)%fftLogSpectrumTotalLength != 0 ){
line(fullMatrix[i].x, fullMatrix[i].y, fullMatrix[i].z, fullMatrix[i+1].x, fullMatrix[i+1].y, fullMatrix[i+1].z);
}
stroke(myRGBColor.getRed(),myRGBColor.getGreen(), myRGBColor.getBlue());
}
}
@Override
public void stop() {
// always close Minim audio classes when you finish with them
audioplayer.close();
// always stop Minim before exiting
minim.stop();
super.stop();
}
void fillTempMatrix(){
for(int i = 0; i plus grosses valeurs de z qu’actuellement
//calmer Z en fonction de i : gros i -> plus petites valeurs de z qu’actuellement
//pour l’ensemble du spectre rempli la valeur
//always get the next to last
tempMatrix[i]= new PVector(x, y, z);
}
}
void updateFullMatrix(){
//la matrice totale fait n*fftLog.avgSize() en taille
//car elle contient l’ensemble des frequences de 0 à fftLog.avgSize() pour chaque mesure dessinée a l’écran
//créer un tableau total (nommé FULL) de taille 100*fftLog.avgSize();
//une fois que le tableau TEMP est rempli avec les nouvelles mesures :
//décaler toutes les valeurs du tableau FULL de telle manière que FULL [0] = FULL[fftLog.avgSize()] et FULL[i]=FULL[i+fftLog.avgSize()]
for(int index=0;index<(TOTAL_TRACE_LENGTH-1)*fftLog.avgSize();index++){
fullMatrix[index]=fullMatrix[index+fftLog.avgSize()];
}
//remplir le tableau FULL avec TEMP : FULL[99*fftLog.avgSize()]=TEMP[0] ; FULL[
for(int i = 0; i<fftLog.avgSize();i++){
//System.out.println("99*fftLog.avgSize()+i = "+99*fftLog.avgSize()+i+" ; i="+i);
fullMatrix[(TOTAL_TRACE_LENGTH-1)*fftLog.avgSize()+i]=tempMatrix[i];
}
}
}
package minimsoundapp;
/**
*
* @author formation
*/
public class RunSpectrum {
public static final String mp3file1="C:\\Users\\Chris\\Documents\\JavaPerso\\SoundSpectrumProjectMinimAndJavaFX\\resources\\DeorroFtElvisCrespoBailar.mp3";
public static final String mp3file2="C:\\Users\\Chris\\Documents\\JavaPerso\\SoundSpectrumProjectMinimAndJavaFX\\resources\\BeFaithfulByFatmanScoop.mp3";
public static final String mp3file3="C:\\Users\\Chris\\Documents\\JavaPerso\\SoundSpectrumProjectMinimAndJavaFX\\resources\\KSHMRWildcard.mp3";
public static final String mp3file4="C:\\Users\\Chris\\Documents\\JavaPerso\\SoundSpectrumProjectMinimAndJavaFX\\resources\\NeverComeDownPRESIDENTIALREMASTER.mp3";
public static final String activeFile=mp3file4;
public static void main(String args[]){
ThreeDimensionSpectrum tds = new ThreeDimensionSpectrum();
tds.main(new String[] { "minimsoundapp.ThreeDimensionSpectrum" });
ThreeDimensionSpectrumCloudPoints tds2 = new ThreeDimensionSpectrumCloudPoints();
tds2.main(new String[] { "minimsoundapp.ThreeDimensionSpectrumCloudPoints" });
//todo faire une classe mère dont deux autres héritent, de manière a avoir un Override de draw() dans chacune et un nom de classe différent pour chaque
//de cette manière je pourrai en lancer plusieurs a la fois
ThreeDimensionSpectrumTriangles tds3 = new ThreeDimensionSpectrumTriangles();
tds3.main(new String[] { "minimsoundapp.ThreeDimensionSpectrumTriangles" });
}
}
package minimsoundapp;
import java.awt.Color;
import static processing.core.PApplet.map;
/**
*
* @author Chris
*/
public class ThreeDimensionSpectrumCloudPoints extends ThreeDimensionSpectrum {
@Override
public void draw(){
super.beforeDraw();
int fftLogSpectrumTotalLength = fftLog.avgSize();
float saturation = 1.0f; //saturation
float brightness = 1f; //brightness
for(int i=0; i<fullMatrix.length; i++){
float color_input = (fullMatrix[i].x);
float color_rescale = map(color_input, 0,fftLogSpectrumTotalLength*X_AXIS_SCALE, 0, 1);
Color myRGBColor = Color.getHSBColor(color_rescale, saturation, brightness);
point(fullMatrix[i].x, fullMatrix[i].y, fullMatrix[i].z);
stroke(myRGBColor.getRed(),myRGBColor.getGreen(), myRGBColor.getBlue());
}
}
}
package minimsoundapp;
import java.awt.Color;
import static processing.core.PApplet.map;
import static processing.core.PConstants.CLOSE;
import static processing.core.PConstants.TRIANGLE_STRIP;
/**
*
* @author Chris
*/
public class ThreeDimensionSpectrumTriangles extends ThreeDimensionSpectrum{
@Override
public void draw(){
super.beforeDraw();
int fftLogSpectrumTotalLength = fftLog.avgSize();
float saturation = 1.0f; //saturation
float brightness = 1f; //brightness
for(int i=1; i<fullMatrix.length-fftLogSpectrumTotalLength; i++){
float color_input = (fullMatrix[i].x);
float color_rescale = map(color_input, 0,fftLogSpectrumTotalLength*X_AXIS_SCALE, 0, 1);
Color myRGBColor = Color.getHSBColor(color_rescale, saturation, brightness);
if((i+1)%fftLogSpectrumTotalLength != 0){
beginShape(TRIANGLE_STRIP);
fill(myRGBColor.getRed(),myRGBColor.getGreen(), myRGBColor.getBlue());
vertex(fullMatrix[i+1].x, fullMatrix[i+1].y, fullMatrix[i+1].z);
vertex(fullMatrix[i].x, fullMatrix[i].y, fullMatrix[i].z);
vertex(fullMatrix[i+1+fftLogSpectrumTotalLength].x, fullMatrix[i+1+fftLogSpectrumTotalLength].y, fullMatrix[i+1+fftLogSpectrumTotalLength].z);
vertex(fullMatrix[i+fftLogSpectrumTotalLength].x, fullMatrix[i+fftLogSpectrumTotalLength].y, fullMatrix[i+fftLogSpectrumTotalLength].z);
endShape(CLOSE);
}
}
}
}
All you need is to create a Java project (in netbeans 8.1 it is nice) and import libraries of 'processing' and 'minim', you will have many jars in your project, these are required.
11/07/2016 11:48 998 602 core.jar
11/07/2016 11:48 4 149 gluegen-rt-natives-linux-amd64.jar
11/07/2016 11:48 3 050 gluegen-rt-natives-linux-armv6hf.jar
11/07/2016 11:48 4 130 gluegen-rt-natives-linux-i586.jar
11/07/2016 11:48 5 076 gluegen-rt-natives-macosx-universal.jar
11/07/2016 11:48 8 159 gluegen-rt-natives-windows-amd64.jar
11/07/2016 11:48 7 577 gluegen-rt-natives-windows-i586.jar
11/07/2016 11:48 345 605 gluegen-rt.jar
11/07/2016 11:44 105 446 jl1.0.jar
11/07/2016 11:48 224 010 jogl-all-natives-linux-amd64.jar
11/07/2016 11:48 164 852 jogl-all-natives-linux-armv6hf.jar
11/07/2016 11:48 217 274 jogl-all-natives-linux-i586.jar
11/07/2016 11:48 443 876 jogl-all-natives-macosx-universal.jar
11/07/2016 11:48 240 721 jogl-all-natives-windows-amd64.jar
11/07/2016 11:48 209 445 jogl-all-natives-windows-i586.jar
11/07/2016 11:48 3 402 029 jogl-all.jar
11/07/2016 11:44 46 786 jsminim.jar
11/07/2016 11:44 100 497 minim.jar
11/07/2016 11:44 24 538 mp3spi1.9.4.jar
11/07/2016 11:44 10 401 tritonus_aos.jar
11/07/2016 11:44 102 673 tritonus_share.jar
December 10th, 2016 at 5:47 pm
this is how it looks like ! https://www.youtube.com/watch?v=2DDoyX5vgpg
April 19th, 2019 at 7:12 am
This is now an open Source project :
https://github.com/chris282/Java-SoundSpectrum
See my linkedin profile for more info https://www.linkedin.com/in/christophe-bordier-01743779/
Christophe Bordier
Toulouse, France
November 1st, 2021 at 6:52 am
Such beautiful depiction of graphic designing and really boost my interest to showcase my graphic design work.