{"id":395,"date":"2009-04-02T22:33:51","date_gmt":"2009-04-03T03:33:51","guid":{"rendered":"http:\/\/gracefulspoon.com\/blog\/?p=395"},"modified":"2015-02-23T18:14:35","modified_gmt":"2015-02-23T23:14:35","slug":"visualizing-sound-in-processing","status":"publish","type":"post","link":"http:\/\/gracefulspoon.com\/blog\/2009\/04\/02\/visualizing-sound-in-processing\/","title":{"rendered":"visualizing sound in processing"},"content":{"rendered":"

\"02_waveforms\"<\/a>
\nThis was the final applet in motion. Using the minim library for processing, each waveform is generated in realtime as the two sounds play over eachother creating a pretty chaotic sound, but there are some instances of overlapping patterns where the mashup works pretty well. In the third version of the code, the boolean of the two waveforms is generated, producing a new way to visualize the waveforms. View the youtube video here<\/a>, but I really need to figure out a way to add sound to the video, silence doesn’t do it justice. Charlie Parker, Iggy Pop and Richard Wagner comparison + code:<\/p>\n

 <\/div>\n

\"graphs_990\"<\/a><\/p>\n

 <\/div>\n


\n\"01_waveform\"<\/a>
\n\"02_waveforms1\"<\/a>
\n\"03_waveforms\"<\/a><\/p>\n

import processing.dxf.*;
\nimport ddf.minim.analysis.*;
\nimport ddf.minim.*;
\nFFT fftLog1;
\nFFT fftLog2;<\/p>\n

Waveform myRects;<\/p>\n

Minim minim;
\nAudioPlayer groove1;
\nAudioPlayer groove2;<\/p>\n

boolean record;<\/p>\n

PFont font;
\nPFont fontoutline;<\/p>\n

void setup(){
\n size(1200,600,P3D);
\n noStroke();
\n minim = new Minim(this);
\n groove1 = minim.loadFile(“groove_iggy.mp3”);
\n groove2 = minim.loadFile(“groove_wagner.mp3”);<\/p>\n

groove1.loop();\/\/repeat each song
\n groove2.loop();<\/p>\n

font = loadFont(“HelveticaNeueLT-Bold-18.vlw”);
\n fontoutline = loadFont(“HelveticaNeueLT-Bold-18.vlw”);<\/p>\n

fftLog1 = new FFT(groove1.bufferSize(),groove1.sampleRate()); \/\/create the FFT logarithmic scale
\n fftLog2 = new FFT(groove2.bufferSize(),groove2.sampleRate());
\n fftLog1.logAverages(22,4); \/\/adjust numbers to adjust spacing
\n fftLog2.logAverages(22,4);<\/p>\n

float w1 = float ((fftLog1.avgSize()+fftLog2.avgSize())\/2);
\n float x = w1;
\n float y = 0;
\n float z = 50;
\n myRects = new Waveform(x,y,z);<\/p>\n

}<\/p>\n

void draw(){
\n background(15);
\n directionalLight(126,126,126,sin(radians(frameCount)),cos(radians(frameCount)),1);
\n ambientLight(152,152,152);<\/p>\n

for(int i = 0; i < fftLog1.avgSize(); i++){\n int w = int(width\/fftLog1.avgSize());\n float zoom = 1;\n float jitter = (max(fftLog1.getAvg(i)*200,fftLog2.getAvg(i)*200 )); \/\/jitter in camera influenced by waveform\n PVector foc = new PVector((myRects.x*.5+jitter*.5), myRects.y+jitter, 0);\n PVector cam = new PVector(zoom, zoom, -zoom);\n if (frameCount < 260){\n camera(foc.x+cam.x,foc.y+(cam.y-1500*(cos(radians(-frameCount+60)))),foc.z+cam.z-400,\n foc.x,foc.y,foc.z-100,\n 0,0,1);\n \/\/println(-1500*(cos(radians(-frameCount+60))));\n } \n else {\n camera(foc.x+cam.x,foc.y+(cam.y+1418.278),foc.z+cam.z-400,foc.x,foc.y,foc.z-100,0,0,1);\n } \n }\n fftLog1.forward(groove1.mix); \/\/play each song\n fftLog2.forward(groove2.mix);\n\n myRects.update1(); \/\/update each waveform+boolean\n myRects.update2();\n myRects.update3();\n\n myRects.textdraw1(); \/\/draw z height for song waveforms\n myRects.textdraw2();\n\n if(record){\n beginRaw(DXF, \"output.dxf\");\n }\n \/\/ DXF will export the stuff drawn between here.\n\n myRects.plotBoolean(); \/\/create surfaces\n myRects.plotTrace1();\n myRects.plotTrace2();\n\n if(record){\n endRaw();\n record = false;\n println(\"Done DXF~!\");\n }\n}\n\n\nvoid stop() { \n groove1.close(); \/\/ always close Minim audio classes when you finish with them\n groove2.close();\n\n minim.stop(); \/\/ always stop Minim before exiting\n super.stop();\n}\n\n\n\nclass Waveform{\n float x,y,z;\n\n PVector[] pts1 = new PVector[fftLog1.avgSize()];\n PVector[] pts2 = new PVector[fftLog2.avgSize()];\n PVector[] pts3 = new PVector[fftLog1.avgSize()]; \/\/needed for boolean waveform\n\n PVector[] trace1 = new PVector[0]; \n PVector[] trace2 = new PVector[0];\n PVector[] trace3 = new PVector[0]; \/\/needed for boolean waveform\n\n Waveform(float incomingX, float incomingY, float incomingZ){\n x = incomingX;\n y = incomingY;\n z = incomingZ;\n }\n\n void update1(){ \/\/plot boolean waveform\n plotB();\n }\n\n void plotB(){\n for(int i = 0; i < fftLog1.avgSize(); i++){\n int w = int(width\/fftLog1.avgSize());\n\n x = i*w-1050; \/\/adjust the x position of the waveform here\n y = frameCount*5;\n z = height\/4-fftLog1.getAvg(i)*10;\n\n stroke(0);\n point(x, y, z);\n pts1[i] = new PVector(x, y, z);\n \/\/increase size of array trace by length+1\n trace1 = (PVector[]) expand(trace1, trace1.length+1);\n \/\/always get the next to last\n trace1[trace1.length-1] = new PVector(pts1[i].x, pts1[i].y, pts1[i].z);\n }\n }\n\n\n void plotBoolean(){\n stroke(255,80);\n int inc = (fftLog1.avgSize()+fftLog2.avgSize())\/2;\n\n for(int i=1; i<(trace1.length+trace2.length)\/2-inc; i++){\n if(i%inc != 0){\n beginShape(TRIANGLE_STRIP);\n float value = (trace1[i].z*100); \n float m = map(value, -500, 20000, 0, 255);\n fill(m*2, 125, -m*2, 140);\n int threshold = 15;\n if (trace1[i].z220){
\n textFont(fontoutline, 24);
\n fill(155);
\n text(“wagner”,200,500,0);
\n text(“iggy”,900,500,0);
\n text(“max(iggy-wagner)”,500,500,0);
\n }
\n }
\n }
\n void plotTrace1(){
\n stroke(255,80);
\n int inc = fftLog1.avgSize();<\/p>\n

for(int i=1; i<\/param><\/param><\/param><\/embed><\/object><\/p>\n","protected":false},"excerpt":{"rendered":"

This was the final applet in motion. Using the minim library for processing, each waveform is generated in realtime as the two sounds play over eachother creating a pretty chaotic sound, but there are some instances of overlapping patterns where the mashup works pretty well. In the third version of the code, the boolean of […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[4,10,12,9,15],"tags":[74,77,80,76,85,92],"_links":{"self":[{"href":"http:\/\/gracefulspoon.com\/blog\/wp-json\/wp\/v2\/posts\/395"}],"collection":[{"href":"http:\/\/gracefulspoon.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/gracefulspoon.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/gracefulspoon.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/gracefulspoon.com\/blog\/wp-json\/wp\/v2\/comments?post=395"}],"version-history":[{"count":13,"href":"http:\/\/gracefulspoon.com\/blog\/wp-json\/wp\/v2\/posts\/395\/revisions"}],"predecessor-version":[{"id":3857,"href":"http:\/\/gracefulspoon.com\/blog\/wp-json\/wp\/v2\/posts\/395\/revisions\/3857"}],"wp:attachment":[{"href":"http:\/\/gracefulspoon.com\/blog\/wp-json\/wp\/v2\/media?parent=395"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/gracefulspoon.com\/blog\/wp-json\/wp\/v2\/categories?post=395"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/gracefulspoon.com\/blog\/wp-json\/wp\/v2\/tags?post=395"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}