Lessons
Lesson Info
Set Up the audioVisualizer Code
alone. Welcome to processing in ordinary No in tandem and stress of gray. And this is a supplement to the original course in which will be learning how to create graphics that visualize audio data from a live microphone input. And we also have Gabriel Herberts in here today, who will be providing trumpet sounds for us live to show. Give us something to visualize in the audio stream. All right, so course supplement to build an audiovisual Isar. Here's an overview of the software, the parts of it. The first thing we'll do is we'll set it up to listen to a microphone on this could just be a microphone either built into your laptop, say, or one that you plug n externally. Um, display that as an audio wave, just the raw audio data as a wave form. And then we'll go into something a little bit more complex using FFT or fast for your transform, which will allow us to analyze that data and find frequencies by band. Basically ranges of frequencies kind of like an e que like display and then disp...
layed detected beats, which will use a beat tracking class in the minimum library that will be using this whole for the whole class to listen to those frequencies and, you know, analyze a certain frequency range. And if it sees a big change in it, then it will consider that upbeat. After that, we'll add in the hardware control using the project box. This is an example of how else you can control these visualizations with manual controls, the argument them. And then we'll do that to modify the way the waves air rendered and then finally will integrate all this audio analysis into the video mixer from the last supplement to this course just to show how you can integrate it with a larger application. So segment one set up the audiovisual Isar code, uh, will be saving in the beginning, you know, start a new sketch, create tabs of parts of the code and two and then just use a basic set up on the application. So, without much further ideal, let's hop into processing here. All right, let's expand out this window here and was gonna do the standard thing with processing void. Well, first I should save it. I'm just gonna call it audio visual Isar. And of course you can call it whatever you want. I like the name things, what they are. So first thing we do is the void set up and then void Draw. And, you know, if you're familiar at all with processing, which hopefully you are, if you're gonna fall along here, set up runs once at the beginning of program draw continuously runs a loop there after until the program has quit. Okay, then we're gonna create a couple of cabs, Um, up here at the little tab menu, The first tab is just gonna be audio. And this is where we'll put the core of our like, actually setting up the audio objects I on then will also be creating another tab called Effects, which will use later on in today's class to put in that Teoh organize the actual visualization code once it starts to become pretty complex. Okay, so then back inside this main audio visual Isar tab inside, inside, set up, we're just gonna add a few things to just kind of prep our application environment. First thing is gonna be a put a call to size, and we're just gonna pass in there. I'm putting in 6 by 4 80 You can make this whatever size you want. It'll probably run pretty well. Um, on most systems, because we'll just be drawing mostly vector graphics and vector graphics run rather quickly. So frame rate, um, as well. So we're gonna make it a 30 frames a second, and then we're gonna say no cursor, just cause we don't really need to see the cursor swimmers will hide it and then put cultists smooth, which is going to smooth everything out. It's just like anti alias ing. And then we're going to change our color mode as well to hue, saturation, brightness, mood, which you just passed HSB into the color mode method of processing eso just to gear up here, this class is gonna rely heavily on the minimum the minimum library by Damien Defeat and hop into Firefox here. We're gonna take a look at I'm just gonna do a search for minimum processing. I think that gets us there pretty quickly. Code log minimum. It's the first thing that shows up this page or this section of the this site has everything you would ever want to know about minimum. However, it is quite technical so I'll try to explain some of the things as we go along and definitely during Q and A. If there's questions, please ask. Um, but I would suggest, you know, if there's anything that you need more detail going here, it's got it. So back into processing. We're gonna go up to the top of this thing to the top of the main tab, and we're gonna go, uh, sketch, important library and then minimum audio. So minimum comes distributed with, um processing. And it added everything in there that we don't necessarily need all of that. So I'm just gonna get rid everything. We'll just keep minimum the main core of it, and then analysis. That's all will be using during this class because we're not gonna be the other stuff. It's for generating signals, which we won't be doing. Then we're gonna declare a minimum objects and notice capital minimum and then lower case. And we're just don't call it minimum lower case just to remind us what it is, Then couple more things below that, um, audio input and actually hold on a second before we get too deep into this. I mean, I cut this stuff out of here and actually put it inside of this audio tab to help us organize it. Okay, so just all I did was just cut and paste into the audio tablet. I had just started here, uh, audio input, eso that's minimums built in audio input. And it just looks for whatever your system settings are for your on your operating system. That's the audio input that it's talking about. So you have to go into your own system preferences and change what that's going to be, um, so just the declare audio end of type audio and put during a create afloat. And I'm using float here again because, um, it can it's a lot easier to do math on it. I suppose this could be an end, although I think that the functions that use it require flu. So this could be the sample rate, which is how often it's taking a sample out of the life signal and ring 44,100 which is a pretty common audio sampling rate. And then we're going to create an editor. That's the buffer size, which is how many samples of the audio that it's gonna pull out of the audio stream for us to work with at a time per frame Basically on. We're gonna make that 2048 now. Buffer size. It's always best to make this the power of to, um just because of the way a lot of the audio analysis math works. So 2048 is is a power of two on. Then we're gonna create a float on that's going to be an array float, and we're just gonna call it volumes and will use this to store the volume data that we get out of the raw audio waveform. So right below that we want to do is create a function to settle this stuff up, just called set up audio. And it's going to have type avoid because it won't return anything. It just does what it does in there. We're gonna in Stan. She ate minimum, and it's gonna equal on new a new minimum new object of class minimum on. We're gonna pass into that. This meaning the entire application space and then right below that was going to a little call to minimum dot debug And this will our debug on and minimum dot Debug on. We'll just print messages to the console down here about what's going on with the library as we're calling things. Then we're gonna go in and in Stan. She ate the audio in object that we declared before, which is our microphone in, and that's gonna be a minimum dot get line in and we're gonna pass into it. Three arguments, the 1st 1 being the minimum class itself has a constant called mono because we're just gonna listen toe one channel of sound. Although it is possible to listen to stereo with this, um, during a passing our buffer size that we set right above and then we're gonna pass in sample rate and then finally, right below there, we're gonna put volumes, and that's going to equal a new flu array. And we're gonna pass into it as its length buffer size because that's all of our audio samples per frame. And this will get updated every frame in later steps. Okay, so now that we've got this set up audio, we need to make a call to it. So you go back over to the main audiovisual Isar tab and then down here in the bottom of set up. We're just gonna put in that set up audio, make a call to that set up audio function so that when the program starts, it opens a connection to that microphone and starts listening. Okay, and then at this point you want to do is just try running it and see what happens. Um, it's not going to show us anything, but it is going to give us some messages down here in the bottom. That just tells us, OK, logged in or it not logged in. But, you know, found the audio signal is running and all that stuff. It's not giving us an error about that anyways, So that's it. That's the That's the beginnings of the set up and what we just did has created a new sketch, created some tabs, put stuff into code into and then added the video library and our I mean the it says video. Every but it's audio library and set up the basic. The change that slight later. Any questions on that? We had one question about the use of HSB versus RGB. Uh, and why you chose HSB. It just be is just a lot easier to do math with when you're trying to select specific colors because you can just if you want to change colors and you just change age. And if you want to change the saturation of everything, just change. Yeah, it's HSB is more commonly used in coding graphics just because it is so much more efficient. Math wines. Okay, and it's easier to visualize what you're actually trying. Dio got it because RGB color mixing stuff. It doesn't act like paint those or anything like right, OK, that makes sense.
Ratings and Reviews
Student Work
Related Classes
Mixed Media Art