Lessons

Lesson Info

Read and Display the Audio Data

all right segment to read and display the audio data. So that audio line that we're listening Teoh, we're gonna create functions that handle that audio and put on Ben store the audio data as an array of floats in that volumes array that we created. And then we're gonna access those volumes to draw away form to the screen. So going back into processing, um, you go into the audio tab again, and I was gonna lower that more room to see and then down below the definition of the set up audio function, we're gonna put another function called of type, void called get Audio Data. And then inside there were just gonna create a four loop and gonna kind of sketch up at four loop and then passing its arguments. It's gonna be just the kind of classic, and I equals zero semi colon and then Aziz, long as I as less than buffer size, then increase. I buy one each time through the loop and then inside there, quite simply, all we're gonna do is take that volumes array, the index I of that array and a sign...

it the audio in that mix, which is the minimum audio in audio inputs mix of both audio signals, which we told that we wanted. And at the beginning, when we in Stan, she ate it. And then we're gonna called Get on that. And what we're gonna do is pass I into that as well, because what we want to do is get this, um, samples of volume amount and then pass it into here, which will then draw the screen. Okay. And then one other thing we need to do to just, you know, is our most basic set up of the audio stuff is we're gonna use something that's called voids Stop, which is not really well documented. It's It's actually pretty much part of the processing core, but it's not really called out too much because it kind of gets run automatically. But in this case, we need to call it explicitly. And it happens when you quit the application. Something to note that on a lot of systems, you know, you can quit your sketch while you're playing it by hitting escape or by closing it with the little, you know, thing at the top of the window. But that doesn't necessarily call stop for some reason, hitting apple or command Q or control Q. On a PC will actually make this function run and the reason we wanted to, because we're gonna put audio in the close and it's gonna close that microphone listener. And if you don't call it explicitly at May or may not, depending on your system, actually close and tidy up all the memory and stuff like that, and you also need to explicitly call stop on minimum to tell it to close down as well. So is there any way to capture if somebody clicks on the little button, um, and then force it to call Stump? There's a There's some discussions on the processing forum that you could dive into on that. But for all intents and purposes right now, if you do close it the other way accidentally, it's not the end of the world. Um, it's just this is kind of recommended way to make sure that things have shut down properly. And then there is also a call to super dot stop, which is basically in a stop. Everything, and I think super is more of a job of thing, right? Basically what this is going to do it is free up. Memory and system resource is when we quit our application. So it's ready to be used again when we started up again as we're testing. Okay, so then, now that we've got our get audio data and our stop function, we can go and grab this, get audio audio data and make a call to it inside of draw. That way, volumes is being updated with the freshest audio data from the microphone. Every frame, then below draw. We're gonna create another function, and this is kind of like a core tool now of our audiovisual Isar, and we're gonna call it avoid volume Wave, and we're gonna pass into it. Ah, afloat. Where is the call and which is for amplitude? And that's how high the waves will render. That'll be, like kind of the maximum height, sort of, um basically, amp amplitude gets multiplied against the volume range and then to create the height of the wave. And then inside there, we're gonna create two floats for X, which we're just going to start off a zero and a flute. Why? Which? We're also going to assign zero and then we're gonna put inside it here something called Begin Shape, quads, strip. And that's all capitals with an underscore in between. And if you just happen to be familiar with open GL programming, this will look familiar if you're not. What I would say is, you know you can always double click on anything that gets colored in here. Like if it turns orange or blue, you can double click on it, so select it and then right click on it or control click on it and in Good on. The finding reference and begin shape has these different arguments. If you just don't pass anything into it, it's just going to draw kind of regular fill sort of lines around the outside sort of thing. But you can pass into it. Other things you can pass in points. It will just draw the points lines. It'll just draw lines that aren't connected, um, and then so forth and we'd start getting down into. These ones are a little more complex, and the one that we're using specifically now is Quad Strip, and what it does is every two points that it draws. It just connects them together so it draws two points and that draws two more points and then draws two more points. And it just connects those together and turns them into rows of rectangles or quarter angles because they could be a random angles to each other and stuff. But that's that's essentially long in the short for quads strip, I'd say, you know, go into these examples in the reference and, you know, try them out. You know, if you're having a hard time understanding what's going on here, um, we're gonna have to every time you call, begin shape you off toe also have to call end shape to tell it that we want to stop drawing its. We're just gonna go ahead and put that in now and then inside there were going to create another four loop that's going toe loop through our volumes array and draw those of Texas Those two Vertex is each time an organ druthers in a row to create our audio Wait for him. So inside this four again, just kind of a standard, and I equals zero I as less than in this case, the total width of the screen cause we're gonna want to draw all the way across the screen, and then we're gonna increase. I buy one each time. Then inside that four loop, we're gonna do a little bit of fancy math. Not too bad. We're gonna create afloat a local variable called percent, and we're gonna sign it. Um, float I and noticed that flow is inside. The parentheses were typecasting. I, um so because it starts out as an end, but we're gonna do some division within. We want to make a percentage, or like a decimal. Um, so we needed to be treated as a floating point number two, Get that dot something 20.0.0 whatever. And then I believe that we're gonna do an end called Index that will use Layer to find which index and the volumes array that we want to use. And then we're gonna go We're gonna round, um, what we're gonna assigned to because it is an imager. It has to be around it number on day. We're gonna take our percent, and we're gonna multiply it by the volumes arrays, length property. So basically, what this is going to give us is is where in the volumes array. Do you want, Uh, do we want to grab? We're going to grab a slice of volume data based off of the X coordinate essentially by by using this system, that makes sense a little bit tricky math, but not shouldn't be too bad. Um, then down here, we're going to assign X I plus one on, and that will be the exposition of each segment of the wave form that we're drawing. And then we're gonna assign why, uh, the absolute value of volumes index that we just discovered up here using this math and the reason we want the absolute value is because what I want to do, what we're gonna do is draw kind of a mirrored wave form so that it's the same on the top and the bottom, um, and volumes in minimum. Just raw audio volumes are in a negative one to positive one range automatically. But as far as I could discern, essentially, when it goes negative, it's actually a full range. It could be considered like a 0 to range, and so this is kind of a way to handle that. Um and then we're gonna multiply this whole thing by the amplitude that we set up. And then finally, we're going to draw a vertex that gives us our exit. Or why? And then we're going to draw another Vertex right below it. That's our X with the same x cornet. But it's gonna be why times negative one so that it's now whatever this amount is below the middle point and we'll be using translate later to push this whole way of form down to the middle of the screen has been a full. Everything starts in the top left, zero would be the top left. So if we just ran this volume wave right now, we would only see half of our way for him up at the top of the screen. So we'll have to use translate. So basically, you're just looping over every single exposition on the width of the screen, right? And then computing how far along you are and taking that position in the volumes and then drawing that strip at that position. Yes, as you go across the screen, right? Exactly right. We're finding the part of the volumes array that corresponds to that exposition on the phone are on the on the screen. So I'm envisioning that sort of waving that That makes sense. Cool. Yeah. Um So what we need to do then, is OK. It looks like in the slides it skips. It skips a step which isn't sliding. Maybe need a little more work, but those are my own notes right now. So you guys don't have to worry about that Inside draw. We're going, Teoh, create a call to volume Wave, and we're just gonna pass into it A mouse. Why? Actors are amplitude. And then right before that, we're going to use that translate that we that I was just talking about and we don't want to translate it all in the exposition, but we do want to move it down. So we're gonna take height, and we're gonna multiply it by 0.5 said that now the middle. So now zero. Why? The zero of the Y axis is the middle of our applications window and you do a couple of things with colors. We want our background to be zero or black and or you make it whatever color you want. And then the film that we're gonna draw our way of forms with is going to be 55 and I'm gonna just turn on no stroke so that we're just seeing the shapes without any outlines around them. So with that in there, did everything correctly, and we should be able to see some audio on game or you. Uh huh. There you go. That's the rothe raw Trump data analyzed through the screen. All right, so that's that segment s. So what we just did is created functions. They handle this audio input and to store it into an array that we can use the volume data into an array first years. And then we use that array to draw the way form. That was the visualization of Gabes trumpet playing. So here we go. Any questions? And I haven't seen any questions in the chat room mix that for a question. A math question about when you multiply the height by 0.5. And they were curious if multiplication and processing is faster than division, divided it by, so it is faster for sure. Okay. And it also helps proven things like type casting as much stuff. Okay, although I don't know, it actually doesn't. It's just multiplication is faster, easier and it's easier to visualize like, Oh, I'm going halfway across the screen, which is 50% which is 500.5. Okay. Right. Although you could put divided by two, but I think you have to divide it by two point of to make sure that you've got a float that you're dividing with, okay?