Lessons

Lesson Info

Media Management & Archiving

Media management and archiving. You're done with your show, okay? You never want to look at it again. You spent all this time, but you really need to at one point, so what's the best way to clean house and make it so that your show is accessible to you later and you also are clearing off your drives and managing locations. And again, this is one of those challenging things about I have all this media. How do I store it, how do I archive it? It's one of the challenges because we are able to shoot so much because we have so many tools to capture video now and video is so large. So I want to kind of just have a conversation to start this off. We'll talk about the importance of media management and archiving. Hear some of your questions, concerns, maybe some of your experiences with trying to save archive projects. We'll look at some strategies. That's part of the discussion. I'm going to actually step in to Premiere and show you the project manager, which is part of the software that allo...

ws you to do a little bit of organizing as you archive, and then we'll talk about some of the best practices that you should follow when you're doing this archiving, once you've actually collated and possibly slimmed down the amount of media you're using. So with that, does anybody have at this point questions about oh my gosh, media management archiving? So I have questions not just about the archiving piece of it, but the best practices of where you want to store things as you're, obviously we talked about ingesting, but where you want to store things as you're working with them versus then where you want to store them later to save space and obviously we know there's the creative cloud, people may have like, I happen to have a network attached storage drive and then there are things like OneDrive and I'm sure Amazon has some sort of a solution as well, other types of clouds that you could use. Probably called Amazon Drive. Probably something like that (laughs). But sort of, what are the best practices for places that you're storing it at different points in your workflow? I think that that's a great place to start because you can actually clear house and organize, where have you kept things to start with? And depending on your personalities, how retentive you are or how free flowing and ADD you might be and having files everywhere. So where you're storing your media to start with is important and some of that is based on whether you're working on it for desktop, which you may have a lot of internal storage or attached storage that you're not going to remove the drives, or you may be working off a laptop or you're always on the go and you have lots of external storage, but sometimes you want to just be able to run and gun and work with just what you have locally. So the challenge is probably it's a little bit of everything. We saw when we actually ingested media, we do have the luxury of working with proxy files. So if you wanted to, you could archive all of your big files on an external drive and then put those proxies on your laptop and then you're a lot more mobile and then you're not filling up the limited hard drive space on a laptop, so that's one thing to keep in mind. But as a best practices option, you should try to always use some sort of storage workflow that works best for your media whether it's storing everything on a production drive. I have a situation where if I'm working for multiple clients I may have a dedicated hard drive for each of those clients, so as I'm doing more work, I'm going to copy and plug in that drive and target that drive and that way if the client needs something, I can find it a lot easier. I also use software to clone those drives or back up those drives in case that drive gets corrupt, I don't lose the entire client's database of information and as I indicated in an earlier lesson, I use a lot of solid state drives but I may back up to a massing network area storage device or Raid, which is cheaper per megabyte or per gigabyte or per terabyte but gives me the redundancy. So deciding where your media should go is important from the beginning because you want to be able to find it and move it when necessary. Now the next question is, well what about things like speed and storage space and whatnot? Video are large files, so you may not want to store it internally. I, myself, actually carry around quite a number of drives because I have so much media and I'm always filling up my internal drive and you can, with some formats, I sometimes record 4K that is ultra high definition files using Apple ProRes, something we learned about earlier and I will fill up a half a terabyte on a half hour shoot, okay? So I mean, that's a ridiculous amount of space, so I can't put that internally and I want to be able to move those files quickly because again, I'm an impatient person, so that's where I use the solid state drives. So choosing a good drive is a good place to start. I currently, this is my personal thing. I have a little greater fear of spinning drives these days than I did five years ago. They're a lot cheaper, but I feel that they fail more frequently and so I tend to lean towards the solid states and I'll back them up to spinning drives, but I always back up and I'm going to emphasize this throughout. The speed of your drive matters in some instances. If you're dealing with having to move very large bandwidth files and if you ever did a Get Info when you're playing a QuickTime movie or something, it can actually tell you. It tells you frames per second, but it also tells you how many megabytes per second and some of these higher definition uncompressed movies, they need a lot of bandwidth and what happens is, if you have a lot of those playing at once, for instance, multi-cam. Maybe you have five or six cameras. You are reading five or six video files off that drive at the same time and sometimes you'll get a performance hit no matter how fast your computer is because you're limited by the speed of your hard drive and there are software programs, applications that are free. One that comes to mind is Blackmagic, which is actually manufactured. They make something called the Blackmagic Speed Test or something, you can just Google it. And what you do is you point to a drive and it says how fast you can read and write to that drive and that's an important thing to understand, how fast your drives are, because maybe you want to store something on an older or slower drive, but when you want to edit, you want to know which are your faster drives and it's surprising sometimes, the result. And just as a side note, I have even discovered that with some of my fast drives, if I use a cheaper USB cable, I actually get a performance hit and sometimes I'll have this, it's like, this drive is really great, but I have a kink in the cable and for some reason it's now writing and reading at a quarter of a speed that it should, so I swap out the cable and I'm back again. So that's just something to file away that it may not be the drive, it may not be the computer, it may be the cable, it may be the fact you plugged it into a USB two port on your PC versus the USB three port on your PC. So these are all variables. So when I'm editing, I want the fastest drive for what I need. So if I'm using something highly compressed like H.264, those are small files. So getting it from the drive to my CPU is pretty easy. The CPU is going to work a little bit harder to decode it in some cases. Apple ProRes, which is nice and big, I may have a challenge getting two streams out of the drive, but the computer would be fine once it gets it, so maybe I need to move that to a faster drive or internal. So it's this juggling piece of information. You'll also, when you start getting more exposed to working in video, you'll do a lot of reading on the web and you'll see things such as oh, you should have one drive that you're reading from and then another drive that you're writing to and another drive for your preview files because you don't have to worry about bottlenecks and speed. In a perfect world, that's really nice, but it's not a perfect world, you're not a production studio. Drives are a lot faster. Computers are a lot more efficient. I use my laptop, I read, I write, and write my preview or render files all to the same drive at the same time and if it slows down a little, I really don't notice. So yes, you will see that stuff, but don't let it confuse you. You can generally read and write to the same drive. If you have big files and you're exporting, sometimes I'll export it to an external drive, so I'm reading off of one and writing to another. Often I don't do that unless it's something really huge like a one hour documentary with hundreds and hundreds of edits and clips. So knowing where things are is important. The problem is and this is where coming to using the project manager is a lot of times when you're cutting a show, you'll bring something and you'll go oh, I'll download this piece of royalty stock footage from the web. I drag it to my desktop, I do it, and then I put it into Premiere and I've successfully now put something into my program that is not properly stored. This is going to happen all the time whether it's a graphic style, maybe you built something in After Effects and you saved it over here and you drag it over here. We're not all perfect, though we hope we are, but ideally you should move it into the right folder first and then ingest it into Premiere. In the real wold, you're running and gunning, so you're going to be moving stuff and that's where the challenge comes in, is what happens to media that isn't necessarily in the right location so I can't just grab that project folder and copy it over. The other challenge that you're facing is well, what if I didn't use all this footage? Why do I need to archive it and save it because it really doesn't necessarily belong with this one show? So I want to be able to consolidate everything as small as possible in a single location so if I need to go back and work with it, I can find it and that's really where we're moving to with this section of the class. So that's kind of understanding the important of knowing where things are, but now we have to figure out the important of cleaning house. One is so you can find everything when you need it and two, you want to be able to delete it from the hard drive or your internal drive so you have the available space to work on your next project. So with that said, let's go ahead and hop into Premiere. I'm going to be working with one of the files that we had open, actually right in the earlier lesson. So this is the opening show. It's (mumbles) we go in. So I have this and I want to go ahead and I want to archive this. I want to. I've already exported, I printed my movies, life is good. Now I have to figure out what to do with all this media. So I'm going to go ahead and under File, there is something called the Project Manager and this is the best way to clean house and make sure you have everything you need. So what we're going to do, really, for the rest of this lesson is we're going to step through what all these options are so you're in power to know what you can do and what you need to do. So as soon as I open up the project manager, I'll see this interface and if you look closely, these are all of my sequences that I built in this project and in a sense, but not in a sense, but I have a couple of sequences that I don't remember building, but what happened was, I created a nest. We talked about nests earlier. Okay, we put a bunch of things inside a container. Well, that nest is considered a sequence, so that's why it shows up. I made basically this nest of some of my animation so it would be easier to work with, so I see that and what's nice here is I can choose what I want to back up, so there might be different reasons I'm backing things up. I might be backing up just that one sequence because I want to just save that or hand it off to somebody and they don't need all the other clutter. I don't have to back everything up from this project. On the flip side, maybe I do want to back up everything but I want to make sure it's all in one place and organized, so I can choose at this point in time, what sequences and I'll see every sequence that is in that project in this first location that I want to back up. So I'm going to go ahead and I am going to check everything. Now I'm going to move down and we have some other choices based upon what our objective is. So the first checkbox. The default one says collect files and copy to a new location. And this is great. This says I'm going to archive this to an external hard drive or another location on my current drive. And when you do that, it will actually take all the files, move it to a single location so you don't have to theoretically worry that you have lost something that later on when you open it up, it was in your music folder. And then you pick the location and what it will do is it will move everything. It will actually copy it. I'm sorry, copies it. So again, non-destructive. You have some options with this over here. One of the checkbox says exclude unused clips. So if you brought a lot of footage in and you didn't use it and you don't think you want to save it, you can check that and it will reduce the amount of space you need to archive that project. You're assuming that you're not going to go back and say oh, I want a different take or I wanted another shot. You can always go back to your camera original that you archived, of course, that SD card that you put in a very safe place when you made the copy of the folder because you never throw anything away. But if you want to save some space, and sometimes I will do that, especially if I brought in a lot of stuff. A documentary, you might have a 100 to one shooting ratio and you want to make maybe even just a version, you know, that it's like, I can put here and I just know this is my final show. You can say exclude unused clips, okay? There are a couple of these that are unavailable for me to check and I'll explain why as soon as I go through the existing ones. I generally do not check these, include audio conform files, include preview files. What those are your render files and the analysis files of when it looks and defines the way for them. The nice thing is, when I open up a project, it will recreate those as needed, so why save or use that extra space? The only time I may say save this is because I want to move it to another drive. I'm still working on the project and I don't want to wait for it to actually recreate my render files if I need them or recreate the audio WAV form files. So if I'm still working on a project, I may check those and have it save it, but if I'm just putting it on the shelf, it's like I can regenerate that as needed while I use the space, okay? You do have an option, when you archive, is if you change the name of a clip within the application, so if you clicked on any of these clips in you browser as you're working on it. I could actually, if you notice, some of these have different names than probably they were recorded when I shot them. The camera gave it a certain number. So if you rename a clip, you have the option when it media manages it to change the name of that clip to whatever you chose it to be while you were editing, which is kind of nice because now when you just look at the move clips, it actually has the right name and Premiere knows the new name to connect it to on this new project file it's going to make. So it is a nice feature to rename it. So again, depending on your workflow, that's one of your options. Before we look at all these other elements, you need to choose where you want it to go. So you're going to say browse. What's the destination? Create a folder to put it in. It creates a folder anyway. I like the redundancy in case I forget something and it's going to put all of this into a single location that you target, whether that's another place internally on your computer or hard drives or maybe on an external hard drive where you're going to archive it. I generally archive, as a rule of thumb, to an external hard drive and I'll explain why in a few minutes. That's my check thing. So here we go. Disk space available. Do I have enough room? It does not know the size of the final project until I hit the button calculate and it will make its best guess that my original project was 33 gigabytes. The new one will be 16 because I'm ignoring all my unused media, okay? But until you hit calculate, it won't figure that out. And then if I'm happy with that, I can say okay. It starts copying, and moving, and organizing, and it will create a brand new project with new sequences and move a copy of all the media into a single folder and you're good to go, okay? And when I say that I like to put this on external hard drive, as a rule of thumb, computers aren't perfect. We are, but computers aren't perfect. So what I like to do is, once I've archived it, I take that drive and plug it into a different machine and open up the project and make sure everything is there and is reconnected because sometimes you may have something very deep into like, a nest, or maybe you have an After Effects or a Photoshop document that doesn't have the information embedded, but it's pointing to the media on another part of the drive and Premiere is not aware of that, so it's nice to know if something's offline and if you just put it in the original machine, Premiere sometimes knows where it looked before, so it sees it still and now, a month later, you plug it in and you've deleted everything off your internal drive and suddenly, that media is gone, ever to be found again. So it's a good check and balances if you have a second system. Of course, if you don't have a second system, it's a little harder, but I do like that as a workflow to always check the moment I archive the file. So let's look at some of the other options that you can use and also how it thinks and how it works. So if you check consolidate and transcode, some of these other options are now accessible. So first of all, we talked about transcoding earlier. That's basically converting from one flavor of media to another. So sometimes when you archive, you want to turn everything into a common flavor or maybe you want to convert it into something that is a smaller file. Maybe everything you shot is ProRes and you want to just archive like, H.264 because it's 1/20th of the space. So as soon as you say consolidate and transcode, you now get to choose what format you want to convert it to and these are archival formats. You'll also notice that as going back to format, remember, I remember your question earlier. It's like, why don't I say H.264? Because in this case, you'll see H.264 once you're inside one of these flavors. And by the way, you'll probably use QuickTime. These, again, are more broadcast formats that you would deal with, so I want to convert it to QuickTime and then I have these different flavors. We're going to do ProRes. If you build a preset in Media Encoder, you can, but usually the idea is with transcoding it, you're doing it for a reason such as you want it all to be the same flavor and easy to work with and that's why ProRes is one of the common formats. At this point, I can actually do something kind of interesting here. I can say include handles and where people get kind of get a little bit surprised is that it only works for some of the media. And this changes over time, so who knows? In six months, they'll release something new and they may change this, but a lot of times, stuff that's recorded at H.264, that highly compressed format, instead of trimming it, it just saves the whole clip, even if you only ever used a few seconds of it. And part of the reason for that is it is so compressed that it actually references information that is further down our earlier in the video clip because literally what it is doing, it is using something called long GOP structure. I say that because it's the end of another day of live and you're like, wow, what is this? And I like saying the word GOP. GOP stands for group of pictures. It's basically the way that they compress things is, many of you are photographers and you're familiar with RAW and JPEG. Well, JPEG just throws away superfluous information that our eye cannot necessarily see, even though technically, it is throwing away really useful information. They do the same thing with video. Every frame can be compressed, but it can only get so small if you compress each frame from say, RAW to JPEG. Now that they do to be clever is they say you know what? What if things don't really change between frame one and frame two, and frame three? Why don't we just not repeat that information, we just look back at the earlier frame? So if, say, you're recording me and it's all RAW. You record every single pixel of this room and the wall. When you get to JPEG, you take a picture that says you know something? There are 35 blue dots over here. I'll just do 35 times blue at this luminance level and then we can to (mumbles), it's a different color. So I can make a smaller file. And then, when you start getting into that lossy JPEG, they say you know something? There's 40 light blues and 35 semi-light blue, so well just do 75 semi-light blues. The guy won't notice. And now you get an even smaller file. So you see, you're throwing away information that we won't say, well, they do the same thing with video, but because video is over time and you'll hear the term temporal just to be, again, fancy, they can say you know something? I was standing here talking. That wall is not changing from frame one, frame two, frame three, so let's only record what changes between each frame and we can make this file even smaller. And so what they do is they can't do that so long because if you've ever played a game of telephone in elementary school or sometimes maybe in a restaurant, I go to the first person. I give you a sentence and you whisper it into the ear of everybody and by the time it gets to the back of the room, it's a completely different sentence. So the same thing would happen with the compression of video. So what they do is like, every 15th person, they tell them the original sentence again. They tell them the original sentence again. They look a the full frame of information to make sure that it actually looks right. So that's called GOP. Long GOP says not only ae we going to look every 15 frames, we're going to make it even smaller. We're going to look a minute down the timeline or a minute down the recording and now, you're getting it really small, but you need to have a lot of information to recreate that one frame that allows us to make such a long file. So a long way to say it's easier and more efficient and sometimes more bulletproof to save this entire little clip than trying to cut it up at the risk of cutting it between those full frames of information. So that's why it happens, but I just don't want you to be surprised when you say oh, trim it and it doesn't trim it because it's actually smarter not to. Some of the things you'll see is covert image sequence to clip, so we're going to learn about image sequences when we actually do time lapse when you have 200 images or 2,000 images. If you want, when it saves it, instead of saving those 2,000 images, it's going to convert it to a real movie that it will reference. Again, how do you want to archive it? Do you want to be able to go back in and get into the original media, or is it okay if you just have the movie? Okay, saving space decisions you need to make. Generally, I actually don't check that. I like to have my original stuff, even if it's going to eat up a lot of space. And then, if you use After Effects files, you can actually convert that into a clip also, make it into a movie. And the other thing is, if you have creative things that use transparency, that's one of the things that we've been working through on this. You could have it preserve those alpha channels, those transparency channels. It creates a larger file when it's saved, but again, if you're going to be using that, it's much more efficient. So that's what you're ultimately doing, but sometimes collecting and copying the first thing is fine, it's very efficient. I find I use that most of the time and I go ahead and I hit okay. I let it do its thing and then I check the files. As a matter of fact, I immediately open the new one, even if I'm doing it on the same drive, I still open it to make sure it was clean and good and nothing got corrupted. And now I have something smaller that I can move. The original media is still there, okay? So you need to physically erase that and move that. It's a safety factor. So I've archived it by an original (mumbles). Maybe I've archived it just because I want to have an interim solution to back up. So I will go into the finder level, the operating system level, and knowing where I put things, I will remove them. I will miss things because I have put things in odd places, but that's okay I'll find them hopefully eventually, but I can move that big folder, that big chunk of information because it's all archived in the same location, either to another drive or to just simply if I know everything is in the first location, I can hit delete. So that's the idea of doing project management through Premiere and cleaning house manually when you're done.