Lessons

Lesson Info

Extended Workflow: Storage Options

Options and recommendations. So, this is where we'll get the nuts and bolts, and I'm sure we'll get lots of questions here. We've already talked about the different types of drives. There's hard disk drives. So, HDD is the abbreviation for that. Solid-state drives, which are no moving parts, they're like compact flash cards. That's definitely the new drive technology that's really exciting, because it's much, much faster in terms of how fast you can read and write to those drives. I had high hopes that solid-state drives would be a thousand times more reliable than hard disk drives. The testing over the last 10 years has shown that they're not really anymore reliable than solid disk, 'cause it's still, the reason why is it's still, you're storing data magnetically. And anytime you have this, I mean, I could go way deep off into the stuff here. I have friends who are physicists, researchers, that study how particles streaming from the sun, neutrinos and all these high energy particles, ...

can actually change the data from back to zero, and flip the magnetic thing on very minute parts of a chip. This is another reason things could change in your safety deposit box, because high energy particles streaming in from the sun or the atmosphere, or the universe, could go in here and affect the chip where this is magnetically stored on tiny, tiny, little parts of that chip. And NASA, this is a big concern for NASA, because there's no shielding or protection for this, or not as much as we have on the planet Earth, because we've got an atmosphere that shields us from a lot of that stuff. So, when you're in space and trying to record to hard drives, you're gonna have to change those stuff way faster. I mean, NASA has to replace their cameras every three to four years because the chips are just destroyed from all these high energy particles hitting that chip and killing particles, or changing the structure of that sensor. So, that's a whole different, we went off into the weeds there, but a little extra bonus info. RAID Arrays. These are another type of hard drives. RAID Arrays can either be built from solid-state or hard disk drives, it just depends on fast you want it to be. Tape drives, we did mention. DVDs and CDs are still around. I haven't recorded to a DVD drive in, I don't know, six or eight years. And I gather most of us, since most of our computers don't even have DVD drives or CD drives anymore, probably don't use those. But that's still a decent archival form of storage for a certain period of time. So, technically, there's only three hard drive manufacturers in the world for hard disk drives, spinning disk drives: Hitachi, Seagate, and Western Digital. And I think Seagate is now owned by Western Digital, and has been for the last two or there years. So, reality is there's two hard drive manufacturers, and one of them owns another company. So, whatever hard drive you buy, if it's a spinning disk, hard disk drive, it's gonna have one of these three brands of drives in it. LaCie has, I don't even know what brand they have inside of their drives. It may be different depending on which LaCie drive, LaCie hard drive you buy. Hitachi owns G-Tech, G-Technology, so all of their hard drives are Hitachis. You can buy all manner of Seagate drives, all manner of Western Digital drives. You know, I'm sure if we took a poll in the class and online, everybody's had one of these three brands fail on them at some point, depending on who they are and their experience. So, it's not to say one is better than the other. There is a company known as Backblaze, which is a cloud backup for your computer, that did testing on these three different brand drives, because they have all three in their giant data center, with thousands and thousands of these drives, and they found out that Hitachi had a slightly lower failure rate than some of the other drives, which you can look that up. If you look up hard disk drive failure rates, you're probably gonna get to that Backblaze blog post pretty quickly. And it was pretty fascinating to read. And, also, enterprise drives will have a lower failure rate, typically, than your average non-enterprise drive. But you're also going to pay a premium for that, as we've talked about. So, solid-state drives. So, here's portable drive, there's an internal drive, this is a solid-state drive. This is actually an internal laptop solid-state. Typically, solid-state drives run four to 10 times faster than a hard disk drive, in terms of read/write speeds to the drive. And that's irrespective of the connection speed, 'cause that doesn't really matter for most purposes. Now, RAID, do we still have any kind of questions here? We're good. All right, excellent. Thank you. So, RAID, this is an acronym that stands for Redundant Array of Independent Disks. So, it's basically saying you have more than one hard drive inside of an enclosure that's being somehow used as one single hard drive. And the reasons you would choose to do this are, A, you could put eight eight terabyte drives together into that enclosure, and now you have a hard drive array that equals 64 terabytes of hard drives, instead of just one eight terabyte drive. So, for large catalogs of images or video, it comes in handy. It also massively helps improve your read/write speeds to and from your hard drive. So, you might see these advertisements about, "Use our super, wicked fast solid-state drive "to access your 8K video footage." which they're obviously marketing to a pretty small number of a people, because not many people these days have 8K video cameras. But when you're working in Premiere Pro, trying to work with 8K video, that is so much data, that you need a crazy fast hard drive array just to even like actually work with that footage in some scenarios. So, there's, I don't know, five or six different RAID Arrays, in terms of how the hard drives are being used. We'll go through just a few of them. There's three that photographers often use. There's what's called RAID 0. So, RAID 0 means it's putting part of the information here and part of the information there, and it's simultaneously reading both drives. And RAID, just to explain, can either be a hardware thing, solution inside the enclosure, or, in some instances, they could be a software solution, where the software's on your computer that's doing all the calculations to read or write the RAID, write to the RAID Array or read from the RAID Array. Typically, hardware solutions have been way more stable and way more reliable. There's been a few software solutions like SoftRAID. And with these software solutions, you can create any one of these RAID Arrays we're gonna talk about, that are now really good. So, the beauty of this, is that let's say we have four of these hard drives put together, and they're running in RAID 0. So, if one hard drive is like 180 megabits per second, let's just say 200, an even number, so it's easy to multiply, and we multiply that by about four, we might actually get up to 800 megabits per second read/write time out to our computer, which means we're gonna be able to access that information way faster, Lightroom's gonna go really fast, because it can access all the images and the previews and everything lightning fast off of that hard drive array. The downside of RAID 0, is if any of one of those hard drives fail, toast, that whole array is gone. Poof. So, that's a little dicey. That's not to say that it's not good, you just have to back it up like anything else. RAID 1 is actually the diciest one for photographer in my book, if you ask me. It is actually called data mirroring, so, basically, it's taking whatever the first hard drive is, and mirroring that one a second, so that it's automatically backing things up. So, you don't have to think about backing up. As soon as you write something to this, it's being written to that. As soon as you delete something off this, it's being deleted off that. So, they're two identical drives. For photographers, I definitely do not recommend RAID 1. Because if you accidentally delete that image, then it's deleted on both drives. And if you don't have it backed up in a third place, well, then, it's gone. So, while it's useful for some things, there's not fault tolerance. Like, if you make a mistake, it's made, if you don't have it backed up somewhere else. So, that's, you know, with Time Machine these days on the Apple computers, you might be able to overcome that, but I haven't always had great luck with Time Machine, so, that still seems a little dicey. So, one of the favorites of photographers is called RAID 5, and RAID 5, again, will use a four-disk array. So, it's an enclosure with four hard drives in it. Part of the data is being written to each of the four drives, and 25% of the drives, in this case, since there's four of them, part of each drive is being used to describe what's on the other hard drives. So, in this scenario, it's called a parity sector on each drive. If one of these hard drives fails, no big deal, we pull it out, we put it in a new one, the other three hard drives rebuild the data on that drive, and nothing is lost. So, this is not a backup of strategy in and of itself, but it just makes a more robust system. If two of these hard drives fail, it's toast. Poof. Gone. So, if you have six or eight hard drives in an enclosure, then maybe you can lose two hard drives. But if you lose more than two hard drives then it's gone. So, how big you make this RAID system depends on your risk management. But, still, that's not a backup in and of itself, that's just a bigger hard drive array. So, the question then becomes, like, well, if I have a 40-terabyte RAID 5 Array, how do I back that up? The answer is buy another 40-terabyte hard drive array, or you buy a bunch of individual drives, and then record pieces of that information out. So, as soon as you step into RAID world, the prices explode and get quite high really fast. Just be aware of that. So, here's something showing you RAID 5. So, this Ap, Bp, Cp, and Dp, that's the parity sector of each drive, showing you how it's kind of striped across the different drives, which, it gets super confusing, super fast, if don't know a lot about drives. And even for me, I'm just like, okay, whatever, it's doing its thing. That's all I care about. So, in my flow chart, I have a bunch of these RAID 5s; one backing up the other, and then I have it also on individual disks. So, there's a wide range of options here. And this is why this is such a huge topic. I did a workshop on this for like two days once, going into the nooks and crannies of all this stuff, and really talking about all the options. 'Cause there's a million different hard drives out there. Here's a giant RAID Array. That could be anywhere from $5000 to $20, for that box of hard drives, depending on what you put in it. Here's my G-Tech G-DOCK, which is a great system, and it can do RAID 0 or it can do RAID 1, 'cause it's only got two drives. So, it can do mirroring or it can do striping. So, why would you wanna do RAID 0? If you want really fast read/write access to your hard drives, you would go RAID 0. But that's definitely not a backup solution. I wouldn't recommend RAID 1. You know, portable drives; this is the brand new G-Tech G-DRIVE Mobile. Those things are wicked fast. They just came out. I'm showing you a lot of G-Tech stuff, 'cause that's one of my favorite drive companies, and knock on metal, I've never had one go down on me. I'm sure they fail occasionally, so, it's not like they never fail. There's plenty of other people making good hard drives out there. So, I've been referencing these read/write speeds a bunch in the whole section. And I think I talked about it earlier in the class. There's these theoretical speeds that these connections will work at. Like, Thunderbolt 3 is 40 gigabytes a second. Thunderbolt 2 is 20 gigabytes a second. Those are theoretical read/write speeds, they don't exist in the real world, actual read/write speed when you start copying files from one hard drive to another. Because, as we talked about earlier, it is the disk itself. And I think I show these exact slide earlier in the class. So, the bottleneck is that actual physical drive, and how fast can it go. This is an SSD, this is an Other World Computing SSD. I think some of those G-Tech drives they just came out with, the new ones, could 600 to 700 megabits a second. So, it's a little faster than this. I know Other World Computing and a few other companies just came out in the last month with solid-state drives that can do 2800 megabits per second. So, that is wicked fast. But they cost $1800 for one of those, and they're like one or two terabytes. So, it's how fast you need it, and how fast can you afford, you know, that whole conundrum, just like the cameras. And here's a RAID 4, a RAID 0 Array. This is the same one I have actually. And you can get up to 1300 megabits per second just by buying four hard drives that are spinning disk drives, and putting them in a RAID 0 configuration. So, how fast you need to access your files might be a factor. So, brands I recommend. Hitachi's my favorite hard drive. I've got a ton of the Seagates as well, the Barracuda drives, that are anywhere from three terabytes, to six terabytes, to 12 terabytes. There's these drives, they're coming out on the bleeding edge of capacity, like 12 and 14 terabytes. I tend to wait a few years before I move into those larger drives. Gotta make sure they've got the technology figured out. So, if they're coming out with 12 terabytes, maybe I would go to like eight or 10 terabyte drives, 'cause those have been around for much longer and perfected. You know, I'm not gonna trust everything to like the latest drive that hasn't really been tested out in the world. G-Technology is one of my favorite. CalDigit. You've probably never heard of CalDigit. US government uses a lot of CalDigit hard drive enclosures. They are they Ferraris of hard drive enclosures. They're still using either Hitachi or Seagate or Western Digital drives in them. I think they use Hitachi. They only use enterprise class drives. G-Technology uses a lot of enterprise class drives in their enclosures. So, when you look at like G-Technology's RAID Arrays, the prices are gonna, your eyeball is gonna pop out of your head. Understand that they're all enterprise class drives. So, each hard drive in that array is two to three times more expensive than maybe the one you're comparing it to from a different company, like LaCie. So, Other World Computing is pretty specific to Mac, but they've got great value to price. Great value. Performance to price is excellent. And they're not maybe as sexy-looking, like this hard drive enclosure's just a warped piece of metal. It's not maybe as stylized as some of the other brands, but they're pretty rock solid in my experience, and they've got a wide array of stuff. Promise Technology makes some amazing arrays. They were the first company to actually come out with Thunderbolt, way back in the day, when Apple introduced it. Synology's another great brand. And there's tons of photographers that have had great experiences with Synology. Typically, their hard drive arrays are a little bit slower, because of the connection speed than some of the other faster drives. But super solid company as well. And you'll notice only the top two of these are actual hard drive manufacturers. The rest of these are just brands that building enclosures, and putting a hard drive in there. And, definitely, if you're buying an enclosure, there better be a fan on the back of that sucker, because there's a lot of heat being generated, like, my girlfriend comes downstairs, she works in the house with me, she's a science writer, and she walks into my office, she's like, "Wow." in the winter, she's like, "It's pretty warm in here." I'm like, "Well, I've got my heating units right here." my whole array of hard drives, blowing that heat off the drives themselves, and keeping me warm during the winter. In the summer, it's not such a great scenario, but just so you can see what's going on. The Hitachi Ultrastar drives are phenomenal. And they do have those enterprise options as well. The Seagate Barracuda, as I said. I've already mentioned the G-Tech drives. The Thunderbay 4 is my main drive of choice these days, from Other World Computing. That's not to say I wouldn't recommend other ones; the G-Technology arrays. There's a company Areca, A-R-E-C-A. They're not very well known in the photo sector, but they're really well known in the industrial cloud service world for their name. They haven't really gone after the photo sector, but they make some pretty amazing products.