Media Library Backups

AsSiMiLaTeD

Two reasons for this thread, to share with others how I go about backing up my library and to see what solutions other people use that may be better.

I've got a fairly large 6TB library with a combined 5,000 or so albums, movies and tv shows. I ripped all this from my own media myself, a process that has taken YEARS to complete, and as a result I'm paranoid about losing my library.

I have 5 total copies of my library. I have the primary copy that I actively use and stream from and another backup here in our fireproof safe. I also keep a copy at work and in a safety deposit box. The fifth copy is an 'in transit' copy that I use to swap out with my other offsite copies and also gets packed when we go on trips. The idea is to always have copies in multiple physical locations so I'm covered if tragedy strikes.

I backup fairly often, more early on when I was ripping my collection. Now that I am adding content less frequently I backup much less often. To do the offsite backups I back up to the 'in transit' copy and take that into the office, bring the office copy back home and update it, which then becomes my 'in transit' copy and......well you get the idea. The primary copy that gets used here for actual playback is a very fast setup with an 8TB WD Thunderbolt drive and that never gets swapped out because I want the best possible performance for playback.

http://www.amazon.com/Book-Thunderbolt-Drive-High-Speed-Storage/dp/B00F0JXDB4/ref=sr_1_2?ie=UTF8&qid=1414253538&sr=8-2&keywords=8tb+thunderbolt

A few notes:
First, I try to use a mix of different models and and manufacturers for my drives. I don't want to have five of the same drive as my storage solution only to find out a year later that there's some type of defect specific to that model or brand.

Second, I don't keep any personal data on these backups, these are for media only so if they get stolen or lost I'm not at risk.

Finally, while I have typically used full size desktop drives with their own external power supply I have been slowly transitioning over to portable drives that draw their power from USB. These are more convenient without having the power supply and even though I have to carry three of those to match one of the bigger drives, they still end up less bulk overall. The main reason I changed though is because I have found them to be more reliable. The desktop drives have more moving parts with things like power supplies and fans and, in my experience, have been more prone to failure and I have yet to have a single portable drive give me problems.

Well this post ended up being longer than anticipated, but there it is. What do you guys do for your backups?

gce

I'm running a Synology with two 2TB internal HD's. I'm only using 20% of one HD. So I can backup on the other one for now. I also have an external 2TB HD that is for backup only. My system automatically backs up every week, so I don't worry about it.

I have mostly burned CD's on my system but also have some HDtracks downloads that is growing. The HDtrack downloads are also on my computer. So worse case I'll still have my CD's and HDtracks on computer and backup. And even worse case I'll have to listen to my LP's...all hard copy's.

tophatjohnny

AsSiMiLaTeD wrote: »

Two reasons for this thread, to share with others how I go about backing up my library and to see what solutions other people use that may be better.

I've got a fairly large 6TB library with a combined 5,000 or so albums, movies and tv shows. I ripped all this from my own media myself, a process that has taken YEARS to complete, and as a result I'm paranoid about losing my library.

I have 5 total copies of my library. I have the primary copy that I actively use and stream from and another backup here in our fireproof safe. I also keep a copy at work and in a safety deposit box. The fifth copy is an 'in transit' copy that I use to swap out with my other offsite copies and also gets packed when we go on trips. The idea is to always have copies in multiple physical locations so I'm covered if tragedy strikes.

I backup fairly often, more early on when I was ripping my collection. Now that I am adding content less frequently I backup much less often. To do the offsite backups I back up to the 'in transit' copy and take that into the office, bring the office copy back home and update it, which then becomes my 'in transit' copy and......well you get the idea. The primary copy that gets used here for actual playback is a very fast setup with an 8TB WD Thunderbolt drive and that never gets swapped out because I want the best possible performance for playback.

http://www.amazon.com/Book-Thunderbolt-Drive-High-Speed-Storage/dp/B00F0JXDB4/ref=sr_1_2?ie=UTF8&qid=1414253538&sr=8-2&keywords=8tb+thunderbolt

A few notes:
First, I try to use a mix of different models and and manufacturers for my drives. I don't want to have five of the same drive as my storage solution only to find out a year later that there's some type of defect specific to that model or brand.

Second, I don't keep any personal data on these backups, these are for media only so if they get stolen or lost I'm not at risk.

Finally, while I have typically used full size desktop drives with their own external power supply I have been slowly transitioning over to portable drives that draw their power from USB. These are more convenient without having the power supply and even though I have to carry three of those to match one of the bigger drives, they still end up less bulk overall. The main reason I changed though is because I have found them to be more reliable. The desktop drives have more moving parts with things like power supplies and fans and, in my experience, have been more prone to failure and I have yet to have a single portable drive give me problems.

Well this post ended up being longer than anticipated, but there it is. What do you guys do for your backups?

while I appreciate your idea of using different Hard Drives in the case the one model you currently has ends up with problems, but that wouldn't leave the same model you transferred your files to in defect status?? That would only mean that that hard drive would then need to be transferred to another hard drive...am I right??? Anyway, after consulting with my friend and a tech person at Seagate, I was told I am safe transferring one SG Expansion to another with zero worries, so I'm feeling good about that. I am still amazed at the number of albums you have ripped, and no way in my lifetime will I exceed the 1500 mark..(if I even get there). Still hoping to hear what others are doing with their media!

AsSiMiLaTeD

I don't think you're following my logic on using the same model for backups. Any drive can fail at any time, there's no way around that and that's the main reason for backups to begin with. All hard drives will eventually fail and most failures out there are very likely isolated incidents and not a result of a fundamental problem with the drive.

However, there are situations where a given model has issues or the increased likelihood of issues across the board. It's similar to how certain vehicles end up with recalls, somebody finds an issue a few months after a model hits the market and we get either updated firmware or a repair or replacement. The trick here is that we often don't hear about these issues until they're affecting people in the real world, you never know which drive is gonna have issues.

I've had this happen to me, TWICE. Once was in my original iMac where the standard hard drives were found to be defective and then replaced by Apple, luckily I didn't lose any data because they happened to catch mine before it failed. The other time was one of the early batches of Seagate 1.5TB internal drives that had a high failure rate, I had those in striped RAID and I was impacted when one of those died, luckily I was just running an OS and didn't have data stored there.

Ironically, both those drives were actually Seagate models, though I don't have anything against Seagate. In fact, the portable drives that I'm using are Seagate and have been rock solid for me so far.

I think it's difficult to make the case that using different models is a bad idea, so I'm not sure why someone wouldn't go that route, but it's ultimately up to you.

cnh

First, I am "impressed" with all that copy upon copy and even a safety deposit box! Wow! You make me feel like I'm living a life of total risk there.

Don't get me wrong I do back some things up. And I keep everything I own in pristine condition. I'm more ANAL than security conscious. But then again a lot of people here are all about "digital" copies, and I'm still in the stone age because I always keep my hard copies of everything as my most precious possessions. And of those, my books are even more important than other media. The only way to back up a book, a real book, not a digital copy of it, but the physical item, would be to buy multiple copies of it and that's financially prohibitive and then there is the space problem.

Sometimes I wonder if a fire that consumes everything wouldn't teach me something more than what is in all those books?

cnh

AsSiMiLaTeD

I think it boils down to priorities, I personally value the content of my media collection over the physical items. I still have the physical copies of much of my collection and would be super annoyed if those went up in flames, but if I were to somehow lose the digital copies as well and the ability to actually listen to my albums I would be CRUSHED.

My usual response to someone who is physical media only is "What happens when your house burns to the ground or is robbed?" The typical response to that is "If that happens then music and movies are gonna be the least of my worries". While I get the sentiment, I think the reality is a little different. Most of the items in our house can easily be replaced or repurchased if needed, media is different because it is more difficult to rebuild an entire medial library if needed. I also have quite a few albums that are out of print that would be impossible or cost prohibitive to repurchase.

It's not my intention to come across as combative and I obviously don't expect that everyone will take my same approach, but it's an interesting conversation.

Thorton

Thanks Assimilated for this thread. I'm always looking for better ways to improve my digital performance and secure my library. Currently, I have my library on a secondary computer with an external hard drive as backup. I have another external hard drive stored at my work office. I believe Falconcry was the one who turned me on to multiple backups not in the same location. I do not want to rip all my CD's again.

cnh

AsSiMiLaTeD wrote: »

I think it boils down to priorities, I personally value the content of my media collection over the physical items. I still have the physical copies of much of my collection and would be super annoyed if those went up in flames, but if I were to somehow lose the digital copies as well and the ability to actually listen to my albums I would be CRUSHED.

My usual response to someone who is physical media only is "What happens when your house burns to the ground or is robbed?" The typical response to that is "If that happens then music and movies are gonna be the least of my worries". While I get the sentiment, I think the reality is a little different. Most of the items in our house can easily be replaced or repurchased if needed, media is different because it is more difficult to rebuild an entire medial library if needed. I also have quite a few albums that are out of print that would be impossible or cost prohibitive to repurchase.

It's not my intention to come across as combative and I obviously don't expect that everyone will take my same approach, but it's an interesting conversation.

I totally understand where you are coming from and was not questioning that either; probably more talking to myself because I'm always expecting the "worst", lol And I wonder how I would respond!

But let's not wish that on anyone!

cnh

ZLTFUL

Besides my RAID5 NAS array and the monthly backups to an external drive that I keep in a fire safe that is kept in an off site climate controlled storage unit, I don't do much else. Most media I have bought in the last 4 years or so has come with a digital copy of some sort that I leave with Amazon or Flikster or the like depending on where I got it from.

I also share my media library with my brother who has a reasonably up to date version of it at all times. Like having my very own disaster recovery off site. Heh.

Gatecrasher

Even if a hard drive fails mechanically, you can still retrieve the data. It just costs money. Probably less than the cost of five backup had drives though. I've had a few hard drives become corrupted too, but could always retrieve the data with recovery software. If you have older hard drives that you are replacing, you can always use the older ones as some form of backup rather than just throw them away.

I have too large of a library (over 30 TB) to have 5 copies of it but only about 15 TB of lossless audio which is what I care the most about. The key for me is to spend the extra money up front and buy a high-quality hard drive like a WD Caviar Black or higher. WD now has a 4TB "Black" hard drive (they have since dropped the Caviar from the name). It is guaranteed for like 500,000 hours use.

The difference between the Black hard drives and the cheaper models are the internals. Black hard drives have dual ball bearing spindles compared to single bearings in cheaper units that are rated for less hours. The Black hard drives also have better (faster) performance than the cheaper ones.

Hard drive's days are numbered though. SSD is coming on strong and becoming more affordable. I can hang on a while longer with maybe buying a couple more 4 TB Black drives until hopefully SSDs become large enough to where I can swap everything over to solid state.

AsSiMiLaTeD

Correction, and a very important distinction, you MAY be able to retrieve the data. You MAY even have a great chance of recovering the data, but while the odds are good the whole idea of backups starts from a world of slim odds anyway. Chances are that your hard drives won't fail and that you'll move on to better technology before the drives reach their end of life.

Of the dozens and dozens of drives I've used to this point, I've only had three fail on me, and the data on only one of those was recoverable. There were some crazy expensive services that I didn't test, but those were several thousands of dollars.

I agree on buying better drives up front and especially since they're not that much more money, the WD Black drives are indeed nice and I've not had any issues with one of those.

Good drives are no substitute for a backup though. Even if you are guaranteed that your drives will never fail there is still the issue of theft or damage by fire to deal with. As I said above I can buy a new house and new belongings with the insurance money, the thought of having to replace my entire collection is sickening since I have years invested in that process.

By the way, I thought I had I had a decent size collection but 15TB of lossless music is damn impressive! Even with a decent mix of hi-rez content you've still gotta be pushing 30,000 albums, that's insane!

tonyb

15tb of music ? Seriously ? I'd be hard pressed to name 200 songs I actually like and will listen to. Very few albums/cd's that I would listen to every song, on average about 3 songs per worth saving or that I'd listen to again.

If I lost everything, even if I lost my whole computer....and all my gear on top of that, I certainly wouldn't be crushed. It's only "stuff"....and that can be replaced. Priorities gents, hard drives for backups isn't one of mine. Might be yours though, and I can see the need, but how much music do you really listen to ? We all have our favorites, and I'm willing to bet we just listen to the same 100 songs over and over.

As a test, I would challenge anyone to write down the songs they listen to over a weeks time....even a month. I'm guessing you'd be surprised at the repeats or the number of songs. 30,000 albums on hard drives....yeah, then your just a musical horder like grandma and her pie tins.

whgarrett1394

I am a musical horder, tonyb. I recognize that and I only own 150gb of music.

Anyhow, AssImilated. I will gladly keep a copy of one of your back ups if you want to keep a drive in a completely different region independent of natural disasters. Just throwing it out there.

EndersShadow

Gatecrasher wrote: »

The key for me is to spend the extra money up front and buy a high-quality hard drive like a WD Caviar Black or higher. WD now has a 4TB "Black" hard drive (they have since dropped the Caviar from the name). It is guaranteed for like 500,000 hours use.

The difference between the Black hard drives and the cheaper models are the internals. Black hard drives have dual ball bearing spindles compared to single bearings in cheaper units that are rated for less hours. The Black hard drives also have better (faster) performance than the cheaper ones.

I'd also throw the WD Red drives in that category. They are meant to be consumer NAS drives for use in RAID arrays. Got a buddy running 4 of the 3 TB's for a couple years w/ nary a problem....

The Green drives are what you watch out for if looking to do a NAS or RAID array...

Gatecrasher wrote: »

Hard drive's days are numbered though. SSD is coming on strong and becoming more affordable. I can hang on a while longer with maybe buying a couple more 4 TB Black drives until hopefully SSDs become large enough to where I can swap everything over to solid state.

Yeah, its gonna be years before that happens. They are just now getting to the 1 TB size for those. Hopefully it scales up quick, but it took quite a few years for spinning drives to break the 2TB limit and we are JUST now at 4 TB.....

But I very much look forward to seeing SSD's that size....

Moose68Bash

I don't know whether Moore's Law applies to SSDs, but if it does, 2TB and 4TB SSDs are not far away. And, the prices should come down.

AsSiMiLaTeD

TonyB you obviously have a much different relationship with your music, which is fine. A clear example of that is that I very rarely listen to music the way you do, I'd say about 90% of my listening is in album form, I tend to listen to albums not individual songs. The other 10% consists of custom playlists that I build for specific things and mostly the mainstream music from artists that are not my favorites where I tend to build greatest hits playlists instead of listening to full albums (Soundgarden, The Beatles, Billy Joel, Alice in Chains, etc). I still buy 100% of my music in album form, and that's how I tend to consume it.

I don't know how many songs I listen to on a frequent basis, but that's not how I look at things. In fact, my guess is that I listen to way less than 100 of the same songs weekly or even monthly. I tend to want to listen to a very specific album and then put it up for a while, I'll pull something like Dark Side of the Moon out and listen to it and then not touch it again for a few months. The trick is really knowing your music collection, I know my current collection very well but seriously doubt I could pull that off with a collection of 30K albums!

I purchased every single album in my library in physical form and individually. I didn't buy a giant lot somewhere along the way, I picked everything out specifically and it has taken me years to build this collection, I literally don't own anything else that covers that scale of time or that I've invested that much time into. I did buy a massive lot of classical CDs a while back, but have yet to go through those and they're not in my library.

My music is probably more important than any other 'material' thing that I own with the exception of my photos (I'd choose those over my music were I forced to make that decision). Music isn't just "stuff" to me, our residence our car and all my audio gear is just "stuff" because all of that can be easily replaced, music is different because it's not a simple replacement. Luckily my music and photos are on the same backups so I'm covered. If our place ever caught fire I'd grab my wife, my external hard drive, my phone, and a couple of my rare swords...in that order.

Ultimately you're right in that we all have our own priorities, and I agree that 30K albums would at least be too much for me to own. I'm happy and likely near capacity at 2500.

tonyb

So what do you do with the other 2470 albums ? LOL

I can understand collecting out of print music/recordings, but so far nothing mentioned isn't available again should a mushroom cloud go up over your house.

Just bustin' your chops pal, your right....we all have a different relationship with our music. Though even 2500 albums borders the crazy horder status.

Carry on gents...

AsSiMiLaTeD

Just to put things in perspective, given the size of my library and the method I use for ripping, I'm looking at somewhere in the neighborhood of 450 hours to get my music ripped and tagged. I did some quick maths, that's the equivalent of over 11 40-hour work weeks, OUCH!

2500 may be pushing it, you may be right there. I certainly don't rotate through that whole collection every year, but when I get the Tower of Power bug every few years it's nice having their stuff on hand.

Moose68Bash

tonyb wrote: »

So what do you do with the other 2470 albums ? LOL

I can understand collecting out of print music/recordings, but so far nothing mentioned isn't available again should a mushroom cloud go up over your house.

Just bustin' your chops pal, your right....we all have a different relationship with our music. Though even 2500 albums borders the crazy horder status.

Carry on gents...

I just learned recently that a friend of mine, whom I did not know was an audiophile, has over 4,000 vinyl LPs. He also has digital media, but I have no idea how many CDs.

I feel terribly deficient!

PrazVT

My music library (~480GB) is on my Synology DS413 NAS; well actually it's on a 2TB disk attached to the main NAS. So for the moment, I don't quite have any redundancy in case that disk goes down. Of course that thought just occurred to me as I'm writing this. So might be a good idea to move the music to NAS proper where there is RAID support. But otherwise, every time I have 25GB in my 'New Music' folder, I burn that music to a blu-ray disc and then deposit it in a CD booklet which I have stashed away. I like the fire safe idea though!

AsSiMiLaTeD

I like the Blu-Ray disc idea, you put those in a fire safe and that should be a fairly comprehensive backup solution. I'd need about 60 discs to back up my whole collection...

ninerbj

Assimilated, check your inbox please. :-)