Long-Term Mass Storage

Not sure if what you want to post fits in the other forums? Post it here!
.
User avatar
Posts: 202
Joined: Wed Sep 06, 2006 5:55 pm
Location: Southern California

Long-Term Mass Storage

Postby Dean Athans » Sun Oct 04, 2015 4:15 pm

Just a general question:

My collection of digital photos has grown to somewhere above 20,000 (not to mention the 14,000 or so actual film slides). I've done due diligence in my cataloging (date-wise, and subject), as well as keeping RAW versions of the JPGs on different disks.

A few (photographer) friends and I have talked about why we keep such mementos (for our children to someday think about Mom and Dad?), for how long, and how. After a many-hour backup of a failing hard disk this weekend onto a new hard disk, this last question is now bugging me: How do we store them? And, yes, I know that heavy-handed filtering is part of the "solution."

But, here is a more specific question: With DVDs at 4.7 GB, lower prices on external hard drives, and high-capacity thumb drives, what are the shelf-life, ease of access and other advantages/disadvantages that I should consider? How about the prospect of outdated technology in say 5 years when I may try to access them (without an appropriate reader)?

I'd really like to hear your opinions and maybe about your storage methods--even if you present additional storage methods, at least it will be something else to keep me awake at nights!

Thanks in advance for your opinions!

- Dean A.
Dean A

Valued Member
User avatar
Posts: 116
Joined: Thu May 12, 2011 3:07 am
Location: Near Coventry, UK

Re: Long-Term Mass Storage

Postby Tonel » Mon Oct 05, 2015 12:25 am

IMO for archival purposes the HDD has to be the best solution currently. DVDs & Blu-ray are not to be trusted for long term storage and are too small anyway. Thumbdrives/memory sticks and memory cards are not designed for long term storage - they are only for temp storage. SATA HDD are currently the most convenient, cost effective and safest storage solution.

I store everything on a Synology NAS (2x4TB HDDs) which is backed up nightly to another one.

I keep photos and videos in their source format, e.g., RAW, JPG, MOV, MTS, and don't expect them to go out of use and have to convert them to another format in what remains of my lifetime.

That's a very simplified response and there's much more but I'll spare you the detail.

So SATA HDDs, the biggest your system can accommodate and back them up to the same.

Tony
Self build Xeon W3690, 12GB RAM, C=250GB 850EVO SSD, Asus 2GB GTX960, Win10H 64bit, PSP9
iMac 27" Retina 5K i5, 24GB RAM, Radeon M295X 4GB, Bootcamp Win10H 64bit

.
User avatar
Posts: 7500
Joined: Wed Nov 29, 2006 6:35 pm
Location: Kirkland,Wash, USA, Earth

Re: Long-Term Mass Storage

Postby gpsmikey » Mon Oct 05, 2015 8:46 pm

I second the multiple hard drive option, but some care needs to be exercised. Use good quality drives (I have had good luck with the Western Digital Caviar Black drives). Make sure you are backing up to different physical drives not just a different partition on the same drive (if the drive dies, you stand a good chance of losing all the partitions). I actually back up nightly across my network to an older machine running Linux. Recognize that some of the raw formats may not be readable in the future (I have seen some heated arguments on that with neither side really proving their case) - I convert my raw to DNG when I extract them from the camera memory card (as well as geotag them if I have a GPS track covering the time the images were taken). Recognize that drive technologies change - that old IDE drive you have your backups on may not be readable by any of the newer machines that only have SATA interfaces (although there are adapters etc. out there currently). Another option is external drives that plug in via either USB or firewire (USB seems to have won that round currently). All that said, consider printing some of the best (or most interesting) ones and putting them in a drawer. Nothing like being able to toss all the technology aside and rummage through that stack of real pictures (even if they are missing a corner or have a crease across them). I have followed a number of discussions on backups and have not seen a single really good answer (something to be said for that drawer of pictures your parents had - limited in number because film was expensive etc.)

mikey
You can't have too many gadgets or too much disk space !!
mikey (PSP6, Photoshop CS6, Vegas Pro 14, Acid 7, BluffTitler, Nikon D300s, D810)
Lots of PIC and Arduino microprocessor stuff too !!

.
Posts: 700
Joined: Wed Mar 19, 2008 5:15 pm
Location: Glen Rock, NJ

Re: Long-Term Mass Storage

Postby tdew » Tue Oct 06, 2015 4:01 am

Also, storage in another location is good. Last Christmas I bought a few portable hard drives and loaded them with all the family pictures. I gave them as Christmas presents to my kids. I also use Backblaze and google photos as well as a bunch of copies on different online storage places.

.
User avatar
Posts: 7500
Joined: Wed Nov 29, 2006 6:35 pm
Location: Kirkland,Wash, USA, Earth

Re: Long-Term Mass Storage

Postby gpsmikey » Tue Oct 06, 2015 5:23 pm

tdew wrote:Also, storage in another location is good. Last Christmas I bought a few portable hard drives and loaded them with all the family pictures. I gave them as Christmas presents to my kids. I also use Backblaze and google photos as well as a bunch of copies on different online storage places.


Yes - forgot to mention that aspect of the backups. Backups on top of backups don't help if you have a fire or someone steals everything from the house. I have 2 identical 2TB drives that can be swapped (power down) and my backup software (I have SyncBack Pro) sees them as basically the same drive. I periodically swap those drives for each other and store the other one off-site.

mikey
You can't have too many gadgets or too much disk space !!
mikey (PSP6, Photoshop CS6, Vegas Pro 14, Acid 7, BluffTitler, Nikon D300s, D810)
Lots of PIC and Arduino microprocessor stuff too !!

Valued Member
User avatar
Posts: 129
Joined: Mon Oct 31, 2005 3:40 pm
Location: San Clemente, California

Re: Long-Term Mass Storage

Postby DonM » Thu Oct 08, 2015 2:24 pm

In addition to the suggestions above, recording on Blu-ray using M-Disks is an option.
The Wikipedia articlehttps://en.wikipedia.org/wiki/M-DISC , discusses testing
that confirms these disks avoid several of the issues that cause deteriation of conventional DVD and Blu-ray disks. They do require a writer that is designed to work with these disks. I have an LG that does this. Once written any Blu-ray player will read them. They are more expensive.

Don

.
User avatar
Posts: 7500
Joined: Wed Nov 29, 2006 6:35 pm
Location: Kirkland,Wash, USA, Earth

Re: Long-Term Mass Storage

Postby gpsmikey » Thu Oct 08, 2015 2:53 pm

That is an option (although I have 450GB in my "Photos" folder), but you need to watch for lack of hardware to read the disks. BlueRay has not quite caught on the way it was expected -seems most people are either streaming (NetFlix etc.) or just using a USB stick with mp4 files on it these days. I would expect blueray to be around for a while, but in 20 years, the disks may be fine, but nothing to read them with would be a problem (like the issue with IDE drives or the even older MFM interface - how many systems these days have the connector for even the IDE.)

I don't have all the answers. As Henry Blake (MASH) said one time - "if I had all the answers, I would be at the Mayo Clinic - does this place look like the Mayo Clinic?" Sometimes I think printing a bunch of the best pictures and tossing them in a drawer for future generations to snoop through might be the best idea!

mikey
You can't have too many gadgets or too much disk space !!
mikey (PSP6, Photoshop CS6, Vegas Pro 14, Acid 7, BluffTitler, Nikon D300s, D810)
Lots of PIC and Arduino microprocessor stuff too !!

.
Posts: 1666
Joined: Sat Apr 07, 2007 5:28 pm
Location: near Chicago - Illinois USA

Re: Long-Term Mass Storage

Postby heckydog » Thu Oct 08, 2015 3:18 pm

I must have started and stopped 5 times trying to reply to the OP's original post because it's such a complicated topic to talk about.

Dean wasn't thinking in terms of long term storage just for himself but also for several generations down the road. The first question I ask myself is will they really care 3 generations from now? Probably not but that's life.

Let's say they just might have an interest in what this generation did. Virtually everything is being digitally stored which means you need some kind of device to retrieve it. As we all know, these devices are constantly changing. Even if it's stored in "The Cloud", you need something to access it and a password too. Plus, you need the company that stores it for you to still be in business for the next 60-80 years. Good luck with that one.

I have an old cardboard suitcase I inherited when my mother passed away. In it are good old fashioned analog photos from the 1940's of mostly people that I have no idea who they are. So it's essentially useless to have most of them. But at least I can just pick them up and look at them. I don't need any device to do so.

Don mentioned M-disks. I've heard of them and supposedly they will last up to 1000 years. But you need a player to view them. That's the weak link.

I could give more examples but I won't for now. For me, the bottom line is don't even worry about any generation past your own children. If you're really THAT important, other people will document you for posterity. :wink:

Joe

.
Posts: 700
Joined: Wed Mar 19, 2008 5:15 pm
Location: Glen Rock, NJ

Re: Long-Term Mass Storage

Postby tdew » Fri Oct 09, 2015 4:15 am

Joe, That might be true for many people, but I hope that somewhere in the future of my family there will be someone like me who would love to have the family pictures and will treasure the work I've put into our Family History Story. Of course, labels on pictures or in file names are important too.

.
User avatar
Posts: 202
Joined: Wed Sep 06, 2006 5:55 pm
Location: Southern California

Re: Long-Term Mass Storage

Postby Dean Athans » Fri Oct 09, 2015 9:00 am

To All -

Thank you so much for your opinions!!!

It has been a long time since I've seen the discussion range from the practicality of storage (the technophiles) to the basic question of "Why?" (the philosophers and existentialists!)

First the bottom line: Yes, I will (almost) certainly get an HDD of at least 3 TB and 3 yr warranty. I'd like to go to a higher capacity but the prices and warranties start to become iffy for my budget and the lifetime I'd like. As for the 3 TB, at this point I could easily triple (maybe even pentuple) my present stash of all my photos/music--and that would be a long time at the rate I'm going!

Now back to your responses--GREAT! They almost exactly mirror what my three photographer friends and I have been through over the past 5 years--precipitated mostly by the shift into digital and the ease of taking multiple shots!

I started photography with a Ricohflex twin lens camera that my father had bought during the Korean war. From there I eventually shifted to 35 mm with my Minolta 101. (At the same time getting a super 8mm movie camera for the family recordings). Over a long time the cameras evolved, but the original media that they had produced didn't!

I remember going from prints to slides, from super 8 to VHS (my friend asked for my super 8 camera as payment for transferring all the [20 reels of] movies and I figured it was worth it as super 8 was rapidly becoming obsolete).

As for the many hours of VHS family records, I'm not sure if we still have them. The problem with all these media is that once we have recorded them, and then edited (such as into ProShow videos) them, even our children seem reluctant to "have to interrupt their busy family schedules" to view them.

Ahh, back to the philosophy of recording images and sounds. I'm sure I will never come to a resolution of why I do it! But, for the time being, my wife and I immensely enjoy seeing them, maybe for the first few months after we have made them. And, fortunately for us, we still have many friends who participate in travel with us and so have a fairly large (about 3 couples) viewing audience whenever we "premier" those shows.

As for posterity, I'm sure that eventually our children will have to clean up after us, and maybe spend no more than a few hours only glancing through our archives--enough to glean those that pertain to their earlier lives when they still lived "at home." I think that should really be enough to satisfy us, and maybe some of you too!

Thank you again for your thoughts on the technology--and on the bigger picture!

- Dean A.
Dean A

Valued Member
Posts: 99
Joined: Fri Oct 03, 2008 2:26 pm

Re: Long-Term Mass Storage

Postby Dewcal » Sun Oct 11, 2015 7:45 am

Dean,

I share your concenrs over longevity of digtial files. My "solution" is to have three copies on HDD with the following workflow -

I download onto a large HDD on my PC. This is then backed up (using Second Copy) to a Synology NAS and then to a second Synology NAS. I have run an ethernet cable around my garden and have the NAS boxes in different "sheds" in the corners of the garden. Hopefully if the house burns down, the sheds won't be touched (and vice versa).

I stay away from the cloud at all costs as that is, in my cynical opinion, allowing my assets to be held to ransom by the cloud companies. I was taught early on that nothing in life is free so am always looking for the catch when compannies offer somethign for nothing. Apart from companies changing their policies, pricing etc so far as I cna make out, all you need is one missed payment (eventhrough a bank software fault) to have your account deleted!

HTH
Dave

.
Posts: 700
Joined: Wed Mar 19, 2008 5:15 pm
Location: Glen Rock, NJ

Re: Long-Term Mass Storage

Postby tdew » Sun Oct 11, 2015 11:21 am

Dewcal, I don't think most responsible on line back ups work like that. From what I've read, if you stop paying, your account stays and gives you ample time to reinstate it at a future date.
In any case, I wouldn't advocate relying ONLY on cloud back up, but rather to use it as one more backup method. I pay for a smugmug account and put my best pictures there. I also pay for and use the backblaze back up for everything. At the same time, I'm putting lots of pictures onto google photos, 'cause it's free and so - why not? If they stop supporting it, I still have the portable hard drives in my house, some in a vacation home and some in my children's homes.
I believe it's best to back up to many different types of back ups in many different places.

Valued Member
Posts: 99
Joined: Fri Oct 03, 2008 2:26 pm

Re: Long-Term Mass Storage

Postby Dewcal » Sun Oct 11, 2015 11:32 am

You are probably correct but I will willingly admit to the charge of being an old cynic!!

I think the main point to take is use HDD and in several physical locations.

Dave

Return to Odds & Ends

Who is online

Users browsing this forum: No registered users and 2 guests