What's the best way to archive my pics?

[mikeci]

Hi Colin,
last week I read an article on the M-disk and that got me thinking about archiving my work which in turn led me to pose the question to you and everybody else on the site. From what I have read of the M-disk it is bullet proof and should out last any other form of storage. It needs a special burner but can be read on any modern dvd unit. its only short coming it that it only has a capacity of 4.7 gb. If I go with the M-disk system I will need burning software that will allow me to break up the burn into 4.7gb segments.
Mike

[Colin Southern]

Though I'm sure DVD/CDs are being continually improved, the problem is the metal layer the data is burned into

Are we talking the same technology here? My understanding is that there's no metal in DVDs - just an organic layer that the laser destroys. I think M-Disks are metal though.

[Colin Southern]

Hi Colin,
last week I read an article on the M-disk and that got me thinking about archiving my work which in turn led me to pose the question to you and everybody else on the site. From what I have read of the M-disk it is bullet proof and should out last any other form of storage. It needs a special burner but can be read on any modern dvd unit. its only short coming it that it only has a capacity of 4.7 gb. If I go with the M-disk system I will need burning software that will allow me to break up the burn into 4.7gb segments.
Mike

Hi Mike,

Yeah - I've had a bit of a read up when someone posted a link here. Be interesting to see if it catches on. Since I'm using similar capacity media at the moment that shouldn't be a problem - I'm usually working on a folder by folder basis anyway - and if there's any room left over I'll also flick on the next folder as well. If they're all in 1 big database though then you'll need something that can handle file spanning.

[jwalsh]

I use both, but to be honest, my biggest issue with "cheap hard drives" is how much data one loses if one fails (and I've seen quite a few fail), whereas with DVDs, they're cheap as chips too - and the limited abount of data is somewhat of a bonus in that there's less to lose if one fails to fire up. Obviously in both cases there are other copies, but personally, I feel like the old DVD isn't quite done and dusted yet. Works for me anyway.

How much is Amazon charging for their storage?

The first year is loaded with tons of discounts so the transfer up/down is free for me currently. I havent had a bill from them yet more then 30 cents. I have about 30 GB of photos and 1 GB of websites backed up in it currently.

Here is their calculator. http://calculator.s3.amazonaws.com/calc5.html
Make sure you use the S3 section.

They even offer 'reduced redundancy storage' option, which is fine for everything I do. It is about a third of the cost of the normal storage. If I remember corrently it takes the normal storage's reliability of like 11 9's down to 4 9's (99.99%). Which is fine for offsite backup when I have two local copies already.

As for hard drives failing, first off in my entire time using computers in the last 20 some years, I've had 2 hard drives fail, and didn't lose any data. Regardless, now a days you have multiple drives in raid arrays that can withstand outages and even withstand hot swap replacements and you lose no time or data. If you build it right hard drives are the best storage device.

I can see CD/DVDs as maybe a medium that you leverage to give clients the photos, but depending on how tech savvy you are, you can leverage the Internet for that type of work as well.

The only right answer through all of this is to ensure you have a local on site copy somewhere on something, and an offsite copy.

[Colin Southern]

As for hard drives failing, first off in my entire time using computers in the last 20 some years, I've had 2 hard drives fail, and didn't lose any data. Regardless, now a days you have multiple drives in raid arrays that can withstand outages and even withstand hot swap replacements and you lose no time or data. If you build it right hard drives are the best storage device.

I think my personal failure rate is probably about the same - I've worked full time as an engineer in the computer / electronics industry since 1978 though, and we still get one or two a year on client PCs. We do RAID 1 configurations for our ocean-going vessels (touch of irony; so far we've had 2 failures - one a disk, the other the RAID controller). We use RAID 5 arrays with hot-swap too, but still back them up to LTO4 tapes every night; it's not so much the chance of a disk failure that will cause data loss, but corruptions from various sources (we've had a brand new IBM server running VMWare dropping virtual servers once or twice a week that the suppliers & IBM have only just been able to trace to a RAID controller firmware issue; we didn't lose any data each time it happened (thank goodness for transaction-based file systems!), but when one of the virtual hosts is running an entire exchange-server installation for a multi-million dollar company, it gets a bit scary). I've also lost data from a RAID 1 configuration; the motherboard malfunctioned and broke the RAID set, and the backup I thought I had turned out to have corruptions in just about every large (read *.PST) file due to a fault with a USB controller on the same motherboard (I never liked that motherboard!); what saved my bacon on that occasion was that I had all of my commercial photography work backed up on DVDs (that worked like a charm) on that occasion.

So not disagreeing - but I do think that one still needs to plan a little for Murphy's law - even with backup solutions that appear to be bullet proof.

[Bear]

Are we talking the same technology here? My understanding is that there's no metal in DVDs - just an organic layer that the laser destroys. I think M-Disks are metal though.

...sorry, CDs aluminum, DVD dyes. M-Disc looks interesting, purporting to resolve all the issues I mentioned. Prices look reasonable too.

At one time gold was being considered as the burn layer, however burning it hot enough with a low error rate was the problem. Don't know what the M-Disc "stone" layer is, but two factors are going against the longevity of the optical disc in general, the format itself and data density (capacity), and M-Dsc apparently has a slow burn speed, 4X.

http://www.extremetech.com/computing...sts-1000-years
http://www.engadget.com/2011/08/15/m...r-a-few-minut/

Funny thing is that the entertainment industry drives the media product development, which is mostly short term (deliberately), however the entertainment industry has the problem of archiving its assets, so the big money goes into consumer media and profits get eaten by archiving.

[mikeci]

Just a quick note on the M-disk. I was about to order a burner and some M-disks to start my journey into archiving my photos but it seems they dont make a unit that works with the Mac, this according to the LG website. I know I can setup a double boot system and load windows on to my mac but I really don't want to go through the trouble. I've got an email into Millenniata to see what they have to say about it. Hopefully they will have a answer for me.

Mike

[Colin Southern]

M-Dsc apparently has a slow burn speed, 4X.

I'll probably give them a go if they appear locally. Burn speeds generally aren't an issue for me; watching disks burn is generally akin to watching grass grow or paint dry -- I generally just start the process and then just go do something else.

What I might do though is put a SSD in an external enclosure -- then I could easily transfer a shoot to SSD - bring it home - and write it to (slow) HDD there.

[Paw Paw Bill]

I use Carbonite. This service is automatic, meaning anything that gets to the harddrive in the folders that I have selected will begin to backup to servers in Colorado. I pay $50/year and have a over a terabyte up on their drives now. Additionally I have terabyte external drives that I periodically back up to. My laptop has 750 gig and all my recent stuff is kept on it as well (about 500 gig in photos at present).

I can access anything I have on Carbonite from anywhere I get Wi-Fi with the laptop. I have downloaded a few older shots to show, from sites away from home and find no problem with them.

I still have a few thousand cd's stored away from the days that predated DVD. Once or twice a year I will pull a few and read them, just to make sure that they are holding up. Some are over 15 years old and still good.

[PhotoRob]

What I might do though is put a SSD in an external enclosure -- then I could easily transfer a shoot to SSD - bring it home - and write it to (slow) HDD there.

Good point Colin. SSD's are very interesting; get a good one and you've got comparable read and write speeds, otherwise write performance can be lacking. All my operating system drives are SSD (MAC, Laptop), however I also have a haul-around 500GB Momenutus Hybrid, which has 4GB of NAND SSD + a 7200RPM Hard Drive and find that the speed is quite good; price was only $99, which may have crept up a bit with the issues in Thailand...

[Colin Southern]

Good point Colin. SSD's are very interesting; get a good one and you've got comparable read and write speeds, otherwise write performance can be lacking. All my operating system drives are SSD (MAC, Laptop), however I also have a haul-around 500GB Momenutus Hybrid, which has 4GB of NAND SSD + a 7200RPM Hard Drive and find that the speed is quite good; price was only $99, which may have crept up a bit with the issues in Thailand...

I'm using a RevoDrive3 x2 for my system drive at work at the moment - pure bliss for photo work, although bigger than 250GB would have been nice (as opposed to "nice and expensive"). Looking for a USB3 SSB Vertex unit now - might be my Christmas present to me

[PhotoRob]

I'm using a RevoDrive3 x2 for my system drive at work at the moment - pure bliss for photo work, although bigger than 250GB would have been nice (as opposed to "nice and expensive"). Looking for a USB3 SSB Vertex unit now - might be my Christmas present to me

OCX does put out a nice product, you bet - SATA III Vertex 240GB's (wow those are fast) are going for about $440US on Amazon. I'll put in a good for you with Santa

[jwalsh]

So not disagreeing - but I do think that one still needs to plan a little for Murphy's law - even with backup solutions that appear to be bullet proof.

I agree, and it sounds like we have similar sys admin backgrounds. But for the general purpose of this thread, photogs wanting to backup their photos, I think that our sys admin backgrounds and backing up of company critical data is a bit different than this. Yes it is someones business, but like we were saying, working copy/local backup/offsite backup, is how you address this. The media is secondary only in that you are positive that it works, and that you can restore from all the locations.

What I just find funny is the large amount of talking about burning cd/dvds in this thread. Those things have all but almost died a slow death due to proliferation of fast Internet connections. Something like Carbonite/S3/Dropbox all have published lower failure rates than any CD/DVD backup solution you could implement yourself.

[Colin Southern]

What I just find funny is the large amount of talking about burning cd/dvds in this thread. Those things have all but almost died a slow death due to proliferation of fast Internet connections. Something like Carbonite/S3/Dropbox all have published lower failure rates than any CD/DVD backup solution you could implement yourself.

Ah yes, but don't forget that in this part of the world, we don't have unlimited data plans; in fact - at present - my work connection is capped at only 10GB per month, and I currently have about 600GB of images held locally - would take 5 years to upload it on the present plan (assuming I didn't use it for anything else).

[mikeci]

I think this is parshley my fault. I should have made a distinction between backup and archive. at the moment I have a backup system albeit none of the backup is offsite. what I was looking for was some way to store all my digital pictures without any deteriasion of the data over time. I am very greatful for all the suggestions in my quest in preserving all my work.
just a side note I wrote this on my iPad with out the aid of a spellchecker so please forgive the spelling.

mike

[Bobobird]

Hi Howker,
unless I'm wrong mac software will not break up the data into a size that will fit to a dvd if it is larger than the disk itself.
Mike

There is a free Windows software called "TheNewFileOrder" that will break up a designated directory into DVD chunks. The latest version I could find was 4.1.1 but is in German. My much earlier version is in English but no longer works in Window 7 x64.

I do not use it much now and prefer shooting the stuff off to 2 different HDDs. Seems good enough for my purposes.

Regarding HDDs failure - I have had only 1 fail on me since 1979 - it was an IBM 75gig but was known to have hardware issues. It died twice, first time IBM replaced the circuit board, a couple of years it died again but by then capacities had increased substantially and there was not point getting it fixed again as it did not have any must keep stuff on it. I still have in working condition HDDs of 2 to 8 gigs.

[ErikJan]

I'm new here and posted the following questions in another (great!) forum (Eileen's Lounge, my post is here: http://www.eileenslounge.com/viewtop...p=86504#p86504) where it was suggested to visit and register here to see if any help would be available.

Over the last couple of years I've scanned all of my personal slides and pictures. Slides were done @ 4000dpi TIF and 48 bit color depth creating 128Mb files each... (I'm a perfectionist, can't help it ).

Next steps are to "fix" them (crop, rotate, scratch removal and contrast inprovements etc.). I'm now trying to find out which software I should use (thinking about buying Adobe Elements 11 or Lightroom 4 - the latter doesn't allow me to save my changes back to the original however, but IS 64 bits. Element is not 64 bit and I don't want to do anything 'creative'). Is there better / other software maybe (I'm an amateur, not a professional...)?

Then I intend to 'export' all fixed material to more 'normal sized' JPG versions.

Finally, I want to archive all my (changed or not) master TIF files (400+ Gb). Now it's on several portable HDs (of which I have backups in different locations). On the short term I trust this, mid to long term however I have more problems. Any severe magnetic impact any time will also erase all. I'm thinking of making another backup, but then optical (DVD / BD). I realize that's also short time and I might need to redo in 10 years or so. Using 100 DVDs is not too expensive nowadays (and I accept the hours it would take maybe) but then I'd need some software that takes all my folders and automatically creates 4.7Gb chunks and burns to DVD until all files are done... (only prompting me to insert the next DVD and creating a catalog on the fly so I can find stuff back on the -numbered of course- DVDs later). Does software like this exist? Is DVD a good option? Other ideas?

Wrt PhotoShop; I know what that is and that's WAY too complex. As said, all I need is cropping, removing scratches and working with white balances. I've played with Lightroom and it's perfect for the job [it can actually do a lot of things!] (but it does not allow me to save my edits back to the original...). An additional problem I did not mention is that I'm now using Picasa for 'managing' my collection and all pictures have face recognition enabled and assigned, I'm afraid I'll lose that anyway (but I did find a tool that writes the face-names in the metadata which can be picked back up by e.g. Lightroom (remaining problem is that in Picasa I know which face is which and in Lightroom I then only see which faces are there in the picture as a whole).

[ErikJan]

I'm now trying to find out which software I should use (thinking about buying Adobe Elements 11 or Lightroom 4 - the latter doesn't allow me to save my changes back to the original however, but IS 64 bits. Element is not 64 bit and I don't want to do anything 'creative').

Update, I installed a trial of Elements 11. First thing I did (spot healing failed)...


Guess that option falls off then...

[Scott Stephen]

I have Elements but have never used it.

As for Lightroom, you can always "Export" each file as a TIFF or JPG if you want to permanently capture the changes you have made, while also retaining the original.

You definitely seem to have a penchant for detail, so you would probably also love the ability to tag each of your photos with keywords (names of people, places, activities, etc.), as well as rate them too (with 1 - 5 stars, or with color codes). You can come up with a list of 10 or 20 or more "tags" with the names of people in the photo, places in the photo, locations of the photos, activities in the photos, etc... You can have multiple tags on each one, of course. Then when you want to search your big library for photos of a certain thing, and/or only the best examples, it is very easy to do.

[Glenn NK]

My personal experience:

1) When I converted to digital from slide film six years ago, I found that the technique I had been using for about 44 years was not quite up to par. Until a colour slide is projected onto a large screen with a high quality projector (what's that?), one will not know how accurately depth of field, lack of motion blur, shutter shake, and focus was achieved - viewing a 36 mm x 24 mm slide at life size won't reveal anything about its quality.

2) Shortly after I switched to digital, I acquired a tripod - again the improvement in quality was noticeable. I never used a tripod during my film days - how many people did? What is the IQ of 35 mm images that I shot without a tripod?

I would be afraid to digitize my slides for fear of finding out how bad they really were.

3) 128 MB TIFF files might be a bit of a stretch. My 21 MP 36mm x 24mm sensor will likely have better quality, and a typical RAW file from a sensor like this is in the order of 25 to 30 MB.

Back to the question: How to handle all these images:

(a) After doing all that work, I would not convert them to JPEG. Unless you are an expert post processor (I'm not), you will gain in skill, and once converted to JPEG, they can't be re-processed many times without loss of image quality. Use the TIFF files.

(b) I would recommend using Lightroom; the process is completely non-destructive; the originals will not be modified and (see (3) above), can be re-processed over and over without loss of IQ.

(c) If you haven't used Lightroom, then there will be a learning curve; during this process, skills and ability to process files will increase - use the TIFF files so they can be redone again and again. I'm a perfectionist too and I'm redoing my files from six years ago.

(d) Using Zerene Stacker, I convert my RAW files to 16 bit (it is recommended by Zerene). I import the finished 16 bit files into Lightroom with no problems at all.

(e) All of Scott's comments obviously apply, and these are major advantages of LR.

(f) Have all the files backed up on a separate HDD.

Glenn