Consolidating photo libraries from multiple drives

I’m trying out “The Big Mean Folder Machine” version 2.29 to consolidate (or “merge”) various years of photo cruft.  The process of culling and consolidation will hopefully allow me to have a single photo directory backed up to Amazon Glacial forever.  I believe as I mentioned in another post, mechanical disks fail so having a redundant and offsite backup strategy is crucial.  And certainly a key part of of this is limiting the number of places you’re storing your local data so that you can create redundancy.

I’ll write more here when I get all this data and drives sorted out, but so far I was able to run a scan and it the BMFM was able to create a folder hierarchy from multiple drives.   I may need to do some merges first and then a split.

These things fail.

Drives. Negatives. All of it. It expires. I’ve been trying to come up with a way to implement simpler, more efficient means to organize and backup important items. Everyone wants a houseboy. They should be exceptionally talented in the mysteries of the Fujitsu ScanSnap iX500, rapidly converting their heretofore analog paper trail into an endless cloud sync’d searchable PDF. Amazon Glacial. Google+ and Picasa auto photo backup. Flickr uploader does not work well. It is slow and tedious. There is this feeling of a freight train rolling in the darkness, never stopping. Go. Get outside. Run. Try to outrun it.

Thom Yorke: Bok Bok & Roska Remix of Modeselektor’s ‘Art & Cash’

Thom Yorke - The Twist (Unreleased Excerpt) + Art&Cash (PHON.O Rmx)http://www.youtube.com/watch?v=dPiA8-qekRo

C liked this. So did I. Pretty killer. Here’s the full mix streaming/download, with some glitches during the playback presumably to limit reuse (full link: Electronic Battle Weapons Xfm Music: Response: Thom Yorke MoneyBack Mix 2011.09.15).

More Modeselektor,’Art & Cash’:

Modeselektor - Art & Cash
http://www.youtube.com/watch?v=7wM0oz5rlHE

On a totally different note, if you use Google Reader it’s never a bad idea to backup your RSS feed to an OPML / XML file, you know, just in case they pull that plug.

Follow Up, 212+ Installation

C was here for a few days. Our last project was getting the Cooler Master Hyper 212+ installed. Installation went pretty well. We likely used too much thermal paste, but nonetheless, it reduced my CPU temperatures by about 20 degrees Celsius, so I’d say it works. It can often be found on sale for $25 from Newegg, so really if you have a large enough computer that’s running hot, this seems like a no brainer. I’m not entirely certain the CPU temperature readings were always correct, but I figure it’s better to err on the side of over cooling.

The one trifling problem I’ve had in my “Almost Vanilla” *OS*X*86 install was not achieving a full power off after shutting down. I found many threads related to power & sleep issues, but I finally found the one thread here referencing the specific settings for the BIOS on my Gigabyte G41M-ES2L motherboard. The answer was this:

Power Management Setup:
ACPI Suspend Type: [S3(STR)]
Soft-Off by PWR-BTTN: [Instant-Off]
PME Event Wake Up: [Disabled]
Power On by Ring: [Disabled]
Resume by Alarm: [Disabled]
HPET Support: [Enabled]
HPET Mode: [32-bit Mode]
Power On By Mouse: [Disabled]
Power On By Keyboard: [Disabled]
AC Back Function: [Soft-Off]

Although it turns out HPET Mode: [64-Bit Mode] works just as well. The crux was the three options below Soft-Off by PWR-BTTN, those three all must be disabled. Now when it turns off the case isn’t consuming 90 watts. Which is still a lot for any computer, but I try to only use it as a part time server for media backup. It consumes 5 watts in its “OFF” state but at least now it shuts down fully.

At least…

The prices on high quality heat sink fans have come down. I recently converted the old Intel e5200 htpc/server into an “OSX”86 server. It turns out that you can now fill the 4 SATA drive slots with about 12TB of storage now for like $100 (I’m joking, but storage is cheap). Right now it only has a single “green” Seagate 1TB drive, but at least I don’t feel constrained any longer. In theory I could’ve continued running 10.04 Ubuntu and Netatalk, but I figured it was worth trying to get a native AFP server going. Lifehacker had a really easy to follow post and it’s pretty quick now as a 1GigE backup/media server (my original build used “compatible” hardware). Faster than the 4 year old Time Capsule when transferring large files, that’s for sure.

But yeah, Intel’s stock coolers, at least in the LGA 775 chipset models appear to be terrible. The 212+ cooler requires a backplate, so I’ll have to pull the motherboard, but once this thing is installed I should be able to run Handbrake 24 hours a day and hopefully the CPU won’t go above 50C. Currently this thing idles at 57C, as the stock heat sink just doesn’t seem to mount flush.

Cooler Master - 212 Plus Cooler

Cooler Master - 212 Plus Cooler

Bottom heat-pipes, direct contact

Bottom heat-pipes, direct contact

Automated Ripping Potential & that Vintage USB Typewriter

Went to Hive 76 for their open house night on Wednesday to check out the space.  While I was there Jack Zylkin demoed his very cool Arduino based vintage typewriter > USB Keyboard hack, and it’s actually quite ingenious and cool.  There are contact relays underneath the main typewriter carriage and he uses magnets for other registers and the Arduino chip figures out the characters pressed based on time delay.  And he’s made the plans available for ‘from scratch’ DIY types under a Creative Commons license.  & who doesn’t want to carry around a 50 pound vintage type writer with their iPad?  Jack’s website is usbtypewriter.com and his Etsy page [here].

Another sweet item I saw at Hive 76 was an older Sony Vaio automated DVD changer that connects via 1394 (FireWire 400).  Supposedly, using DBPowerAmp and some basic scripts it is possible to batch rip up to 200 CD’s at a time into .flac image files with good metadata and .cue sheets.  Alex Wetmore wrote several years ago, and I’m paraphrasing his sentiment here, that he had better things he’d like to do with his day, like go on bicycle rides, because the fact is that he is not a CD changer.

So the general idea is you use one of these big Sony Vaio XL1B* changers, load it up with your music, walk away from it for about 24 hours and when you come back hopefully you’ve got a hard drive full of music in a format that is future proof.  If I can actually get this to work it would be a beautiful thing, and Brendan said it was good go.

Sony VAIO XL1B* Series 200 Disc Changer

Sony VAIO XL1B* Series 200 Disc Changer

One Stop Ripping All Your FLAC Audio Needs

Thought I’d throw out a quick post while I was doing some searching for “best practices” in FLAC image extraction of audio CD’s for backup purposes.   Basically follow this guy’s instructions:

http://hiphopiscoolagain.com/jiggafellz-step-by-step-guide-to-secure-cd-ripping-with-exact-audio-copy/

In EAC you’ll click the “Actions > Extract Image + Cue Sheet > Compressed” function using your basic FLAC settings.  Also be sure to enable Accurate Rip and do the automatic detection on all the drive settings.  Save a .log file.  There should be three files upon final output: the FLAC image, the CUE sheet, and the .LOG.  That’s it.  Simple is as simple does.  If you feel like embedding the CUE sheet in the FLAC you can use Foobar2000 to do such, know however that it’s non-standard as far as burning back out.  Apparently it’s always handy to have a separate CUE sheet.  Good to know.

UPDATE: So, two things I realized.  One: XLD (X Lossless Decoder) for OS X does pretty much the same thing as EAC.  However, if you’re still hell bent on using EAC and XP (couldn’t get EAC to go under Darwine) it’s important to know that it won’t easily embed cuesheets (.cue sheets) automatically as XLD does.  Embedded cuesheets do make life a little easier as your playback software can look at either the .cue or the .flac file and see all the tracks from the single disc rip.

To embed a .cue using Foobar2000 right click on the .flac image file select “Utils > Edit cuesheet” a window will pop up, check “Embed cuesheet” and then “Load .cue file” from your extraction directory, click “OK” to save.  Bingo bango.  XLD can do this automatically.  I really cannot say which is better per data integrity, YMMV.  Obviously if you’re running Windows EAC is a “reference” standard.

Album art is another grey area as either the %Album Name%.jpg or simply “cover.jpg” can be used inside the album folder.  Also, what about album dates?  Certainly, we can agree that the album’s original release date is the important date here in metadata, and per genre conventions I tend to trust MusicBrainz.  For a good read I really liked Daniel Stout’s article about simplifying down to 25 main “top level” genres and tagging comments/notes metadata with sub-genres.  The moral here is you should read up as much as you can.  Either EAC or XLD will do fine with rips to backup your audio CDs.  Cover art and sub-genre notes I’m going to have to keep exploring.

Exact Audio Copy

Exact Audio Copy

WWDC no 802.11n in on the 3G S iPhone!? Drat!

So, with the new iPhone 3.x OS you will not be purchasing an 802.11N enabled wireless chipset.  Apparently the new 3G S model does support 4G HSDPA but not 5 GHz 802.11N wireless with the new lower powered Broadcom BCM4325. Apparently the 3G S model is a bit snappier. When is the 802.11n iPod Touch coming out? That’s what I’m waiting for. I saw that Garret had a very small LG phone that’s basically free from Verizon, if I could find that used/new and then go with a Touch remote… we’ll see how it goes…. Windows 7 RC is OK. It works, for what it’s worth.

I like Ubuntu 9.04 with MythTV…. I’m going to go back to that but probably do a dual boot leaving Win7 RC just in case…. Sadly, for Netflix playback due to Silverlight DRM a Windows XP/Vista/7 VM or even an OS X VM is necessary for Netflix streaming…. or buy a Roku box, unfortunately our TV doesn’t have 2 HDMI inputs… I’m surprised no one has been able to reverse engineer the Roku Box’s chipset and put the Silverlight DRM code out there…. Or even just have a “Roku Box VM”…. Why not, right? Seems possible, when you consider how almost all old video console games can now be played and fit on one flash drive.

I brewed with the Yama 5 Cup Vacuum brewer today. Thank you Conor and Leigh, awesome gift BTW; best of luck in your trip West. The coffee is very good, very clean, very smooth. I currently am grinding Gimme’ Coffee’s Picolo Mondo variety. Thank you Japanese vacuum brewing technology and to Chris for the awesome vintage German Peter Dienes grinder which does it job remarkably well for a hand grinder over twice as old as I am.

Yama Brewing

Yama Vaccuum Brewing by Digital Colony

My Vintage PeDe looks similar, all metal on top though

My Vintage PeDe looks similar, all metal on top though

Hackable?

Hackable? Image from ehomeupgrade.com blog

Still Not Ice Climbing – but lets build a MythTV!

Sadly my Saturdays and Sundays have not been filled with ice climbing trips to hidden frozen waterfalls of the Northeast.  My hand feels better despite having a small gash where it hit the edge of the BMW car door. I imagine I’ll be able to go back to the MPHC climbing gym soon.  

I miss the outdoor rock.  Ryan, Josh and I went on a few climbing trips before it got cold and before weddings and Texas.  Ryan has photos of us climbing in the Gunks and Brewster, NY available here: http://picasaweb.google.com/ryanwesleywebb these are just a sampling.

 

Peterskill - Mark, Josh

Peterskill - Mark, Josh

Ryan in Peterskill

Ryan in Peterskill

Ryan, Ice Pond

Ryan, Ice Pond

 

Mark's hand Ice Pond

Mark's hand, Ice Pond

 

 

 

 

 

 

 

 

 

 

 

 

 

 

I spent most of yesterday reading about MythTV.  If you feel like setting up an open source home DVR (TiVo like) then Mythbuntu or KnoppMyth seem to be a couple good stable options for creating a home Linux TV recording solution.  Obviously this will only work if your signal is “free and clear”. Dish, DirecTV, ComCast, Time Warner and basically any cable or satellite provider, encrypts all premium content.  So on the one hand, local network channels and basic cable should be “free and clear” (over digital cable local channels are usually free using QAM) but with satellite it’s more likely you’ll need to rent the DVR from them.  The only reason I’ve been considering it is because I’ve needed some sort of iTunes server back-end (see my Firefly MT-DAAPD post) and it might as well DVR network HD shows we want to watch.  Also, our ATSC tuner is old (2nd or 3rd generation) and I think the latest 5th or 6th gen tuner chipset should allow us to get all the local HD broadcast channels without fussing with the antenna. 

There are three parts to most MythTV setups, which could all be integrated into one system or distributed into three smaller systems:

  • The back-end is your dedicated server which should have low power and processing requirements.  I’ve been looking at the MSI Wind ‘Nettop barebones PC (MSI’s product page) which sells on Newegg for like $139.99.  It uses an intel Atom 1.6GHz processor with minimal power draw, I think around 35 Watts max, but you could probably optimize it to use somewhat less than this.  Obviously, HDTV content takes up a fair amount of disk space, so for starters lets say a 1.0 Terrabyte hard disk drive.
  • The actual TV tuner (which now ‘a days will need to be digital ATSC HDTV compatible, unless you still have analog cable) could be as simple as a PCI card in the back-end but something even cooler is the SiliconDust HDHomeRun.  This product is two ATSC tuners in one.  It transfers HD broadcasts from over-the-air (or QAM) to your server via ethernet.  It requires DHCP to obtain an IP address, but otherwise it’s just a little box with one 10/100 ethernet jack and two antenna inputs; ATSC to IP as it were. Your back-end will recognize it as two tuners in the setup. You could watch one while the other records or both could record simultaneously.
  • The front-end will be how you actually watch the content you have archived. Say you ripped all your DVD’s and you’ve been time-shifting several seasons of Top Chef, the “front-end” is what will do the heavy lifting of playback depending on the bit rate and resolution of the video.  It should look just like any other media playback menu.  From my reading, MPEG-2 is the optimal format to save content to (thankfully DVD and ATSC signals are already MPEG-2 so no transcoding is required) and MPEG-4/h.264 require more horsepower on your machine to playback smoothly.  My MacBook Pro Core Duo 2.0 GHz would be a fine front-end however I’d need to plug it in to watch shows.  I’d be able to watch live TV wirelessly on the laptop anywhere in the house though. Note: the Elgato EyeTV is another option for Mac OS X HDTV time shifting however it doesn’t have the front-end/back-end model, your laptop has to be on and plugged into the tuner to record.
Yup.  So that’s where I’m at.  I’ve already wired the 1GigE cable to the bedroom.  I still need to do a cable run to the kitchen, but that’s not really a priority.  I think having a small machine like the Wind PC in a closet would be a nice way to have both an iTunes server and a DVR.  Apparently MythWeb allows you to easily setup recording from your web browser, so you just login and set the shows you want to watch for the week and let the back-end do its job.  There are ways to “Sling” content to yourself if you’re out of town or something, but personally I think I could wait until I’m home, and besides most of the content by the big networks is instantly available online now for free viewing.

Special Day Photo Secrets

So, we’ve learned the hard way that when you’re about to get married you should reformat your camera’s memory card.  Reformat it three times, actually.  Wipe it clean, freshy frischmacher clean.  The Canon PowerShot SD800 IS had really never given us any major trouble, it’s been a good little point and shoot.  But as we found out the hard way (after losing I think about 40% of our wedding day photos to bad writes/corruption) you should alway reformat your flash memory cards after dumping the photos onto your computer.  I attribute a lot of it to the fact that we’d kept some photos on the card but deleted others, and I think it’s likely iPhoto was writing to the card somehow screwing with FAT16 system.  The moral is this: download all your photos then reformat your card “in camera” using the camera’s menu setup, each and every time.

Thankfully we had a friend shoot two rolls of film on an old fully manual Minolta SRT 101 35mm camera and a bunch of those came out, so it’s all good.  And we could always reshoot it once it’s not 10 degree outside and snowing.  Or we could do one month anniversary photos.  

The best software I found for free file/image recovery was Zero Assumption Recovery. The other software, that seems to be the best for the 2GB SanDisk SD card was Rescue Pro by LC Technology in the UK.  ZAR gave me similar results though.  I also tried PhotoRescue, however it did not recover as many photos as the other two.

Here are the best pics from two rolls of film: A Wedding on Film

And here are the best digital pictures we saved: Zoe & Mark Get Married

More gauzy flare

Excited

Kissing (in front of fountain)

Flowers and dress

Future computers, cyclo-cross and dreams

Apparently Apple is getting back into the Chip manufacturing game again.  We can only hope as x86 PC users go, that they don’t abandon Intel’s architecture completely in the near future so that we can still dual boot and run Virtual Machines like Parallels and boot alternatives like Ubuntu.  What is more likely, if we’re going to play the speculation game, utilizing PA Semi Apple plans to continue to push the envelope with mobile computing implementing low power parallel processors to the point where Intel chips are not relevant.  This seems possible.  If you consider the computing shift from massive mainframes, to workstations and now laptops, the future of computing is largely OS and processor agnostic.  The excitement over so called ‘Netbooks isn’t just that you can “hack” a $400 MSI Wind Laptop and make it run OS X Leopard it’s that the model is shifting from proprietary systems and dedicated hardware moving computing straight into the ether.  Which begs the point that the iPhone is only the beginning of Apple’s mobile computing strategy and that in 10 years time the current generation of MacBook’s with their svelte machined aluminum uni-body enclosures will seem elephantine.

For the record, I did get the 7200 RPM Seagate 320GB hard drive to fit the MBP and upgraded the Airport mini PCI card to 802.11n Extreme.   It goes  pretty well over the Time Capsule for backup with Leopard’s Time Machine.  Yes, I upgraded to Leopard too.  And it boots way way faster than Tiger ever did for whatever reason (not just the 7200 RPMs).  I slip-streamed XP Service Pack 3 and then did the whole Boot Camp installation on a 100 GB partition, SolidWorks Student Edition worked fine which was something of a relief for me.

Also, I have a new 2009 Trek XO2 frame to ride off the pavement of New York.  There are a few more ‘cross races this season and I’m not sure if I’ll be able to make them all as they tend to be 1-2 hours away in Jersey and it’s like $30 per race.  My hopes are to ride the Croatan Aqueduct trail, do the mountain bike loop in High Bridge Park in Harlem, and perhaps take Metro North and ride some of the trails in the Hudson River Valley.  There is a race weekend in the Hampton’s and with entry to a citizen’s race your number is entered to the raffle for a Richard Sachs’ custom cyclo-cross bicycle complete with full SRAM kit (so as to skip the 7 year waiting list and selling of kidney).

My new ride

My new ride the Trek XO2


Richard Sach's Cyclo-Cross

Richard Sachs - The best of handmade steel

halfway

I’m about halfway there on the upgrade of the larger notebook harddrive and 10.5.4 with XP BootCamp (all of this is to get SolidWorks 2008 Education version running, mind you).  Leopard is noticeably faster than my old run down version of Tiger.  Or at least that’s what they’d have you believe.  No seriously, this system is running snappier with the fresh install. I’ll still need to migrate all my iPhoto stuff, as the version of Leopard I bought didn’t come with iLife.  I’ll need my address book and some bookmarks but that’s about all I need to migrate.  It’s pretty straightforward.  I did create a bootable external drive backup of Tiger in case something goes wrong using, Carbon Copy Cloner.  That worked pretty well too. You just hold down the (alt) option key at startup and choose the firewire disk you want to boot from.

Currently the TimeCapsule is backing up this fresh install.  The TC 500GB drive is a little loud when writing even though its encased in white plastic.  We were able to play a DVD video_TS file over 802.11n the other night to the TV.  As a note, you should turn off Time Machine automatic backup when you’re planning on watching something on your laptop because it’ll keep trying to backup while trying to stream the video and it’ll get choppy.  Lesson learned.  Under XP it said we’d connected to the router at 128 MB/s.  So yeah, this stuff seems to be working alright.  Another nice feature of Leopard is in the Dictionary Application it goes to Wikipedia as the default.  Pretty cool.