Skip to content
May 26, 2011

Cloning a Failing Bootcamp-enabled Hard Drive

You can read this on my new site if you want:

http://www.jmblog.org/blog/2011/05/26/cloning-failing-bootcamp-enabled-hard-drive

One of the first things I did when I got my new MacBook Pro was replace the hard drive with a larger, faster drive. I ended up with a Seagate 7200.4 500 GB (ST9500420AS). I hadn’t had any problems with it, and it got hit pretty hard throughout the course of a week. I partitioned it as ~427 GB for Mac OS X and ~72 GB for Windows 7 Pro. The Windows 7 install is primarily for gaming, with some Windows 7 cross platform testing thrown in.

Recently, I had imaged a customer’s drive to a disk image on my Mac partition, which seemed to go fine. Unfortunately, when I tried to move this image to an external drive for cold storage, it would fail with an input/output error. I figured the most likely cause was a bad block, but that didn’t help me move that image. I ended up deleting it, since I had a separate copy of their data (I like redundancy). I left it at that for a while, since the drive’s SMART status was still OK.

Then I had another input/output error when trying to back up a Windows XP VMWare image. I installed the smartmontools using Homebrew, and checked the SMART status again, as well as doing a hard drive self test. The SMART status was still ok, though it was listing 88 bad blocks, which is supposedly over the threshold of 36. I’m not sure why that’s considered OK. The self test failed, however. So I ordered a new 500 GB Western Digital Scorpio Black (WD5000BPKT). Now the question was, how do I move all my data over?

I could have just reinstalled and pulled my files from Time Machine for the Mac OS side, but that takes forever. It would also mean that I would definitely not be able to recover my XP VMWare image, since I don’t back it up in Time Machine. Plus, I wanted to preserve my Windows 7 Bootcamp partition.

I read a bunch of tutorials online, and there isn’t really a nice simple way to do all of this. The two main camps were the SuperDuper! or Carbon Copy Cloner plus Winclone people and the ‘dd’ people. Both have their pluses and minuses – the GUI utilities are arguably easier to understand, the ‘dd’ method is (in theory) a one step process. Also, Winclone is unsupported at this point, as the company that made it no longer exists. You might still be able to find a copy, but I wasn’t sure I wanted to trust it. Also, I have used ‘dd’ before a few times, so I figured I’d give it a shot.

I put the new drive in a Firewire case, hooked it up, booted off of a USB Techtool Pro rescue disk, and ran the following command:

sudo dd bs=4096 conv=noerror if=/dev/disk0 of=/dev/disk2

To break that command down:

  • sudo – do this command as administrator
  • dd – the actual command being run
  • bs=4096 – read and write 4096 B at a time (4 KB)
  • conv=noerror – if there is an error reading, continue processing
  • if=/dev/disk0 – read data from disk0, which in my case is the internal drive (you can find out these names in Disk Utility if you don’t know what they are on your machine)
  • of=/dev/disk2 – write the data to disk2, which in my case is the new drive in the Firewire case

I let it go until it finished (2.5 or 3 hours or so), which seemed successful. Unfortunately, when I tried to boot off of it, it would crash at the boot screen. I could see the files in Mac OS, but at least some of them were corrupted. In retrospect, this method may have worked had I specified conv=noerror,sync as this should pad any areas where there was a read error with nothing, instead of skipping to the next block. I haven’t tested this, though.

I really just wanted to get back up and running and not be on a potentially failing drive as quickly as possible, so I used Carbon Copy Cloner to copy over my Mac OS X install to the new drive, so that I could at least get work done. I installed the new drive internally after I verified that it booted, put the old drive in the Firewire case, and was off and running Mac OS-wise. My XP VMWare image was lost, as a couple of parts of the virtual hard drive were corrupted, but I have an older Parallels one that I can convert. Luckily those were the only corrupted files.

I still really didn’t want to have to reinstall everything in Windows, though. So I started up the Boot Camp Assistant, repartitioned the drive the same way I had the old one, and used Disk Utility to copy the data from the old BOOTCAMP drive to the new one. I tried rebooting, and it didn’t show up in the boot menu, so I popped in the Windows 7 DVD and went into the repair area. It didn’t see my Windows install. I somehow managed to get to the screen of repair choices by using one of the options on the initial repair screen and then canceling, and went into the command line. I tried the bootrec command with /scanos, and it didn’t see my Windows install. I tried /fixmbr and /fixboot, which did make my drive show up as bootable, but it still wouldn’t boot, and /scanos still wouldn’t see the Windows install.

So I tried another tactic. I booted back into Mac OS, deleted the bootcamp partiton, recreated it, and did a new install of Windows. Once Windows was installed and running, I booted back into Mac OS, unmounted the new and old BOOTCAMP partitions, and ran the following command:

sudo dd bs=4096 if=/dev/disk2s3 of=/dev/disk0s3

This copied the Windows 7 install from the old partition in the Firewire drive to the new one on the internal drive. When I rebooted, it stated to boot but then errored out. I booted up the Windows 7 DVD again and ran the repair, which did see it as a Windows install this time, and proceeded to fix it automatically.

I booted back into it, and it worked perfectly. It didn’t ask to re-activate or anything!

So I am finally back to where I started, without having to reinstall and re-activate and re-download everything. It was a bit of work, but worth it overall. I would like to try the ‘dd’ method again with the ‘sync’ option to see if that was my point of failure, but I don’t have another drive laying around.

Leave a Comment
November 24, 2010

My weapon of choice: jEdit

jEdit Setup

My jEdit setup. I like lower contrast text, so I customized the zenburn Editor Scheme.

You can read this on my new site if you want:

http://www.jmblog.org/blog/2010/11/24/my-weapon-choice-jedit

jEdit is by far my favorite developer tool. I have tried every text editor I can get my hands on. Free, commercial, I’ve tried it all. Of course, I’m only looking at Mac compatible editors, since I’m usually coding on my MacBook Pro. I’ve been using jEdit on and off for years, first on Windows, then on Mac and Linux. There are always a few little things that bug me about it, so whenever I see another editor pop up, I try it out. Some have been close, but so far I’ve always come back to jEdit.

Here are some of the editors/IDEs I have tried:

I’m sure there have been others, but those are the ones that stick out. I could do a massive review of all of these editors, but I think I’ll stick with why jEdit is the one that sticks.

The first thing I do whenever I install jEdit is to make sure I pick the native Swing look & feel under “General Options -> Appearance” and restarting. I prefer a semi-native feel to the standard Swing Metal. Then I set a few things like changing the tabs to 4 spaces, getting rid of the wrap indicator, changing the font to Andele Mono, turning on line numbering, turning off the buffer switcher, turning on the search box and turning off backups on save. These are, of course, my preferences – to each their own – but the point is that it has the power to allow all of these nice little customizations.

Then the really important part comes. Plugins. There are plugins for just about everything that will make life much easier. First off, the general non-language specific ones.

  • BufferTabs – Uses a tab bar for switching buffers.
  • CandyFolds – Uses colors (that are customizable using regex) to indicate code blocks by tab level.
  • Color Chooser – Allows you to browse through colors in various ways to get their hex values.
  • Column Ruler – Numbers the text columns, indicates tab-stops, allows multiple column indicators.
  • DirtyGutter – Shows what lines have changed since last save.
  • Editor Scheme – Allows changing/saving the text area color scheme, including some nice presets.
  • ErrorList – Needed for a lot of the language parsers.
  • Finish Him! – Autocompletes words by parsing the current buffer.
  • Highlight – Allows highlighting the word under the cursor, or the selected word, etc.
  • JDiff Plugin – Awesome diff plugin.
  • MyDoggyPlugin – Better dockable manager.
  • Navigator – Allows you to go back to previous cursor points. Great for large files.
  • Optional – Puts all of the settings in one tabbed window.
  • Project Viewer – Allows the creation of projects. Great for quick access to all the files in your project.
  • SideKick – Needed for a lot of the language parsers.
  • TaskList – Parses the buffer looking for any TODO: or other task-based comments.
  • TextTools – Text manipulation tools. Allows re-ordering lines, etc.
  • WhiteSpace – Customization of the display and save-time actions relating to spaces/tabs.

That’s just a quick overvue of some of my favorites, but I certainly encourage looking at the rest. I particularly like Finish Him!, because unlike a lot of other autocompletion stuff, it just completes words, regardless of syntax, and it only does it when I use my assigned keystroke. Highlight is also great, as it can show you occurrences of a variable or other keyword throughout your code. JDiff is a really good Diff manager, I use it all the time. Project viewer is nice, too. A lot of other plugins integrate with it, like the SVN plugin for instance. Editor scheme is one of theose plugins that you sort of use once and then forget about, but it makes it easy to find a good starting point for your color scheme and then saving any customizations you do.

I mostly develop PHP using MySQL, CSS and JavaScript. Sometimes I’m working on XML/XSLT or Perl, or just doing some good old fashioned data processing. Tailoring jEdit to what you code is pretty much just a matter of choosing the plugins that go along with you language(s) of choice. The PHPParser plugin is amazing. Activate it, start coding some PHP, and hit save. It parses the buffer and highlights any syntax errors/warnings you have. It has saved me so many times. Similarly, there is PerlSidekick and many other language parsers. Some, like the XML plugin have other options like highlighting the opening/closing tag that goes with the one under your cursor, and automatically finishing the closing tag when you type ‘</’.

After all of this is set up to your liking, don’t forget to check out the built-in features, like code-folding and it’s excellent regex searching/replacing. The ability to split the text area in different ways is really handy for when you need to work on 2 or more areas in your code at once. Put your cursor just after a closing bracket in a block of code. Look at the gutter. See that line indicating where the code block started? Genius. It’s such a simple thing, but it makes coding easier. I don’t use them much, but there is also the Macro ability. You can do some really powerful stuff with the Macros.

And remember, just about everything is customizable. Don’t like the toolbar buttons? Change them to anything you want. Change where things are docked, or make them floating palettes, if that’s what you like. Change the keystrokes to whatever you want. Set up abbreviations that get expanded automatically. Change the context menu entries. I could go on and on, but I think you get the idea. If you can think, it, you can probably do it. Or if you know Java, you can make a plugin and contribute it. Don’t you love open source?

5 Comments
October 26, 2010

My version of the jQuery PNG fix

You can read this on my new site if you want:

http://www.jmblog.org/blog/2010/10/26/my-version-jquery-png-fix

Let me first say that I hate IE 6 with a passion. I’m fairly certain that it started as some sort of sick joke and it just spiralled out of control from there.

At any rate, I’ve been spending (more) time fixing IE 6 bugs in our current site redesign. Some layout issues here and there, some CSS transparency, and of course, transparent 24-bit PNGs. We’re using a bunch of those this time around. We’ve always had at least a couple, but it generally wasn’t worth running a JavaScript to fix them, since there were only a couple. So I would just hard code our fix of choice, the AlphaImageLoader.

For instance, the logo HTML would be something like this:

<div id="logo">
    <a href="/">
        <img id="logo-img" src="blank.gif" alt="" />
    </a>
</div>

The CSS for non-IE 6 browsers would be something like this:

#logo, #logo-img {
    width: 100px;
    height: 50px;
}
#logo {
    background: transparent url('logo.png') no-repeat 0 5px;
}

The IE 6 CSS would be:

#logo {
    background-image: none;
    filter:progid:DXImageTransform.Microsoft.AlphaImageLoader(enabled=true, sizingMethod=image src='logo.png');
}

This works great for a couple of images, but when you have a bunch, it’s not really sustainable. So I went in search of a good JavaScript solution. We use Drupal as our CMS, which has jQuery by default, so I figured a jQuery-based solution would be most efficient.

After a bit of searching around, everyone seemed to point to this one by Andreas Eberhard. But as pointed out by people on various forums and Q&A sites, it had some limitations. It has issues with hidden images in that if the image itself is hidden, it will be shown as a result of the process, and if the image’s container is hidden, the height and width will show up as 0px. One of the posts on the StackOverflow link above has a fix for the hidden images showing, but I didn’t find it to work very well.

The other problem is that it just hides the images that it is replacing, so if you happen to be using jQuery to show your hidden images at some point, you’ll have the processed and non-processed version showing.

So I hacked around with it a bit and came up with some fixes that seem to work pretty well for me. It now hides the unprocessed images and then removes them. It also sets the height and width based on the CSS values if they can’t be calculated. Finally, it detects the current ‘display:’ CSS value and preserves it if it’s anything but inline. If it’s inline, it still sets it to inline-block like the original script, but anything else will get set to it’s original value. So things hidden with ‘display: none’ will continue to be hidden, and things explicitly set to ‘display: block’ will still lay out correctly.

I also added the lines to invoke the fix at the end of the file, so you can just include the script on your page after jQuery, and you’re good to go. If you want to invoke conditionally, just take out the last 3 lines of code and invoke it wherever you need to.

There are still a couple of limitations. For one, I’m not taking into account other methods of hiding, just ‘display: none’. It could probably be adapted to do so, but since jQuery uses ‘display: none’ for all of its hiding anyway, it should work in most cases. The other issue is that background images that get processed will be scaled to fit whatever they are the background of. So you have to make sure that the size is set explicitly to make sure it doesn’t get stretched out of shape.

Well, enough explanation, here’s the file. Let me know if there are any more issues with it. It works great for my needs, but I’m sure there’s room for improvement.

Download Here

Leave a Comment
October 23, 2010

How (not?) to create a large, expandable storage/backup server

You can read this on my new site if you want:

http://www.jmblog.org/blog/2010/10/23/how-not-create-large-expandable-storagebackup-server

Backup to tape sucks. That’s why we started looking at large NAS devices to be able to back up all of our data to disk. It would be much easier and faster than our current tape jukeboxes that consistantly have hardware issues, or have tapes failing, or run out of room. The problem is, we have a lot of data, and we’d like to be able to keep a few versions of said data. We also have been burned enough times by RAID 5 that we wanted something more redundant. And, to top it all off, we’re on a budget.

So here’s a basic rundown of our requirements:

  • ~$1800 or less
  • ~6TB of usable storage space, but expandable
  • Can handle more than a single disk failure
  • Preferably not tied to particular hardware
  • Supports at least CIFS/SMB, NFS, SSH and preferably some RSYNC

It’s a fairly short list, true, but much harder than you would think to put together. Pre-made NAS devices just weren’t going to fit all of our criteria, so it was time to look at rolling our own. First, I found this case on sale for $300. This was the starting point I needed. It has 20(!) slots for drives, plenty of cooling fans, and we can put whatever hardware we want in it.

So I came up with a plan. I would get this case, a motherboard, power supply, etc. and use ZFS to create a nice large pool of storage that we can carve up as we need. ZFS is an amazing filesystem for large storage pools like this, but it’s only supported on a few OSes, none of which are my forté; basically some flavor of Solaris or FreeBSD. I decided to go with OpenSolaris, since it has the best native support for ZFS, and at the time (this was several months ago), arguably the largest community of the Solaris ports. I also briefly considered NexentaStor Community Edition, but it only allows up to 12TB, and this box has the potential for far more than that. Nexenta also has a Core Platform, but like BeleniX and SchilliX, there didn’t seem like any real advantage over plain OpenSolaris. FreeBSD seems to have pretty good support for ZFS, but setting it up as the filesystem for the root drive (thereby allowing you to mirror the root drive, yay, redundancy!) is difficult to say the least, and the learning curve of FreeBSD can be a bit much when coming from LinuxLand.

Why ZFS instead of a nice fast hardware RAID? Well, for one thing, price. For another, we didn’t want to be tied to a single type of RAID card. And most imortantly, because ZFS rocks! It has RAID built in, at single, double or triple parity. It can have as much space in one single pool as you can possibly throw at it. It has built in error checking and checksumming. It can compress files on the fly and de-dupe them at the same time. You can snapshot it to your heart’s content. It slices, it dices, etc., etc. It really is pretty amazing. I’m surprised Sun and Nexenta are the only companies that seem to be taking advantage of this.

I started shopping around for the rest of the stuff I would need to make this happen. Here’s the hardware I purchased out of the gate:

The idea was to hook 4 drives up to the motherboard, 4 up to the RAID card (which we wouldn’t actually use the RAID on) and 2 laptop drives we had hanging around up to the remaining 2 SATA ports on the motherboard for the OS. Then, when we needed more space, we could add another PCIe x1 card with 4 more SATA ports, add 4 more drives, rinse, repeat. I went with the HighPoint RAID card because budget constraints were such that a beefier card that could handle all of the SATA/SAS connections in one was out of the question. And I wasn’t having any luck finding a cheap board with at least 4 PCIe x4+ slots that was cheap enough, either. The Gigabyte board has 4 PCIe x1 slots, though, and it’s a really nice board.

Problem #1, the Promise breakout cables I got didn’t work. Little did I know I actually needed to have reverse breakout cables. So I returned the Promise cables and got new reverse (and cheaper) ones, plus an adapter to make it easier to plug in the cooling fans behind the drives, and some SATA power to Molex connector adapters:

So I got it all put together, and everything was up and running. Then the next problem reared it’s ugly head. OpenSolaris doesn’t support the HighPoint RAID card I bought. Not a great start so far.

I started looking for cards that were compatible with OpenSolaris, and would fit in the 4 PCIe x1 slots I had available. I came across This set of articles talking about doing something very similar to what I wanted to do. Would that I had found it prior to making my order. He had no luck finding a card that would work in OpenSolaris that was cheap and didn’t have RAID capability, so he ended up going with a very expensive hardware RAID card, and gave up on the dream of ZFS. I decided to use an OS that supported the card, instead. ZFS is supposedly fairly stable in FreeBSD, so I figured I’d go with that, instead.

I loaded up Freebsd 8, and find out that the card isn’t supported out of the box, you have to install HighPoint’s binary drivers. Unfortunately, there aren’t any drivers for FreeBSD 8. Argh!

I don’t really know FreeBSD very well, but I have used FreeNAS effectively for another NAS, and it’s based on FreeBSD 7.2, which is supported by HighPoint. So FreeNAS it is. FreeNAS uses version 6 of ZFS, which I wasn’t particularly crazy about, but it has a great interface, is easy to set up and maintain, and allows us to use the hardware we already have.

Set up was a breeze, getting the drivers up and running wasn’t a problem, but in order to get the drives hooked up to the RAID card to show up I had to make each disk it’s own JBOD. ZFS likes to have access to the raw drives, but it didn’t seem to hurt performance much, and it worked, which is more than I can say for previous attempts. I was pretty happy with the results. I had a big storage server with a good web interface that fit all the criteria.

One oddity that came up was that the S.M.A.R.T. status of the drives hooked up to the RAID card were unavailable. Not a big deal, but it should have been and indicator of what was to come. I started testing to see what would happen in the event of drive failures. I pulled a drive from the 4 hooked up to the motherboard, and a drive from th 4 hooked up to the RAID card. The one hooked up to the mainboard showed up as missing, but the RAID attached drive did not. Not good. Would the OS keep trying to write data to the missing drive? I have no idea. And I wasn’t about to find out.

The search for my holy grail continued. The RAID card wasn’t going to work. I needed a PCIe x1 card that could handle 4 SATA connections, and didn’t have a RAID. I searched high and low, and finally found the SYBA SY-PEX40008 at mwave. I was a little worried as the reviews on newegg were not very flattering, and it has RAID capability, but it was only $50 and I was desperate. I also found this page, on which a Mr. Stephen Green described having pretty good luck with it as a non-RAID card. Under OpenSolaris, nonetheless!

I ordered the card and came full circle back to OpenSolaris. I installed the latest stable version, 2009.06, which went very smooth. There was plenty of help online to get me going on an unfamiliar OS. Now I had native ZFS for both the root drives as a mirror and the storage drives as RAIDZ2. I can create a dataset in the pool with one command. I can add NFS or CIFS/SMB sharing to that dataset at the same time, no problem. It’s great!

Except for the fact that the SMB server keeps hanging. What. The. Hell?

I started searching online, and it seems I’m far from the only one experienceing this problem. At least one person seemed to think that it was triggered when you tried to browse the share using Windows Explorer. Not that anyone would ever browse a SMB share using Windows Explorer. People were saying it seemed to be fixed in later versions of the SMB server, but you can’t just install the new version as it’s part of the dev version of OpenSolaris that will become 2010.03 (which apparently will be at least 2010.04 at this point).

I installed the dev version. It wouldn’t be my first choice, but SMB was unusable as it was, and that made the whole server somewhat less than usable. The install itself went fine. I love how it creates new Boot Enviroments when you do a full upgrade. It makes it much easier to go back if things go awry. Did I say ‘if?’ Oh, I meant ‘when.’ It turns out there are a bunch of caveats when doing an upgrade to the dev version. Like not being able to use the command line at all, etc. After a little searching I found a whole page dedicated to fixing the little issues that crop up after the upgrade. Luckily, all the fixes Just Worked™ and I was up and running.

We moved a bunch of files over to the new box and started sharing them through SMB, but some of the files were coming up as missing, even though they were definitely there. I banged my head against permissions and ACLs and such to no avail. Then I noticed that the file that was being looked for was mixed case on the OpenSolaris server and lowercase on the Windows box. Turns out the best way to create a dataset for SMB sharing isn’t the default. After following the advice on this page, we were officially up and running.

Or so I thought. The pain continued. Everything works, but mysteriously the machine will just hang when making data transfers through SMB. And, worse, OpenSolaris has been killed by Oracle. So where to go from here? My best 2 options are Nexenta Core and OpenIndiana. At the moment, I’m leaning toward OpenIndiana, for ease of installation and minimal reconfiguration. I basically just upgrade my current OpenSolaris install. I’m going to keep my eye on it for a while before making my decision, though. I think the future of whatever OS I go with will be the determining factor here.

Here are some the most helpful OpenSolaris pages not mentioned above that I found:

Leave a Comment