Bad Sectors? Low-Level Format

It seems like there IS a way to “clear” bad sectors from your hard drive so you can use tools like GParted and the like – but I use “clear” in a very loose sense here!

First off, I MUST point out that I’m talking about file-system bad sectors. I’m NOT talking about physically damaged disk platters.

It seems NTFS keeps a list of bad sectors, and as long as those sectors are there, most partition-resizing tools will refuse to touch the disk with a 10 foot pole. HOWEVER… those of you who are beyond a certain age might remember something called a “low-level format.” (I’ll wait a moment for you while the moment of nostalgia passes.)

I thought I’d never see a need for low-level formatting in today’s world of super-reliable, super-fast, super-S.M.A.R.T, super-big hard drives – but it seems there is still one use for it.

The hard drive manufacturer’s low-level formatting utility will detect “bad” sectors and put them in the drive’s internal list of “bad” sectors – this is in the drive’s own firmware mind you, not in any file system structure (because at this point, your file system has been wiped out!).

Once this is done, the drive’s own controller will silently avoid those bad sectors – from any software’s point of view, those sectors or clusters just don’t exist anymore. (Since software rarely – if ever – directly addresses the disk, this sort of behind-the-curtain hiding of sectors or clusters is easily done by the drive’s on-board controller.)

After the low-level format, you can partition & format the drive normally, and your OS or whatever disk-checking tool you use should find a nice, clean disk with no errors.

Astute readers will note the downside to this “solution” – you have to low-level format your hard drive! Obviously this erases everything on it, without any possibility for recovery. So it’s not for the faint-of-heart.

And, coincidentally, it’s not for me, either. I’ve simply got too much data on my 2nd hard drive to back it up easily (and cheaply). And doing this to my primary hard drive (and thus being forced to re-install Windows) is simply out of the question.

For those that are interested, I found a lot of this information in this EASEUS Software Forum posting, while looking for the reason why (you guessed it) their software wouldn’t re-size my partition.

In the end, it looks like I’m stuck running the Windows 7 beta in a virtual machine – which is of course super-slow.

Maybe someday I can shell out the cash for an external hard drive so I can back up my data and do the low-level format… but until then… I’m stuck with my partitions the way they are. Bummer!!

Interested readers may want to catch up on the previous entries in this saga:

GParted, Why Hast Thou Forsaken Me?

UPDATE March 16 2012 – there is finally a solution to this problem – read the updates at the end of this article for details.


Longtime readers will remember the computer drama that unfolded back in May of 2008 when I bought my new 500 GB hard drive. Basically, when I tried to resize my NTFS partitions, GParted (or, more specifically, the tool ntfsresize) said “you’ve got bad sectors on your disk, I’m not touching anything” and refused to run. I eventually lost the battle and settled for a (less than ideal) configuration for my new disk.

Fast-forward to today – the public beta of Windows 7 is meant to come out today, and although there’s always the virtual machine route, for something like this I thought it might be worthwhile to dual-boot so it could have the full power of my machine (Vista under Virtual PC is OK, but sloooooow – imagine what a beta version of an even newer OS would be like?).

Since Microsoft’s web servers seem to be totally overwhelmed with the demand for Windows 7, they’ve taken the download off-line for the time being. So, given that it seems like I might have to wait a bit, I figured I might as well download the latest GParted LiveCD (version 0.4.1-2 in case you’re curious) and make a little partition space available.

As you might imagine from the title of this post, I wasn’t able to do it. It was the same problem as before – there were “bad” sectors on the disk, so ntfsresize refused to do anything. Oh, sure, I could have tried to do it from the command line and used the --bad-sectors option of ntfsresize, but I’m not one for specifying disk sizes based on cluster locations, which is how you’d have to do it. And I couldn’t even get the parameters that the GUI interface would have used when resizing, because it wouldn’t even let me try to resize under the GUI to get the parameters!

After all, what’s the point of a GUI LiveCD if you can’t use the GUI!

What’s even more confusing/infuriating is that every forum post or help document I could find on this subject seems to go like this:

Novice User: I’m getting this “bad sector” warning – but I ran CHKDSK two times like it said, it found no bad sectors (or it found one and repaired it), but I still can’t run ntfsresize!

Linux Guru: Your disk is failing, get a new one.

Novice User: But…

Linux Guru: YOUR DISK IS FAILING AND WILL DIE SOON GET A NEW ONE IT’S NOT A PROBLEM WITH NTFSRESIZE YOU JUST NEED A NEW HARD DRIVE DON’T QUESTION ME.

I have, of course, embellished this somewhat for dramatic effect, but you get the idea. The assumption is always that the problem is with the hardware. (Even the Wikipedia article on ntfsresize seems to support this point of view, although in fairness it is marked as [citation needed]!) I can understand a certain amount of healthy skepticism (in many cases bad sectors are a legitimate sign of a dying hard drive), but there seems to be a certain amount of… I don’t know, denial – for lack of a better word – going on with respect to bad sectors that aren’t actually all that “bad.”

If your hard drive develops a bad sector (as a result of a sudden power loss situation, for example – which is what caused my bad sectors), CHKDSK for Windows will find the problem, recover what data it can, and mark the sector as “bad” so it doesn’t get used again. Barring another power loss situation (fixed for me since I now have a UPS for my computer), the drive should be just fine for the rest of its natural life.

Likewise with defects that were detected by the manufacturer (but not bad enough to reject the disk over) and marked as “bad” before they left the factory. I know they used to do this with disks – I assume they still do.

So, it certainly seems like there are situations (possibly quite common) where a drive will have “bad” sectors, but should still be safe to use and operate on. After all, the fact that they were marked “bad” means that they were detected and won’t be used again.

Of course, I do have to mention again that ntfsresize has a --bad-sectors option which will ignore bad sectors… but there’s no way to access this option from the GParted GUI. You HAVE to do it from the command line – which kind of defeats the point. The whole wonderful thing about GParted in the first place is that it’s a graphical way to resize your disk partitions. I’m a geek, but even I don’t like to have to specify partition sizes by sector/cylinder offsets.

So, if you were to ask me, I’d say that either:

  • ntfsresize should be a little more forgiving regarding bad sectors; or
  • The GParted UI should detect the ntfsresize error and allow the user to proceed with the --bad-sectors option, obviously with a stern warning about making sure you’ve checked the disk thoughougly with CHKDSK or some other tool to make sure there are no further problems and the drive isn’t failing even more.

I know this post ended up being sort of just me whining – but I do wish that I could resize my disks with GParted. I like GParted – I think it’s a great tool – but, unfortunately, it’s just not a tool I can use anymore.

If anyone has any tips or ideas on how I might be able to work around this limitation – or of perhaps another partition-resizing tool (free and/or open source preferred, of course!), I’d love to hear from you – feel free to speak up in the comments!

UPDATE: I’ve found a program called EASEUS Partition Manager – it’s free for the “home” edition. (But not open-source, unfortunately.) I’ll update with a new post if I’m able to resize my partition, but first I need to do a full backup, just in case!

UPDATE 2: Follow up article here – the news isn’t good, I’m afraid.

UPDATE 3: A very nice reader emailed me to point me to this solution from Unfinished Teleporter: Gparted won’t shrink an NTFS partition with a bad sector.

Basically, you create an executable bash script which adds the --bad-sectors option to the original ntfsresize program. You can see the steps in the article I linked above, or follow these general steps:

  1. Locate the ntfsresize executable (for the GParted live CD it will be in /usr/bin, for Parted Magic it is in /usr/sbin).
  2. Rename it to ntfsresize.orig.
  3. Create a new bash script at the same location named ntfsresize. (See below for what you should put in this script.)
  4. Use chmod to ensure the new script is executable (chmod 755 will do the trick).
  5. Run GParted as normal.  It will ignore the bad sector(s).

In the bash script itself, you’ll want to add these two lines:

#!/bin/bash
exec ntfsresize.orig --bad-sectors "$@"

This will call the original ntfsresize that you renamed, tack on the --bad-sectors option, and then pass whatever other options GParted (or whatever you are using) called it with originally.

I haven’t been able to try this myself (I’ve long since upgraded to another, newer computer), but I’ve been told this works like a charm. So if you are having this same sort of problem, give this a try!

Of course, before doing this you should be sure you’ve backed up all your data, since this is technically overriding a fail-safe feature of ntfsresize.

Anyway, I hope someone finds this useful – and thanks again to that very nice reader who pointed out this solution to me!

Don’t Reboot – Just Logoff

A while ago I wrote about how Windows still works just fine for me, and I wanted to follow up on that.

Today I was having some problems opening new windows – from any program. Being a programmer myself and knowing a thing or two about how Windows works on the inside, I suspected that some program had perhaps used up a bunch of “handles” and now there weren’t any more to create new windows with. Being in the middle of work, I didn’t really want to spend time debugging the problem (who does?) but I’d had a lot of big programs open lately – it wasn’t unreasonable to think that one of them had some sort of memory leak.

For many people in this same situation, they would reboot their computer to “fix” the problem. But I don’t like doing that – for one thing, it takes a while to reboot!

Also, I’m sort of proud of my “uptime.” (You can find programs all over the place that will tell you how long your system has been up & running for since it’s last reboot – this time is often known as “uptime.”) As of right now, my uptime is 11 days 23 hours and 20 minutes. Sure, it’s not the months (or years) of uptime that a Linux box might have… but it’s not bad! (For Windows!)

So, I didn’t want to reboot, but I needed to free the handles (or memory or whatever had leaked) from whatever program had been “bad.” What to do? Just log off!

A lot of people forget that logging off does a lot of the same good as a reboot. Unless you suspect your problem is in your system’s kernel resources (or in an always-on system service or something), a log off is often all you need to do to get things working smoothly again. (Note that this won’t work if you’ve just done an installation of a new program and it requires a reboot – that sort of thing really does require a full reboot to replace in-use files.)

I did that myself, and sure enough – some system resources were freed up. So maybe later I’ll get around to tracking down what program leaked them – but for now, I’m just happy to be running smoothly.

So, a tip for the impatient – instead of rebooting, try just logging off and logging back on to Windows. It might do the trick for you!

Power button image courtesy of the Crystal Icon Set.

Audio Feedback from Computers

There are some subtle audio clues that I use to tell if my computer is doing something – mostly, whether I can hear the hard drive churning or the CPU fan whirring up to a higher RPM – these tell me that something is going on.

When I’m remote controlling computers, it’s a bit disturbing to have absolutely no audio feedback to my actions. Often times I have to ask the person on the other end “is anything happening?” because without sound, I can’t tell what’s going on.

I suppose I just find it interesting how much I rely on sound to tell me what the computer is doing. Something future computer designers should take into account!

Books

The other day, I decided to go to my local bookstore and look for some computer books. I’d read Jeff Atwoods’s recommended reading list (as well as the latest podcast from StackOverflow which mentions “must read” programming-type books), and I decided it was time to update my library.

Unfortunately, most of the books weren’t available in-store – which is a shame. I don’t like ordering big, expensive books sight-unseen – call me old-fashioned, but I like to flip through some pages before I commit to buying a book.

I was able to find some other books on the list – although not many of them struck my fancy. The great age of reference books is behind us, thanks to the power of the Internet (and Google).

However, I did find one book that I really liked – “The Design of Everyday Things.”

After reading it, I’m amazed that we get anything done at all in our modern world, what with all the poor design everywhere.

Even if you’re not a computer programmer or a product designer, you should read this book – it’ll really give you an insight into why some things seem frustrating (and what effect that can have on you). Also interesting – the book was originally titled “the Psychology of Everyday things.”

Very interesting!