User:Skierpage/Disk recovery
Disaster!
I followed instructions to create a bootable SD card,
zcat blah.bootable.gz > /dev/sdX
This is dangerous because it overwrites the partition, but I had done it before, I knew there was nothing on my SD card and /dev/sda is my SD card.
But /dev/sda is my hard drive. The whole thing.
I checked with df and the size looked wrong, then I read my notes (should have done that first) and I was overwriting the wrong disk. Ctrl-C!
Then I tested the .gz file and it was corrupt, so I thought nothing had been written. My computer worked fine.
The next day I got an error coming out of standby about not reading C:\$MFT or something. So I decided to reboot. Big mistake, I should have read partition tables, backed up some key files, and prepared for disaster.
- Rule 1: do what you can while it's working.
Errors shutting down, then could not reboot, bad disk.
Failed recovery
I couldn't find my Windows XP CD-ROM, even though Falcon Northwest make a big deal. Of course my Windows XP is so dramatically different thanks to all the updates and service packs that the original disk has been useless for System File Checking and file recovery and such. I had always meant to follow instructions to slipstream a new boot CD-ROM, where you make a fresh Windows CD-ROM that contains the latest files, but never did.
- Rule 2 You need a boot disk around.
Falcon Northwest provided me a personalized Recovery Disk, I inserted this and nothing happened. Zero feedback.
Into Kubuntu Live CD
I had built a Kubuntu Live CD. That booted fine.
The Kubuntu Live CD startup menu has "check disk" menu choice is useless, it just checks the CD-ROM. BUG: it is badly named.
So I ran the Live CD environment, and that worked fine. I'm updating this web page from the Live CD. Hooray for free software that is happy for the world to make bootable CDs.
Create a recovery disk/USB?
There are lots of dedicated "recovery CD" distributions. Distrowatch lists dozens, and the search leaves out the GParted distribution.
The problem is, I'm running from my only CD drive, so I can't burn another CD. Besides, I don't want to waste another CD blank. I have gigabytes of space on my USB flash drive and SD memory card.
BUG: All the distributions I checked have crap guides to creating a bootable USB. Some even talk about making a floppy. The functionality and documentation for making a bootable USB needs to be separated from all these distributions.
Use what I have
But I'm already running Linux. Kubuntu has the parted partition editor and the NTFS tools.
BUG: all the online web recovery instructions are crap. They tell you how to make a recovery CD or floppy or whatever, tell you to stick it in the machine and reboot, but fail to tell you what to do next. I guess they assume you'll walk through their beautiful menu system. It is stupid and short-sighted not to mirror help and guidelines on a web site. You should never have to create physical media and boot from it just to read documentation.
The first four pages of Google results for "linux partition recovery tutorial" are either sites trying to sell me something or yet another guide to creating a recovery CD. I gave up finding any explanation of what to do.
Find your hard drive
You have to find the UNIX device name for your disk, like hdb ("hard disk B") or sda ("SCSCI disk A" but also "Secure Digital" memory card). Then tools can access your disk using its entry in the special /dev directory. That's how I was able to scribble all over my hard drive in the first place.
Unfortunately, the /dev directory has lots of devices in it.
Another approach is to look for boot messages where the Linux kernel finds hardware devices. In a Linux console, enter
dmesg | grep disk dmesg | grep drive
one of those should identify promising disk devices, or enter
dmesg | less
to read the whole boot thing (which might have hints about the state of the drive).
The best approach I've found is the somewhat undocumented one of looking in the special file /proc/partitions:
cat /proc/partitions
This identifies the partitions on the various devices, and their blocks. I guess if your hard drive was really badly trashed, it wouldn't have any partitions.
All these confirmed my hunch that /dev/sda is my poor hard drive. The good news is /proc/partitions showed my sda1 and sda2
Run parted
Now start parted. I should probably start this as the root user, using sudo
$ parted Using /dev/sdb
that's wrong, I need to select /dev/sda.
(parted) help ... lots of commands (parted) select /dev/sda Error: Error opening /dev/sda: Permission denied
so I do have to be root.
$ parted GNU Parted 1.8.8 Using /dev/sda (good, now I'm root it picks sda as I can write to it) Welcome to GNU Parted! Type 'help' to view a list of commands. (parted) check Partition number? 1 Error: \8. is 512k, but it has 15994 clusters (63976k). Ignore/Cancel? i Fatal: Bad FAT: cluster 12 is cross-linked for \0.8. You should run dosfsck or scandisk.
there you go! Let's see if the Linux partition is OK at least.
(parted) check Partition number? 1 Error: Partition doesn't exist.
Damn.
(parted) print Model: ATA ST3120026AS (scsi) Disk /dev/sda: 120GB Sector size (logical/physical): 512B/512B Partition Table: msdos Number Start End Size Type File system Flags 1 17.4kB 2031MB 2031MB primary fat32 boot, lba
damn. That's completely wrong, I should have an enormous Windows partition. My original destructive zcat bootable scribbled a new partition table on the front of my disk that's completely wrong.
How bad is it
So as a result of scribbling over the front of the disk I have
- probably trashed the special loader (called grub? maybe) that let me boot from Windows or Linux
- probably trashed the Windows boot code, I'm not sure if it's in the first partition or somewhere else (the MBR "Master Boot Record" ?)
- probably trashed a big chunk of my NTFS Windows partition
- somehow completely messed up my Linux setup. With firefishe's advice I had set up the second half of the disk with root, swap, and home, now it's nowhere to be seen :-(
parted has a rescue command that will try to locate and revive partitions, but I thought there's a backup somewhere on the disk with partition information. The "MBR" again?? But since the first partition table looks OK, the program trusts it.
Recovering partitions
So now I'm back to trying to find out more about disk repair.
- http://sysblogd.wordpress.com/2008/01/05/data-recovery-with-linux-from-hard-drives-cd-roms-or-dvds/ seems useful, has other links.
Fortunately I made notes when I created the partition setup.
/dev/sda1 ntfs Size 90025 MB (83.8 GB) hopefully untouched /dev/sda2 ext3 Mount / Format? checked Size 20003 MB Used 495 MB /dev/sda3 ext4 Mount /home Format? checked Size 8003 MB Used unknown /dev/sda4 swap Mount [blank] Format? [blank] Size 1998 MB Used unknown
Rather than guess at partitions with parted's rescue, testdisk sounded promising. But it's not on the Kubuntu live CD.
I tried using aptitude to install it, but that didn't work on the LiveCD. So I downloaded it. There's a Ubuntu package available, navigate to the Jaunty version, get the URL of the amd64-bit version. I have no idea how you can install new software to a read-only CD-ROM, so I did this in the temporary directory
cd /tmp wget http://ubuntu.cs.utah.edu/ubuntu/pool/universe/t/testdisk/testdisk_6.10-1_amd64.deb
then
sudo dpkg -i testdisk_6.10-1_amd64.deb
to install.
testdisk to the rescue!
Praise be, this has a pretty good intro, so I walked through that while running
sudo testdisk
and then turned on its expert mode and went slowly.
Like every other tool testdisk initially found only the bad small partition:
Disk /dev/sda - 120 GB / 111 GiB - CHS 14593 255 63
Current partition structure:
Partition Start End Size in sectors
Warning: Incorrect number of heads/cylinder 63 (FAT) != 255 (HD)
Warning: Incorrect number of sectors per track 62 (FAT) != 63 (HD)
1 * FAT32 LBA 0 0 35 246 237 22 3966909 [LIVE]
But then its quick analyze command found my various Linux partitions!
And then, I pressed Enter then chose [Deep analyze], which went back through my disk slowly. Glory be, this found evidence of the bigger partition!
Disk /dev/sda - 120 GB / 111 GiB - CHS 14593 255 63
Partition Start End Size in sectors
D FAT32 0 0 35 246 254 63 3968021 [LIVE]
P HPFS - NTFS 0 1 1 10944 254 63 175831362 [Windows XP]
D Linux 1902 0 1 4419 254 63 40451670
* Linux 10945 0 1 13376 254 63 39070080
D Linux 12139 2 1 14571 254 63 39086019</span
P Linux 13377 0 1 14349 254 63 15631245
P Linux Swap 14350 0 1 14592 254 63 3903795
wahoo! There's my Windows XP partition! It also found two extra bogus partitions, but by matching the size ("sector" is 1/2 kB) and using the info testdisk prints as you choose each partition, I figured out the proper partitions whose Start follows the End of the previous, and used the left and right arrow keys to enable the good partitions as shown, and delete the bad ones.
Press Enter, confirm writing the new thing. I'm not sure about the geometry errors.
Then it immediately displayed information about my NTFS partition:
Boot sector Status: Bad Backup boot sector Status: OK Sectors are not identical. A valid NTFS Boot sector must be present in order to access any data; even if the partition is not bootable.
so I chose its [Backup BS] to copy the backup boot sector.
I tried [Repair MFT] to check the NTFS Master File Table, but that requires rebooting.
One last check of the format (using the Advanced menu, confusing), and I exited and prepared to reboot!
Testdisk glitches
[Quit] everywhere to return to previous menu is very confusing. It should support [Esc]ape key, and the command should be [Back]
Analyze forces a Quick search
Also, if you choose analyze, you have a [Quick search] option, so there's no way to return to the main menu. You have to start a quick search. If you stop that, it'll be half-way through finding the partitions, so it looks as if it's lost all the information I've carefully brought back.
It's crazy that the only way I can see the current MBR is by choosing "Advanced".
So the main menu should have a "Current status" that shows
- what's on the disk MBR and partition info
- indicator if what testdisk has in memory is different (because you haven't written it yet)
- if not, show what's in its idea of the MBR and partitions is
- any other info it has or hasn't written to disk.
Ctrl-C
I tried to copy some text to here with Shift+Ctrl+C, and accidentally pressed Ctrl+C and testdisk immediately exited. Please trap Ctrl+C and ask "Are you sure?"
give user-friendly MB/KB size
When people are looking for partitions, they probably know the size in kB or kibibytes, but not in sectors. So maybe in parentheses give a user-friendly size like 5,203 KB or 12,304 MB.
Add partition is confusing
This is weird. First, I thought this would add the partition that testdisk had found that I've highlighted with the cursor keys, since D(eleted) is the opposite of Add.
Then, when I really did want to add a partition, the information line has no explanation
D No partition 0 0 1 1020 251 62 15952104
and I can't choose a particular partition type with the cursor, it highlights a row of three at a time!!
List under [Analyze] > [Boot] is confusing
I re-entered TestDisk to view some files on my corrupted partition. I remembered this feature was called [List], but I had trouble finding it. I finally found it under [Analyze] > [Boot] , but the help for [Boot] is "Boot sector recovery". There's no indication that the [Boot] screen also has a "[List] directories and files, copy data from NTFS" feature.
Back to booting
I told TestDisk that Linux was my boot partition, however booting from the hard drive was still a complete failure, it just hangs forever. I have my partitions, but I've lost any knowledge of booting. So I had to boot the Kubuntu LiveCD again.
Kubuntu had installed the grub boot menu that gives me a choice of whether to boot Windows or Kubuntu, so all I need to do is create another one of those.
As usual there are tons of guides to restoring grub, and they're all crap out-of-date stuff that mostly explain nothing. E.g. my hard drive is /dev/sda, but grub turns this into hd0. And unlike TestDisk there's no way to know what the heck you just did or verify what you put on the disk. It's a shot in the dark.
https://help.ubuntu.com/community/RecoveringUbuntuAfterInstallingWindows is probably the best instructions, so I followed that.
Linux boot back
Now my hard drive boots into Kubuntu, but seems to ignore windows.
grub woes
The grub command is very poorly explained. I want a program to tell me what the current boot menu in my MBR looks like without rebooting. Does grub do this? Who knows. It seems to show a typical boot menu, but maybe that's coming from my Linux drive. Who knows?
Windows partition corrupt
Kubuntu automatically mounted my NTFS partition as it did before using the NTFS-3G driver:
/dev/sda1 on /media/WindowsCdrive type fuseblk (rw,nosuid,nodev,allow_other,blksize=4096)
but that was giving me tons of
ls: cannot access /media/WindowsCdrive/AUTOEXEC.BAT: No such file or directory ls: cannot access /media/WindowsCdrive/boot.ini: No such file or directory
as well as a directory listing. Strange, since I was able to navigate in TestDisk to some of these files in "My Documents".
- fsck doesn't work yet for ntfs-3g mounts.
- There's an ntfsfix command, but it's in a different older NTFS for Linux package.
- Can I run Windows' chkdsk command under Wine?
- I found my Windows XP CD-ROM and booted into its Recovery Console, but that doesn't acknowledge I have a hard drive at all. It's supposed to ask what Windows installation I want to work with, but doesn't.
I asked about my problems on the NTFS-3G forum.
Back up the partition
I bought a Toshiba 400GB External USB 2.0 Hard Drive Model: HDDR400E03X , so I can just dump the entire NTFS partition onto it, insert it into a Windows laptop, and run chkdsk on it.
But how to "dump"? I think the command is
dd if=/dev/sda1 of=/dev/sdc2 conv=noerror,sync
but I'm unclear how to create the NTFS partition.
Repartition USB hard drive
My hard drive according to TestDisk is
CHS 14593 255 63 1 * HPFS - NTFS 0 1 1 10944 254 63 175831362 [Windows XP]
TestDisk thinks the Toshiba has the same heads/sectors, so each "cylinder" is the same. fdisk -l thinks the Toshiba is
Device Boot Start End Blocks Id System /dev/sdc1 1 48641 390708801 c W95 FAT32 (LBA)
- Clonezilla can clone partitions, but it's yet another boot CD to burn and a layer of software above.
- ntfsclone from NTFS tools can clone NTFS, but the doc doesn't make it clear whether it can clone to another partition.
- parted has a cp command, but the features page indicates that this doesn't work to create or copy NTFS partitions.
So I guess I'll create an NTFS partition on windows. I used parted to resize the FAT32 partition smaller to make room:
(parted) unit cyl (parted) resize 1 0 20000 (parted) print Model: Toshiba External USB HDD (scsi) Disk /dev/sdc: 48641cyl
Number Start End Size Type File system Flags 1 0cyl 19999cyl 19999cyl primary fat32 lba
(parted seems to count from 0, not from 1.)
Now to create an NTFS partition like
1 32256B 90025689599B 90025657344B primary ntfs boot
on Windows, I called it partition X:. 90025657344 bytes is 87915681 kiB. I guess I'll round up to 86GiB for safety. That was a stupid idea because it makes it hard to copy back.
Backing up
From Kubuntu,
dd if=/dev/sda1 of=/dev/sdc2 options blah blah
worked fine.
Then reinserted disk in Windows. Explorer convinced the drive is broken; but from the command line I was able to run
chkdsk /f X:
lots of errors but it all looked pretty good.
Reinserted in Kubuntu, realized it was too big to copy back plus I want to make yet another backup for safety.
Permissions messed up
Reinserted in Windows to reduce file size. But first, a weird problem. Computer Management > Disk Management shows X: drive "86.00GB, Healthy (Active, Primary Partition)", 20GB free.
But Windows Explorer thinks X:\ is not accessible, access denied. Finally figured out that the ACL for it only allows administrator access!!
Compared with C security
- Authenticated Users
- Special permissions Allow grayed out
- SYSTEM
- Full Control, Modify, Read & Execute, List folder contents, Read, Write all Allow
- Administrators (Machine Name\Administrators)
- same as SYSTEM
- Users
- Read & execute, List folder contents, Read
C: Advanced security
- Administrators (Machine Name\Administrators)
- Full Control apply to this folder, subfolders, and fileto
- SYSTEM
- same as Administrators
- Users (Machine Name\Users)
- Read & execute apply to this folder, subfolders, and files
- Authenticated Users
- Special apply to Subfolders and files only
- Authenticated Users
- Create folders/append data, apply to This folder only.
Compared with new NTFS partition' security
- Authenticated Users
- Modify, Read & execute, List folder contents, Read, Write
- SYSTEM
- Full Control, Modify, Read & Execute, List folder contents, Read, Write
- Administrators (Machine Name\Administrators)
- same as SYSTEM
new empty NTFS partition's Advanced security:
- Administrators (Machine Name\Administrators)
- Full Control
- SYSTEM
- Full Control
- Authenticated Users
- Modify
- Users (Machine Name\Users)
- Read & execute
all applying to this folder and subfolders.
Started applying permissions to X: but got errors as it trundled through sub-folders setting permissions for SYSTEM:
- cygwin\bin\get , \cygwin\etc\passwd , \cygwin\etc\profile.mytemp, \cygwin\etc\sshd_config, \cygwin\etc\ssh_host_dsa_key, more ssh, \cygwin\home;
- \cygwin\lib\perl5\5.8\CPAN\Config.pm{,~}, \cygwin\lib\perl5\5.8\{Auto, Bundle, \cygwin\auto, \cygwin\file, \cygwin\HTML}, site_perl\Date blah blah more 5.8 perl stuff
- \cygwin\usr\local\bin\mergebox.pl, \cygwin\usr\local\src
Reset it to make Advanced security match "empty NTFS partition" (i.e. not quite the same settings for Authenticated Users as on the laptop C: drive. Same errors.
This is probably why I can't access the drive from Recovery Console.
Make another backup partition...
Just to be safe I backed up again. I want to make an exact copy of /dev/sda1, but parted is hella confusing because
- the first partition doesn't actually start at the very start, it starts many bytes in to skip the boot record stuff
- parted indiscriminately rounds without telling you, e.g. "Each cylinder is 8225kB", actually each cylinder is 8225280 bytes.
- parted's math still doesn't work out. 10944 cylinders with geometry of 255 heads on 63 512 byte sectors is 90,017,464,320 bytes, but parted says my original partition is 90,025,657,344 bytes, quite a bit bigger. Arghhhh.
Number Start End Size Type File system Flags 1 0cyl 10944cyl 10944cyl primary ntfs boot Number Start End Size Type File system Flags 1 32256B 90025689599B 90025657344B primary ntfs boot
I guess I'll work in bytes. Remove the existing end of Toshiba
(parted) rm 3
create new partition starting at the same place but with an end that's exactly the right size in bytes.
(parted) mkpart primary ntfs 256847642624B 346873299967B
check
Number Start End Size Type File system Flags 1 32256B 164505599999B 164505567744B primary fat32 lba 2 164505845760B 256847642623B 92341796864B primary ntfs boot 3 256847642624B 346873299967B 90025657344B primary ntfs
partition 3 is exactly the right size.
Now make the remaining space another partition:
(parted) mkpartfs primary fat32 346873299968B 400085844479B
Number Start End Size Type File system Flags 1 32256B 164505599999B 164505567744B primary fat32 lba 2 164505845760B 256847642623B 92341796864B primary ntfs boot 3 256847642624B 346873299967B 90025657344B primary ntfs 4 346873299968B 400085844479B 53212544512B primary fat32 lba
Now we can repeat the dd. parted has its own
cp [FROM-DEVICE] FROM-NUMBER TO-NUMBER copy file system to another partition
command, but stick to dd. From a guide, try bigger block size and force all blocks to be read and written:
sudo dd if=/dev/sda1 of=/dev/sdc3 bs=64M conv=sync,noerror