(Obviously this doesn’t apply only to rPi images)
I decided to restore the backup of the SD card of my raspberry Pi to a different card because I was having the file system corrupt too many times. As always, when you try to do something that seems to be easy enough and pretty straight forward, actually, it’s not, NEVER, at least with me. I always get stuck at some step of the process. So here is a tutorial to help you in case you decide to do the same.
The problem was when restoring the image to a different card that supposedly should be the same size (8GB) is actually not the SAME!!! The image has 7969177600 bytes and the new card 7948206080 bytes so I had to shrink the partitions so the image fits the card.
The below tutorial is not actually mine (credits to RomanG from raspberrypi.org forums for his tutorial), I just gave it some adjustments.
His cards were 4GB (3963MB and 3904MB) respectively.
Modifying the image size and writing to the new card
First, create a copy of your backup image to play with:
cp pi_30112012.img test.img
Searching google I found that image files can be mounted as a block devices using losetup command:
sudo losetup -f --show test.img
/dev/loop1
Let’s see what we got:
sudo fdisk -l /dev/loop1
Disk /dev/loop1: 3965 MB, 3965190144 bytes
255 heads, 63 sectors/track, 482 cylinders, total 7744512 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000714e9
Device Boot Start End Blocks Id System
/dev/loop1p1 8192 122879 57344 c W95 FAT32 (LBA)
/dev/loop1p2 122880 7744511 3810816 83 Linux
As you can see, this image is 3965MB and 7744512 sectors (512b x 7744512 = 3965190144 bytes)
Also geometry of the card is important to note, 255 heads and 63 sectors. (In some linux versions the fdisk command doesn’t show this information right away. If this is your case, run “sudo fdisk -l -u=cylinders /dev/loop1“. You can also get this information for your card from wikipedia.
The last sector of the last partition is Total sectors – 1, so there is no spare room left on the card.
First find your SD card mountpoint
lsblk
Now, let’s have a look at the target SD card
sudo fdisk -l /dev/mmcblk0
Disk /dev/mmcblk0: 3963 MB, 3963617280 bytes
128 heads, 63 sectors/track, 960 cylinders, total 7741440 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/mmcblk0p1 8192 7741439 3866624 6 FAT16
The target SD card is 3963MB and 7741440 sectors, so it is smaller than the image by 3072 sectors or 1572864 bytes. Also note the geometry of the card showing 128 heads and 63 sector. For some reason this is incorrect as this card has 255 heads and 63 sectors. For some reason fdisk does not show this always correctly. If you can’t see these values, please remember -u=cylinders option.
We need to shrink second partition of the image, otherwise if we attempt to write image to the card, it will result in a corrupted file system.
Let’s mount the second partition of the image as a loopback device so that we can resize it.
The image starts at sector 122880, which is 512*122880=62914560 bytes
Now that we have the offset for mounting the second partition of the image,
we can now use with losetup’s -o argument to set an offset.
sudo losetup -f --show -o 62914560 test.img
/dev/loop2
Scan for errors before resizing the image
sudo e2fsck -f /dev/loop2
e2fsck 1.42 (29-Nov-2011)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/loop2: 80300/245760 files (0.2% non-contiguous), 604445/952704 blocks
Using resize2fs we can shrink the partition, using -p option to show progress and specifying the new size.
-p Prints out a percentage completion bars for each resize2fs operation, so that the user can keep track of what the program is doing.
size parameter may be suffixed by one of the following the units designators: ‘s’, ‘K’, ‘M’, or ‘G’, for 512 byte sectors, kilobytes, megabytes, or gigabytes, respectively.
We need to scale down the image by 3072 sectors. Image size is End – Start = 7744512 – 122880 = 7621632 sectors, subtracting 3072 sectors, the new size is 7621632 – 3072 = 7618560 sectors.
sudo resize2fs -p /dev/loop2 7618560s
Resizing the filesystem on /dev/loop2 to 952320 (4k) blocks.
The filesystem on /dev/loop2 is now 952320 blocks long.
Now that the filesystem was resized, loopback devices are no longer needed:
sudo losetup -d /dev/loop1 /dev/loop2
To write the modified image to the SD card, use command:
(this may take a while, depending on the speed of your card, in my case it took more than 30 minutes)
sudo dd if=test.img of=/dev/mmcblk0
dd: writing to `/dev/mmcblk0': No space left on device
7741441+0 records in
7741440+0 records out
3963617280 bytes (4.0 GB) copied, 1908.13 s, 2.1 MB/s
As you can see, there is an error message ‘No space left on the device’, but because we re-sized the file system, card will be usable and Raspberry Pi will boot up without any issues.
However, if you attempt to look at the partition(s) on the card with a tool such us parted or gparted, it will end up with an error “…partition extends beyond end of device…”
To correct this, we need to modify MBR (Master Boot Record) of the SD card.
Editing the MBR
First, we will extract the first sector from the card which contains MBR.
dd if=/dev/mmcblk0 of=sd.mbr bs=512 count=1
This will create a file sd.mbr which we will need to edit with a hex editor.
To play with the MBR in the hex editor, you need to know what values at which address need to be modified, Wikipedia has very good reference material about MBR.
From there you can see that Partition entry #1 starts at address 0x1BE and Partition entry #2 starts at address 0x1CE.
Layout of the partition table entry is described here.
Values for the CHS address of last absolute sector in partition, LBA of first absolute sector in the partition and number of sectors in partition need to be calculated and changed.
Checking again the original SD card, we got:
sudo fdisk -l /dev/loop1
Disk /dev/loop1: 3965 MB, 3965190144 bytes
255 heads, 63 sectors/track, 482 cylinders, total 7744512 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000714e9
Device Boot Start End Blocks Id System
/dev/loop1p1 8192 122879 57344 c W95 FAT32 (LBA)
/dev/loop1p2 122880 7744511 3810816 83 Linux
Using your favorite hex editor, open the sd.mbr file.
According to Wikipedia article linked previously, 2nd partition entry in MBR starts at 0x1CE address and is 16 bytes long:
Address 0 1 2 3 4 5 6 7 8 9 A B C D E F DUMP
........
000001c0 03 00 0C A5 1E 07 00 20 00 00 00 C0 01 00 00 00 ...¥... ...À....
000001d0 C1 80 83 03 10 AF 00 E0 01 00 00 4C 74 00 00 00 Á€ƒ..¯.à...Lt...
1st byte indicates partition status (00h)
2nd to 4th bytes indicates CHS (cylinder, head, sector) address of the first absolute sector in the partition (00h C1h 80h)
5th byte indicates partition type (83h)
Next part is the important one for this modification because we only want to change the end of the partition:
6th to 8th bytes indicates CHS (cylinder, head, sector) address of the last absolute sector in the partition (03h 10h AFh)
6th – 03h is the head which is 3 dec
7th – 10h indicates sector in bits 5-0 and in bits 7-6 are high bits of cylinder. 10h = 00010000 (binary) and in bits 5-0 we have 010000 which is 16 dec
8th – AFh indicates low bits of cylinder which in binary is 10101111, adding two high bits from previous address we have 0010101111 which is 175 dec.
So the last sector is at cylinder 175, sector 16 head 3 in the original card.
9th to 12th bytes indicates LBA of the first absolute sector (4 bytes) which is 122880 = 1E000 = 0001E000 = 00 E0 01 00 (little endian).
13th to 16th indicates number of sectors in partition (last 4 bytes). These are in little endian notation (meaning you have to read from right to left), so we have 00 4C 74 00 which is 00744C00h and 7621632d sectors in decimal format.
fdisk above shown us that second partition starts at sector 122880 and ends at sector 7744511, which is 7744511-122880=7621631 sectors.
But we know that the target SD card has only 7741440 sectors and that the second partition starts at sector 122880 so we will need to adjust MBR record accordingly,
7741440-122880=7618560 sectors which is 744000h, so the the last 4 bytes of the 2nd partition record will be 00 40 74 00 (little endian, least significant bit first).
CHS tuples can be mapped to LBA address with the following formula:
- LBA = (C × HPC + H) × SPT + (S – 1)
where:
- C, H and S are the cylinder number, the head number, and the sector number
- LBA is the logical block address
- HPC is the maximum number of heads per cylinder (reported by disk drive, typically 16 for 28-bit LBA)
- SPT is the maximum number of sectors per track (reported by disk drive, typically 63 for 28-bit LBA)
LBA addresses can be mapped to CHS tuples with the following formula (“mod” is the modulo operation, i.e. the remainder, and “÷” is integer division, i.e. the quotient of the division where any fractional part is discarded):
C = LBA ÷ (HPC × SPT)
H = (LBA ÷ SPT) mod HPC
S = (LBA mod SPT) + 1
Now we know the number of sectors is 7741440 and last sector of the 2nd partition is 7741439 which is also LBA of the last sector and we just need to convert it to CHS notation. This Wikipedia article describes how it is done.
We have 255 heads and 63 sectors/track, so:
LBA=7741439
HPC=255
SPT=63
C = 7741439/(255*63) = 481 => 0111100001b, high two bits 01b=01h, low eight bits 11100001b=E1h
H = (7741439/63) mod 255 = 224 => 11100000b => E0h
S = (7741439 mod 63) + 1 = 63 => 111111b => 3Fh
so
6th byte will be E0h
7th byte will be 01111111b => 7Fh
8th byte will be E1h
Now we can reconstruct new row at address 000001d0 of MBR record:
000001d0 C1 80 83 E0 7F E1 00 E0 01 00 00 40 74 00 00 00
Edit the file accordingly, save it and then write it back to the SD card, using dd command
dd if=sd.mbr of=/dev/mmcblk0 bs=512 count=1
Checking:
sudo fdisk -l /dev/mmcblk0
Disk /dev/mmcblk0: 3963 MB, 3963617280 bytes
4 heads, 16 sectors/track, 120960 cylinders, total 7741440 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000714e9
Device Boot Start End Blocks Id System
/dev/mmcblk0p1 8192 122879 57344 c W95 FAT32 (LBA)
/dev/mmcblk0p2 122880 7741439 3809280 83 Linux
We can see that the end sector of last partition is less than total number of sectors.
If we run now parted or gparted, the card partitions will be recognized and can be further manipulated by using these tools.
randomproj 