LVMのPhysical Volumeが見つからないとどうなるか

新しくLVMのLogical Volume作ろうとしたらエラーが出て作れなかった

WARNING: Device for PV fkpfAD-HZZG-JmI0-yOTu-TnMZ-J8Ul-B7Hg6D not found or rejected by a filter.
Cannot change VG libvirt_lvm while PVs are missing.

物理デバイスが消えてる? pvsコマンドで見てみよう.

$ sudo pvs
WARNING: Device for PV fkpfAD-HZZG-JmI0-yOTu-TnMZ-J8Ul-B7Hg6D not found or rejected by a filter.
PV VG Fmt Attr PSize PFree
/dev/sdc1 libvirt_lvm lvm2 a-- 1.82t 1.82t
/dev/sdd1 libvirt_lvm lvm2 a-- 1.82t 1.82t
[unknown] libvirt_lvm lvm2 a-m 1.82t 672.11g

たしかに"/dev/sdb1"であるべきところが"[unknown]"になっている.

dmesgでIOエラーなど出てないし, fdiskでも様子が見れていることを確認して, ふと思い出してみると, この前"dd if=/dev/zero of=/dev/sde"するのを間違えて, /dev/sdbにしてあわててCtrl-Cした後に, だいたいこんな感じだっただろとfdiskでパーティションを作り直したようなことを思いだした. その時のがうまくあってなかったんだろうな.

多分, /dev/sdbと/dev/sdcは同じ型だったしfdiskで見ればアタリはつけられそう.

$ sudo fdisk -l /dev/sdb
Disk /dev/sdb: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: D2D7EE18-BD3C-4E32-926B-D606E7056837
Device Start End Sectors Size Type
/dev/sdb1 4194304 3907028991 3902834688 1.8T Linux filesystem
sudo fdisk -l /dev/sdc
Disk /dev/sdc: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 3E260EEF-CEAB-4747-94DB-5E7F2296BD3F
Device Start End Sectors Size Type
/dev/sdc1 3999744 3907026943 3903027200 1.8T Linux filesystem

たしかに/dev/sdb1は"4194304"セクタからで, /dev/sdcは"3999744"セクタからで, 開始位置がずれている.

ただ, これで本当に同じ開始位置でいいのかの自信はなかった.

そこでLVMのヘッダがきっと先頭あたりにあるだろうことを期待して, 確認することを考えた. まず/dev/sdc1をダンプ

$ sudo hexdump -C /dev/sdc1|head
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000200  4c 41 42 45 4c 4f 4e 45  01 00 00 00 00 00 00 00  |LABELONE........|
00000210  e9 1e eb 95 20 00 00 00  4c 56 4d 32 20 30 30 31  |.... ...LVM2 001|
00000220  67 57 68 61 65 4e 49 64  65 31 32 77 4c 44 73 79  |gWhaeNIde12wLDsy|
00000230  53 6c 41 48 58 66 6e 54  42 38 44 6f 73 34 68 57  |SlAHXfnTB8Dos4hW|
00000240  00 00 f0 46 d1 01 00 00  00 00 10 00 00 00 00 00  |...F............|
00000250  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000260  00 00 00 00 00 00 00 00  00 10 00 00 00 00 00 00  |................|
00000270  00 f0 0f 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

一方で/dev/sdb1の先頭のダンプ.

$ sudo hexdump -C /dev/sdb1|head
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
0004c490  fe a7 00 00 fe a7 00 00  fe a7 00 00 fe a7 00 00  |................|
*
0004d460  ff a7 00 00 ff a7 00 00  ff a7 00 00 ff a7 00 00  |................|
*
0004e430  40 ad 00 00 40 ad 00 00  40 ad 00 00 40 ad 00 00  |@...@...@...@...|
*
0004f400  61 ad 00 00 61 ad 00 00  61 ad 00 00 61 ad 00 00  |a...a...a...a...|
*

見たまんまext4の先頭のメタデータ(inode bitmapとかGroup Descriptorのあたり)で確かに以前よりもパーティションの開始位置が後ろにずれているっぽい.

とりあえず"LABELONE"を頼りにgrepしてみる.

sudo hexdump -C /dev/sdb|grep -C5 LABELONE
*
7a0e1b10  c6 18 00 00 c6 18 00 00  c6 18 00 00 c6 18 00 00  |................|
*
7a0e2ae0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
7a100200  4c 41 42 45 4c 4f 4e 45  01 00 00 00 00 00 00 00  |LABELONE........|
7a100210  33 c2 fa 36 20 00 00 00  4c 56 4d 32 20 30 30 31  |3..6 ...LVM2 001|
7a100220  66 6b 70 66 41 44 48 5a  5a 47 4a 6d 49 30 79 4f  |fkpfADHZZGJmI0yO|
7a100230  54 75 54 6e 4d 5a 4a 38  55 6c 42 37 48 67 36 44  |TuTnMZJ8UlB7Hg6D|
7a100240  00 00 f0 46 d1 01 00 00  00 00 10 00 00 00 00 00  |...F............|
7a100250  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

見つかった. 最初のエラーにあった"fkpfAD-HZZG-JmI0-yOTu-TnMZ-J8Ul-B7Hg6D"っぽい文字列も見える. 信頼できそう.

(0x7a100200-0x200(/dev/sdb1の"LABELONE"の位置))/512 = 3999744 ということでやっぱり/dev/sdbと同じパーティションテーブルでよかった.

あとはfdiskするだけ

$ sudo fdisk /dev/sdb
Welcome to fdisk (util-linux 2.28).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Command (m for help): p
Disk /dev/sdb: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: D2D7EE18-BD3C-4E32-926B-D606E7056837
Device Start End Sectors Size Type
/dev/sdb1 4194304 3907028991 3902834688 1.8T Linux filesystem
Command (m for help): d
Selected partition 1
Partition 1 has been deleted.
Command (m for help): n
Partition number (1-128, default 1):
First sector (34-3907029134, default 2048): 3999744
Last sector, +sectors or +size{K,M,G,T,P} (3999744-3907029134, default 3907029134):
Created a new partition 1 of type 'Linux filesystem' and of size 1.8 TiB.
Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

最後にpvscan

$ sudo pvscan
PV /dev/sdb1 VG libvirt_lvm lvm2 [1.82 TiB / 672.11 GiB free]
PV /dev/sdc1 VG libvirt_lvm lvm2 [1.82 TiB / 1.82 TiB free]
PV /dev/sdd1 VG libvirt_lvm lvm2 [1.82 TiB / 1.82 TiB free]
Total: 3 [5.45 TiB] / in use: 3 [5.45 TiB] / in no VG: 0 [0 ]

認識された. よかった.