I need to reclaim space from LVM and cannot add a new disk
I used up all the space in one LV/VG and need it for another LV
Suppose you have 3 disks having one or multiple LVs and VGs, one of the LV is consumed completely while for some other there is hardly any usage, you can resize your PV and extend the LV.
Assume there are 3 disks, all have PVs formatted on the raw disk (no partition). There are VGs containing each PV. One VG has 2 partitions as LVs while the others only have 1 partition, extending 100% of the PV. To make it simpler for our use case I'll provide an alias for the LVs in question, we'll keep it simple, I'll name the single LV part of VG 0 as lv0 and the 2 LVs part of VG 1 as lv1 and lv2. Let's get started!
We want to extend lv0 since that file system has been consumed fully. Now, we can't just extend the lv using lvextend since the underlying pv doesn't have any more physical extents left which the vg can map. And, for the purpose of this article, we wouldn't be adding any more space to the underlying disk which is part of the VM.
For the purpose of this article, our VM consists of 3 disks. The steps involved and which we'll execute to extend lv0 are:
1. Backup the data or compress it - i.e create a compressed archive of the contents on lv1 and lv2 filesystems and move them to the file system mounted on the 3rd disk (not the pv which will be resized)
2. Delete the LVs, delete the VG, delete the PV, partition the underlying disk, commission new PVs for the partition(s) or the remaining raw disk, extend existing VG over the partition, create a new VG and add the PV commissioned over the remaining space, extend the lv part of vg0 and extend the file system part of lv0, create new LVs (as required) over vg1, format them with a file system and mount the file systems.
3. Move the original data back, uncompress if needed. At this point your application should benefit from the new diskspace.
- To create a compressed archive please follow my other article which can be found here.
- You can use any of the utilities to move the files to the different file system: mv (preserve ownership and permissions), favourites include rsync -ravu or cp -arx, which retains most permissions needed. Note: we won't delete /etc/fstab entries since we'd mount the file systems again after resizing and retain the vg/lv names)
- If you're not able to unmount the filesystem, due to files/services keeping files locked, use lsof or fuser to find the services, i.e.
lsof +f -- /device/nameor
fuser -muv /mnt/point
- Ensure any services and processes accessing the filesystem are stopped.
- Unmount the filesystem and remove the LV: umount /mnt/point ; lvchange -an lvX ; lvremove lvX
- Delete the vg using vgremove and pv using pvremove
- Create a partition using fdisk /dev/device-name or parted. Depending on using the entire disk or splitting it up! With fdisk, use all space we'll just hit enter to go with the defaults (or you could specify explicit values; w writes the table to disk.
- The kernel may not pick up the new partition so you will have to look at running partprobe /dev/device-name or partx -a /dev/device
- Create a pv using pvcreate /dev/device-name1 and pvcreate /dev/device-name2
- Add the pv created over /dev/device-name1 to the existing VG (vg0) : vgextend /dev/vg0 /dev/device-name1
- Extend the lv to use the remaining 100% space of vg0 : lvextend -l +100%FREE /dev/vg0/lv0; lsblk to confirm
- Resize the filesystem, resize2fs /dev/vg0/lv0 ; df -h to confirm
- For the other pv, which is, /dev/device-name2, we may need to (re)create the vg
- Create a new vg ; vgcreate name-of-vg /dev/device-name2
- Create a linear lv; lvcreate -l 100%FREE -n name-of-lv name-of-vg
- Format and mount the lv ; mkfs -t filesystem-type (check this before you unmount the filesystem; df -ThP or check fstab) /dev/vg1/lv-name and then mount -t filesystem-type mount-point (again, please make a note of the mount-point prior)
- If this was a new filesystem, add it to /etc/fstab and test umount and mount!