As many users get more advanced with their projects, they also tend to increase in size and, as a result, they have to increase the size of their disks. In the following guide, we will use the command line to perform this action as we at times do not have AWS console access. Feel free to translate this into using the GUI should you prefer that option.
NOTE: Some customers have more than one Executor Template for one reason or another so please make sure, in such case, you are using the correct one.
Requirements:
- SSH access with root in your Deployment.
- AWS Access key and ID.
- AWS cli tools.
We will need the current InstanceId of the running executor template. You can find this by running the following command. Modify the region to match your settings:
aws ec2 describe-instances --region ap-south-1 --filter Name=tag:Name,Values=executor-template --query 'Reservations[*].Instances[*].[InstanceId,ImageId]'
In my case I got this:
[ [ [ "i-0c773c80fc5ca21f2", "ami-06832d84cd1dbb448" ] ] ]
We can now find the volume of this instance with the following command:
aws ec2 describe-volumes --region ap-south-1 --filters Name=attachment.instance-id,Values=i-0c773c80fc5ca21f2
My output from this command looks like this:
{ "Volumes": [ { "Attachments": [ { "AttachTime": T08:20:31.000Z", "Device": "/dev/sda1", "InstanceId": "i-0c773c80fc5ca21f2", "State": "attached", "VolumeId": "vol-062fda3725b29daac", "DeleteOnTermination": true } ], "AvailabilityZone": "ap-south-1a", "CreateTime": T08:20:31.547Z", "Encrypted": false, "Size": 400, "SnapshotId": "snap-0ff238716e7a4f5f4", "State": "in-use", "VolumeId": "vol-062fda3725b29daac", "Iops": 100, "VolumeType": "gp2" } ] }
On my excutor-template here we got a 400GB drive and I am going to resize that to our new standard of 1TB.
"Size": 400, "SnapshotId": "snap-0ff238716e7a4f5f4", "State": "in-use", "VolumeId": "vol-062fda3725b29daac",
We resize this drive with the following command:
aws ec2 --region ap-south-1 modify-volume --volume-id vol-062fda3725b29daac --size 1024 { "VolumeModification": { "VolumeId": "vol-062fda3725b29daac", "ModificationState": "modifying", "TargetSize": 1024, "TargetIops": 3072, "TargetVolumeType": "gp2", "OriginalSize": 400, "OriginalIops": 100, "OriginalVolumeType": "gp2", "Progress": 0, "StartTime": T09:44:18.000Z" } }
Let's confirm AWS correctly resized our drive:
aws ec2 describe-volumes --region ap-south-1 --filters Name=attachment.instance-id,Values=i-0c773c80fc5ca21f2 { "Volumes": [ { "Attachments": [ { "AttachTime": T08:20:31.000Z", "Device": "/dev/sda1", "InstanceId": "i-0c773c80fc5ca21f2", "State": "attached", "VolumeId": "vol-062fda3725b29daac", "DeleteOnTermination": true } ], "AvailabilityZone": "ap-south-1a", "CreateTime": T08:20:31.547Z", "Encrypted": false, "Size": 1024, "SnapshotId": "snap-0ff238716e7a4f5f4", "State": "in-use", "VolumeId": "vol-062fda3725b29daac", "Iops": 3072, "VolumeType": "gp2" } ] }
Great, our new size is 1TB. Let's move on to make this new space useful:
You will need to modify these commands to fit your deployment.
Run pvdisplay which allows us to see the attributes of one or more physical volumes like size, physical extent size, space used for the volume group descriptor area and so on:
pvdisplay --- Physical volume --- PV Name /dev/xvdg VG Name docker PV Size 200.00 GiB / not usable 3.00 MiB Allocatable yes (but full) PE Size 4.00 MiB Total PE 51199 Free PE 0 Allocated PE 51199 PV UUID EuAFb8-YgfM-Zf7q-XLVG-5E5Z-qqnL-yZUO5n --- Physical volume --- PV Name /dev/xvdf VG Name vg0 PV Size 400.00 GiB / not usable 4.00 MiB Allocatable yes (but full) PE Size 4.00 MiB Total PE 102399 Free PE 0 Allocated PE 102399 PV UUID ScRtxE-cejk-2nk5-emK2-56gi-1bQq-39ZncD
Let us resize the volume with pvresize:
pvresize /dev/xvdf Physical volume "/dev/xvdf" changed 1 physical volume(s) resized / 0 physical volume(s) not resized
Let's view the new size:
pvdisplay --- Physical volume --- PV Name /dev/xvdg VG Name docker PV Size 200.00 GiB / not usable 3.00 MiB Allocatable yes (but full) PE Size 4.00 MiB Total PE 51199 Free PE 0 Allocated PE 51199 PV UUID EuAFb8-YgfM-Zf7q-XLVG-5E5Z-qqnL-yZUO5n --- Physical volume --- PV Name /dev/xvdf VG Name vg0 PV Size 1000.00 GiB / not usable 3.00 MiB Allocatable yes PE Size 4.00 MiB Total PE 255999 Free PE 153600 Allocated PE 102399 PV UUID ScRtxE-cejk-2nk5-emK2-56gi-1bQq-39ZncD
Now we extend this disk to use all this space:
lvextend -l %FREE /dev/vg0/domino Extending logical volume domino to 992.00 GiB Logical volume domino successfully resized
Let us look at this new disk:
lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT xvda 202:0 0 30G 0 disk └─xvda1 202:1 0 30G 0 part / xvdf 202:80 0 1000G 0 disk ├─vg0-logs (dm-1) 252:1 0 8G 0 lvm /domino/logs └─vg0-domino (dm-2) 252:2 0 992G 0 lvm /domino xvdg 202:96 0 200G 0 disk └─docker-lv0 (dm-0) 252:0 0 200G 0 lvm /domino/docker
And finally we extend the filesystem itself to use this space:
resize2fs /dev/vg0/domino resize2fs 1.42.9 (4-Feb-2014) Filesystem at /dev/vg0/domino is mounted on /domino; on-line resizing required old_desc_blocks = 25, new_desc_blocks = 62 The filesystem on /dev/vg0/domino is nowblocks long.
Finally confirm your new size:
df -h Filesystem Size Used Avail Use% Mounted on udev 3.9G 12K 3.9G 1% /dev tmpfs 799M 412K 798M 1% /run /dev/xvda1 30G 5.2G 23G 19% / none 4.0K 0 4.0K 0% /sys/fs/cgroup none 5.0M 0 5.0M 0% /run/lock none 3.9G 216K 3.9G 1% /run/shm none 100M 0 100M 0% /run/user /dev/mapper/vg0-domino 977G 278M 933G 1% /domino
Now you can snap a new AMI and use this as your new executor template. This is well documented in the following support article: https://support.dominodatalab.com/hc/en-us/articles/360001055263-Creating-a-new-executor-AMI
Comments
0 comments
Please sign in to leave a comment.