Loading large-volume images of OS

VK Cloud has restrictions on the size of uploaded images of operating systems. If the limit is exceeded, a message appears like:

Copy
An error occurred (InvalidArgument) when calling the UploadPart operation: Part number must be an integer between 1 and 10000, inclusive

Next, we consider downloading images larger than 500 GB through the Cloud Storage.

Will be used:

• a virtual machine of at least 500 GB in size;
• a local machine of the Linux family with the utility installed gzip.

1. Check the possibility of migration. The virtual machine must meet the following requirements:

   • The VM operating system has a 64-bit architecture.
   • VM uses BIOS emulation.
   • The current user has administrator rights.
   • At least one disk is connected to the VM.

   To migrate a VM with UEFI emulation, use Hystax or transfer data to a new Hyper-V VM with BIOS emulation.

2. Create an account and bucket uc_bucket.

3. Make sure that you have installed and configured AWS CLI. Specify the data for connecting to the bucket in it (Access key ID and Secret key). Open the configuration file ~/.aws/config and make changes to it:

   Copy
   [default]region = ru-mskoutput = jsons3 =    max_concurrent_requests = 20    max_queue_size = 10000    multipart_threshold = 1024MB    multipart_chunksize = 384MB    addressing_style = path

4. Create Ubuntu 22.04 VM to VK Cloud.

5. Install the gzip utility on the VM.

6. Create a disk of at least 600 GB in size and connect it to the VM.

1. Run the command:

   Copy
   dd if=/dev/vdX bs=32M | gzip -c | aws s3 cp - s3://uc_bucket/image.raw.gz --endpoint-url http://hb.ru-msk.vkcs.cloud

2. Make sure that the download has started using the command:

   Copy
   aws s3api list-multipart-uploads --bucket uc_bucket --endpoint-url http://hb.ru-msk.vkcs.cloud

   Example of command output

3. Check the partitions using the command:

   Copy
   aws s3api list-parts --bucket uc_bucket --endpoint-url http://hb.ru-msk.vkcs.cloud --key image.raw.gz  --upload-id 3ceXH7brs7r8DohqQ9BsJzfjkkhMxQux67Z8MQXYGh9BvfrmffLufWMzwiLWPbU3XkWE3ibSefgQ1GU81ER66EEHfKMZM8xxqRsDkBaN63XXXX

   Example of command output

4. Wait for the upload to the object storage. The output of the aws s3api list-multipart-uploads command should not contain data in the Uploads block.

1. Connect to VM via SSH.

2. Check for a connected disk using the command lsblk.

   Example of command output

3. Move the image to disk using the command:

   Copy
   wget https://uc_bucket.hb.bizmrg.com/image.raw.gz -O /dev/vdb/image.raw.gz

4. Unpack the image using the command:

   Copy
   gunzip -c image.raw.gz | dd of=/dev/vdb bs=32M

Use the instructions.

Create a VM by selecting the downloaded image as the operating system, according to instructions.

The created resources are charged and consume computing resources. If you don't need them anymore:

• Delete загруженный образ из объектного хранилища.
• Delete or stop VM.
• Delete диск.