Tags: nanovms/ops
Tags
oci: multipart image upload, part timeout, and flexible shape sizing (#… …1792) * oci: upload the image in parts instead of one PutObject Importing an image on OCI failed for any image of real size. CreateImage sent the whole qcow2 in a single PutObject, and the SDK's HTTP client gave up on its own timeout while still awaiting the response headers: Put ".../o/<image>": context deadline exceeded (Client.Timeout exceeded while awaiting headers) The transfer package's upload manager splits the object into parts, uploads them in parallel and retries a single failed part instead of the whole transfer, which is what the OCI CLI does for the same reason. The upload manager wants the concrete ObjectStorageClient, while the provider deliberately hides Object Storage behind the StorageService interface so it can be mocked. So the upload is now behind an ImageUploader seam: Initialize wires the multipart uploader over the real client, and NewProviderWithClients wires a single PutObject over the injected StorageService, which keeps the existing tests exercising that seam unchanged. qemu-img now writes the qcow2 compressed (-c). The image is mostly empty space, so this is the difference between minutes and tens of minutes of upload on an ordinary connection, and Object Storage imports a compressed qcow2 exactly like an uncompressed one. * oci: give the image upload a timeout sized for a part Splitting the upload into parts was not enough on its own: the SDK's HTTP client applies its timeout to the whole request, sending the body included, and defaults it to 60 seconds. No part of a useful size meets that on an ordinary uplink, all the more so with several parts sharing it, so the upload still died, one part further along: Put ".../u/<image>?uploadId=...&uploadPartNum=3": context deadline exceeded (Client.Timeout exceeded while awaiting headers) The uploader now gets its own Object Storage client whose timeout is sized for a part, leaving the client every other call uses at the SDK default, and an explicit OCI_CUSTOM_CLIENT_TIMEOUT still wins. Parts drop to 16 MiB, which makes a retry cheaper. It stays a bound rather than an absence of one: a part that stops making progress fails instead of hanging forever. * oci: honour the configured cpu and memory of a flexible shape A flexible shape does not carry its cpu and memory in its name, and CreateInstance hardcoded one ocpu and one gigabyte for every *.Flex, a "hack as we don't have a system for 'flex' today", as the comment put it. On OCI's Ampere shapes that is the smallest slice the shape allows, and nothing in the configuration could change it, so anything that needs more than a gigabyte of memory could not be launched at all. Ocpus and MemoryInGBs are now read from the cloud configuration, keeping the previous 1/1 as the default so nothing changes for a configuration that does not set them.
Fix local package regexp for packages with hyphen in name (#1596) Currently there is a bug where because the regexp is liberal enough to allow either _ or - to delimit the break between name and version, a package with dash will fail. So for instance, a package like my-package_1.2.3 Will have a name of "my" and a version of "package_1.2.3". Not wanting to change this behaviour by simply changing the regex to only accept _ as a delimeter, I have changed the regex to split at the *last* match.
re-add support for TFS version 4 encodings This commit returns support for reading and optionally writing TFS version 4 images. 'ops image' commands that read an existing image will read either version 4 or 5 images, and commands that create images may either accept a "-4" or "--tfsv4" commandline option or a "TFSv4" config bool to create a version 4 image. This support is intended to help ease the transition towards use of TFS version 5.
PreviousNext