For downloading data from minio to your container to populate your PV
Parameters are read from the following environment variables
MINIO_URL= address and port for the minio server. Does not need scheme.ACCESSKEY= Minio access keySECRETKEY= Minio secret keySRC= Source of the files you wish to copy to your container.
Takes form ofbucketname/path/fileorbucketname/pathfor all contents in a folderDEST= Path to where you wish data to inside your container
Special note for the DEST paramert. If you are downloading all files from a dir, don't include that dir name in your DEST Value. For example. If you were downloading all the contents from users/roles do not make your destination /data/users/roles. Do make it /data/users. The roles directory will be created as part of the download.
Because the access key and secret key should not sit in plane text in a git repo or ConfigMap. These credential will be stored in Kubernetes as a Secret
Kubectl create secret generic minio-creds --from-literal=accesskey=youraccesskeyhere --from-literal=secretkey=yoursecretkeyhere Include this in your current deployment. Modify the env values to match your environment.
The name of the container can be changed to whatever you wish
initContainers:
- name: data-loader
image: jpweber/minio-init-dl
env:
- name: MINIO_URL
value: "minio.k8sdev.example.com"
- name: ACCESSKEY
valueFrom:
secretKeyRef:
name: minio-creds
key: accesskey
- name: SECRETKEY
valueFrom:
secretKeyRef:
name: minio-creds
key: secretkey
- name: SRC
value: "training-data/foo/"
- name: DEST
value: "/data"
volumeMounts:
- mountPath: /data
name: db-storageAssumes you already have a mount named db-storage in your main container.
Set your variables like the example below
export MINIO_URL="192.168.2.66:9000"
export SRC="vectorizer/roles"
export ACCESSKEY="LEB3JJ3OCBN4HTDIS5IZ"
export SECRETKEY="xs3fx83cMkV7Oh+6jlGGTt9kTmT5D6yoQLm9+L5X"
export DEST="/tmp/minio-tests/examples"Then you can run the binary with out any extra parameters
docker run -ti -v /tmp:/tmp \
-e MINIO_URL="192.168.2.66:9000" \
-e ACCESSKEY="LEB3JJ3OCBN4HTDIS5IZ" \
-e SECRETKEY="xs3fx83cMkV7Oh+6jlGGTt9kTmT5D6yoQLm9+L5X" \
-e SRC="vectorizer/roles" \
-e DEST="/tmp/minio-tests/examples/" \
jpweber/minio-init-dl:0.2.4