cki.cki_tools.sync_files

Sync files to S3 bucket

This script reads files from a given URL and stores them at the given target path in the given S3 bucket spec.

Environment variable Required Default Description
BUCKET_CONFIG_NAME Yes Name of environment variable with S3 bucket spec to use as a backing store
SYNC_FILES_CONFIG Yes yaml configuration with a file_list key holding a list of source_url and target_path mappings
SYNC_FILES_CONFIG_PATH Yes Path to yaml file if not provided above
SENTRY_DSN No Sentry DSN

Example SYNC_FILES_CONFIG yaml:

file_list:
  - source_url: https://example.com/file1.txt
    target_path: 'file1.txt'
  - source_url: https://example.com/file2.html
    target_path: 'file2.html'

CKI_DEPLOYMENT_ENVIRONMENT

On staging developments (CKI_DEPLOYMENT_ENVIRONMENT != production), the source URLs are downloaded but not uploaded to the S3 bucket.