Docs / Spaces

Sync with rclone or s3cmd

10 minute read

rclone and s3cmd are handy for backups, migrations from another provider and scheduled syncs from a server. Both need the Spaces endpoint, an access key and path-style addressing.

1. Create a key for the job

Use a separate key for each machine or job, such as backup-cron, so you can revoke one without breaking the others. Create it under Spaces object storage, Access keys, New key.

2. Configure rclone

Add a remote called simplicloud to the rclone config file. Run rclone config file to see where yours lives.

[simplicloud]
type = s3
provider = Other
access_key_id = SCJ6VBS8HFDUAXPVSYMT
secret_access_key = <your secret>
endpoint = https://spaces.simplicloud.ai
region = lagos1
force_path_style = true
acl = private

• •rclone lsd simplicloud: lists your buckets.
• •rclone ls simplicloud:kora-backups lists objects in one bucket.

3. Configure s3cmd

If you prefer s3cmd, put the same details in ~/.s3cfg. Both host settings point at the endpoint because Spaces uses path-style addressing.

[default]
access_key = SCJ6VBS8HFDUAXPVSYMT
secret_key = <your secret>
host_base = spaces.simplicloud.ai
host_bucket = spaces.simplicloud.ai
bucket_location = lagos1
use_https = True
signature_v2 = False
# then
s3cmd ls s3://kora-backups
s3cmd put backup.tar.gz s3://kora-backups/servers/

4. Run a nightly sync

Copy a folder from a server to a private bucket every night. Use copy if files should stay in Spaces after you delete them on the server; use sync to mirror exactly.

# 03:00 every night, after database backups finish
0 3 * * * rclone copy /var/backups simplicloud:kora-backups/ops-box --transfers 8 --log-file /var/log/rclone.log

5. Move data from another provider

Add the old provider as a second rclone remote and copy bucket to bucket. Run it from a SimpliCloud server so the upload stays inside Lagos.

rclone copy oldcloud:shop-images simplicloud:kora-media --progress --checksum
rclone check oldcloud:shop-images simplicloud:kora-media