Cloud Storage

Customer-owned, S3-compatible object storage with offload organized into four layers: a driver (the byte backend), one shared engine + lifecycle (the offload orchestration and admin flow), visibility (public vs private, a property of the store), and a per-consumer profile (which table, what an object-per-row looks like, which visibility). The public-files consumer is the first consumer; new consumers add only a profile.

Overview

Each Joinery instance can be configured with one S3-compatible account (AWS S3, Backblaze B2, Cloudflare R2, Wasabi, DigitalOcean Spaces, MinIO, etc.). The platform offers two stores on that one account: a public store (world-readable by key) and a verified-private store (a separate bucket, proven non-public before any byte lands). Public uploads (photos, gallery images, blog images) are asynchronously moved to the public store; the customer carries the storage cost rather than the platform.

Uploads themselves are unchanged — they always land locally first. A scheduled task pushes eligible files to the store on the next cron tick. The private store ships inert until an admin configures and gates a private bucket; the public files consumer never uses it (private files stay on local disk).

Architecture

Upload arrives → UploadHandler → File row created (fil_storage_driver='local')
                                                     │
                                                     ▼
                                    Cron tick (every 15 min)
                                                     │
                                                     ▼
                            CloudStorageSync iterates eligible rows:
                              - public per is_public()
                              - fil_storage_driver = 'local'
                              - fil_sync_failed_count < 5
                                                     │
                            Push original + variants concurrently
                            Re-check is_public()
                              ├── still public → flip flag to 'cloud',
                              │                  delete local copies
                              └── went private → undo bucket pushes,
                                                 leave row at 'local'

The per-row fil_storage_driver flag is the source of truth. A misconfigured global setting cannot strand existing files because each row independently records where its bytes live.

Unified offload architecture

The orchestration is table-agnostic. Three pieces in includes/cloud_storage/ do the work for every consumer:

• CloudOffloadEngine — syncBatch(profile) (local → cloud) and reverseBatch(profile) (cloud → local). It owns the bounded batch, the per-row advisory lock, the failure-count cap, and the ordering invariants: forward pushes → reloads → flips the row to cloud → only then unlinks local bytes; reverse *pulls to temp → commits the row to local → only then best-effort deletes from the bucket. It is visibility-blind — it resolves its driver from forVisibility($profile->visibility()).
• CloudStorageLifecycle — the admin save/test/activate/health flow, parameterized by store (visibility) and profile. It holds the per-visibility setting bindings and the two integrity guards (below).
• StorageProfile — the per-consumer seam. It declares the table, the pkey/driver/failed-count/last-attempt columns, the visibility, an eligibilityWhere() SQL gate, and the per-row object enumeration (itemsForRow for forward, reverseItemsForRow for pull-back — the latter computed from the row's key scheme without needing local bytes, since on pull-back none exist yet). FileStorageProfile is the public-files adapter over the existing File methods.

The (endpoint, bucket) identity of a store is immutable while that store holds any cloud row (summed across every profile of that visibility). A Save that would change the bucket or endpoint of a store with offloaded objects is rejected: pull them back to local first (Disable and Pull Files Back to Local). Access-key rotation — same (endpoint, bucket) — stays allowed. Activating the forward task deactivates the reverse task and vice versa, so a row can never ping-pong between stores.

Declarative profile registry

Profiles are declared, not self-registered at runtime, so the registry and the immutability guard see them whether or not the owning plugin is active — matching how the platform declares plugin settings and menus:

• Core profiles are listed in storage_profiles.json at the public_html/ root; the class file lives at includes/cloud_storage/<ClassName>.php.
• Plugin profiles are listed under a storage_profiles key in the plugin's plugin.json; the class file lives at plugins/<plugin>/includes/<ClassName>.php.

<site_template>/<filename>            ← original
<site_template>/<size>/<filename>     ← variants (thumb, avatar, ...)

• Pause Cloud Storage — disables the feature and deactivates the sync task. Existing cloud-stored files keep serving from the bucket. Click Save to re-enable.
• Disable and Pull Files Back to Local — same as Pause, plus activates CloudStorageReverseSync to pull all bucket-stored files back. Confirmation dialog shows the count of files and free local disk space.

Backwards-compatible /uploads/ redirect

Pre-migration /uploads/<filename> URLs (in sent emails, search index caches, RSS feeds, embedded HTML) keep working: serve.php's /uploads/* route checks fil_storage_driver and 302-redirects cloud rows to the bucket URL with Cache-Control: public, max-age=86400. After the first hit the browser caches the redirect; subsequent hits skip PHP entirely.

Bucket Policy / Setup

Backblaze B2

1. Create a bucket; set its type to Public.
2. Create an application key scoped to that bucket with read+write+delete.
3. The endpoint is s3.<region>.backblazeb2.com (region matches the bucket's region: us-west-002, us-west-004, etc.).
4. Front the bucket with Cloudflare for free egress via the Bandwidth Alliance: - Add the bucket's domain to a Cloudflare zone. - CNAME a custom domain (e.g. images.example.com) to the bucket hostname. - In the Joinery admin, set cloud_storage_public_base_url to https://images.example.com.

AWS S3

1. Create a bucket; disable "Block all public access" for the s3:GetObject policy you'll add.
2. Apply a bucket policy granting public read on the prefix:

      {
        "Version": "2012-10-17",
        "Statement": [{
          "Sid": "PublicReadGetObject",
          "Effect": "Allow",
          "Principal": "*",
          "Action": "s3:GetObject",
          "Resource": "arn:aws:s3:::YOUR-BUCKET/*"
        }]
      }

3. Create an IAM user with a policy granting read/write/delete on the bucket; use its access key and secret in the admin form.
4. Endpoint: s3.<region>.amazonaws.com. Region matches the bucket region.
5. Strongly recommended: front the bucket with CloudFront or another CDN to keep egress costs reasonable.

Cloudflare R2

1. Create an R2 bucket.
2. In the bucket's settings, attach a public custom domain or enable the public bucket URL. R2 is free for egress.
3. Endpoint: <account-id>.r2.cloudflarestorage.com.
4. Region: auto.
5. cloud_storage_public_base_url should be the public custom domain you attached.

Provider Compatibility

Failure Modes

File-by-File Architecture

Out of Scope (v1)

• Per-tenant / per-user buckets within a single Joinery instance.
• Bucket-level encryption configured via the admin UI (customer responsibility at bucket creation).
• Resize-on-demand (variants still generated upfront).
• Automatic CDN setup (customer responsibility, documented above).
• Egress monitoring or alerts based on actual bytes served.
• Storing public-files' private/permissioned variants in any bucket. The public-files consumer keeps permissioned files on local disk. (The platform's verified-private store exists for other consumers that declare visibility = private; the public-files profile never uses it.)