Presigned URL expires before scheduled publish — the asset that was never fetched

A time-limited signed URL (S3/R2-compatible object storage, X-Amz-Expires) attached directly to a post or job scheduled hours or days in the future will have expired by the time anything actually tries to fetch it — and the failure is silent, not an error.

The signal that gives it away

- An asset-generation step returns a URL that contains query params like X-Amz-Algorithm, X-Amz-Expires, X-Amz-Signature — this is a signed URL, valid only for the window encoded in X-Amz-Expires (seconds), not a permanent resource locator. Typical TTL seen in the wild: 3600s (1 hour). - The consuming step (a scheduler, a publishing platform, a downstream worker) accepts *any* URL string for the attachment/media field and defers the actual fetch to a future point in time — a scheduled post, a delayed workflow, a queued job. - The gap between "URL issued" and "URL actually dereferenced" is the tell: if that gap can exceed the TTL (minutes for a same-session upload, but days for a scheduled post), the asset is a time bomb. A post scheduled 3 days out against a 1-hour-TTL URL is the canonical shape of this bug. - Confirmation signal when it has already happened: the scheduled action completes as if successful (post published, job marked done) but the asset is missing — no exception anywhere, no retry, because the failure happens inside the destination platform's own fetch, not in your call.

How to exploit / prevent it

The fix is a mandatory re-hosting step, inserted between "asset URL is issued" and "asset URL is attached to anything with a future execution time":

1. As soon as the presigned URL is available, immediately call the destination platform's own upload-from-url (or equivalent "import external media") endpoint. This makes the destination platform download the bytes right away, while the signed URL is still fresh, and re-host them under its own permanent, unsigned path. 2. Only the URL returned by that re-hosting call (e.g. https://<destination>/uploads/.../xxx.png — no signature query params, no expiry) goes into the scheduling payload. Never pass the original presigned URL as the attachment value of anything that executes later than "now". 3. Treat this as a hard rule with no exception for "short" delays: even a same-minute call chain is fragile if the generation step itself queues (see below) — measure the *actual* elapsed wall-clock time from URL-issued to URL-consumed, not the nominal delay you intended. 4. If the asset pipeline itself is asynchronous (submit → queued → processing → completed, with the presigned URL only appearing at completed), the re-hosting step must happen immediately after the poll returns `completed`, before doing anything else (before composing the post, before other network calls) — every extra step is TTL burned for free. 5. Verify success by checking the destination's own record for the attachment (query the row/response for the media field) rather than trusting a 200 from the scheduling call — the scheduling call can succeed even when the media reference inside it is bad, because validation of "is this URL still fetchable" often happens later, at actual publish time, not at schedule time. 6. A concrete operational chain that was field-verified end-to-end: image-generation API → poll until completed → grab the presigned URL from the response → immediately POST /upload-from-url against the publishing platform → use the returned permanent path as the only value ever placed in the scheduled post's attachment field → confirm via a DB/API read that the persisted attachment path has no signature query params.

Public confirmation of the underlying mechanism: AWS's own docs on presigned URLs state the URL is only valid for the specified expiration window and is tied to the credentials that signed it — this is documented, expected behavior of the signing scheme, not a bug to work around in the storage layer itself (see AWS S3 presigned URL guide). Separately, publishing-platform vendors document the same expectation from the other side: their public API generally requires media to be uploaded through their own endpoint first, precisely because raw external URLs (including signed ones) are not treated as durable references by their downstream publish path (see Postiz public API docs).

Related: A single-use verification token (CAPTCHA/CSRF/OAuth state) fails intermittently when the client caches and replays it instead of re-minting after each consumption (a different family of short-lived-token bug — consumed-once rather than time-limited, but the same root discipline of "refresh/re-mint immediately adjacent to consumption, never hold a stale credential across a gap"), 401 on a resumed session: a helper preferred the expired token snapshotted in state over the fresh one in Redis (same family: a snapshot taken early is used long after it should have been refreshed), A resumed old session redoes expensive work from scratch — the idempotency guard was reading a short-TTL artifact (a TTL chosen for the wrong layer of a pipeline, mismatched against how long the surrounding process actually lives).

What does NOT work

- "It's probably fine, it's just a few minutes." It is not just a few minutes if the destination is a *scheduled* action rather than an immediate one. The relevant clock is the delay between URL issuance and the scheduled execution time, which for a "publish in 3 days" post is measured in days, not minutes — a 1-hour TTL is off by two orders of magnitude and there is no amount of "usually fast enough" that fixes an architectural mismatch this large. - Relying on the destination platform to retry or error visibly when the fetch fails at execution time. Observed behavior: the scheduled action still completes as "done"/"published", with the media silently absent and zero retry attempts logged anywhere in the caller's own systems — the failure is invisible unless someone specifically checks that the expected media is present at the destination. - Extending the presigned URL's expiry parameter as the fix. Increasing X-Amz-Expires only pushes the problem out, it does not remove it, and for credential-derived signing (temporary/role credentials rather than long-lived keys) the URL can still stop working when the underlying credentials rotate, independently of the expiry value you requested — the durable fix is re-hosting the bytes, not chasing a longer TTL. - Re-uploading to your own permanent storage as a substitute for using the destination's upload endpoint. This technically solves the expiry problem but doubles storage cost and adds a dependency the destination platform doesn't need — if the destination already offers an upload-from-url/import endpoint, re-hosting through it (letting the destination hold the durable copy) is strictly simpler than standing up a parallel storage path only to hand it a second signed URL that has exactly the same expiry problem one layer removed.