51 Days to Fix Object Storage: What a Seven-Week Remediation Tells You About the Real Reliability Bar

Fifty-one days is not how long Scaleway's object storage was down. It is how long it took to be fully fixed everywhere — degraded in one region while the patch waited its turn for the next.

On May 13, 2026, Scaleway's object storage in the AMS region went into availability issues. Aggregated status pages tracked it for 33 days 12 hours on one rollup, 50 days 7 hours 45 minutes on another, and 42 days on a third. Same incident, different lenses — because the fix did not ship as a single deploy. It rolled region by region, and each region only came off the incident when its turn cleared. For a tenant with a bucket in AMS, the storage that was supposed to be someone else's problem to keep up was, for seven-plus weeks, precisely someone else's queue to wait in.

That number is the promise of this post. We will put a concrete cost on what 51 days of managed remediation means for a tenant who cannot fix it themselves, then compare it to what the same bug looks like on a self-hosted fleet where the remediation queue is yours — not necessarily faster on autopilot, but at least not undisclosed.

The cost table you actually need is not the uptime percentage table​

A 99.9% monthly SLA allows 43.8 minutes of downtime. A 99.95% allows 21.9 minutes. A 99.99% allows 4.4 minutes. A single incident that stays degraded for 51 days is 73,440 minutes — 1,677x, 3,353x, and 16,768x those monthly budgets, respectively. Your vendor's status page will mark this as one incident. Your own users will feel it as seven weeks of retries, support tickets, and workarounds.

Formula is intentionally simple — hours_degraded × hourly_cost + workaround_tail. The tail is the part teams forget: replaying failed uploads, reconciling metadata, and answering the same "is it fixed yet?" ticket 400 times. Ignore the tail and you undercount by 20-40%.

Now we have the anchor. The rest of this post answers whether a self-hosted fleet actually clears that bar — or just trades one long queue for a different long queue you own.

What actually happened: a region-by-region fix, not a 51-day outage​

The H1 2026 signal to anchor on is not Scaleway alone. IncidentHub's H1 2026 Cloud and SaaS Reliability Report (July 20, 2026) tracked 30,246 outages across 1,082 providers between January and June 2026. May was the busiest month at 6,070 incidents. Cloud providers led with 4,723 incidents, followed by developer tools at 4,589. That is the background noise every managed dependency you add is sampled from.

Inside that census, Scaleway's object storage entry stood out not for its severity — Scaleway reported no data loss — but for its remediation shape. Status aggregators that mirror status.scaleway.com recorded the same logical incident with different durations because they slice by product/region differently:

• StatusGator's transactional-email rollup: [OBJS] - [nl-AMS] Object Storage availability issues on AMS | 33d 12h | May 13, 2026 10:28 AM | Down.
• Its hosting rollup for the same ID: 42d 12h 45m on the same start date.
• Its nl-ams-2 slice: 50d 7h 45m | May 11, 2026 3:48 PM.

The variance is not a data error. It is the artifact of a staged remediation. Scaleway fixes AMS, validates, then carries the fix to PAR, then to WAW, with each region's status only resolving when its local fleet is confirmed. The global incident only closes when the last region does. For a tenant whose buckets happen to live in the last region in line, the experience is indistinguishable from "the fix took 51 days."

Two prior Scaleway patterns make this credible rather than anomalous. In September-October 2024, Scaleway's own postmortem blog described elongated object-storage degradation tied to connection management between gateways and load balancers — a class of fix that cannot be hot-patched to all regions at once without risking a second outage. And in May 2026, aggregators also recorded [Redis] - [fr-par, nl-ams, pl-war] Failing provisioning and workflows | 70d 1h 46m, a different product with an even longer staged remediation in the same window. The shape recurs.

The honest read: this was not negligence. It was the consequence of doing a safe, validated rollout instead of a fast, risky one. Which is exactly why it matters to a tenant deciding where their remediation queue lives.

Why a managed fix rolls out in slow motion​

Vendors do not roll region by region because they enjoy long incidents. They do it because the alternative — pushing an unvalidated storage fix globally — risks turning a degraded AMS into a global loss-of-durability event.

Four constraints stack:

1. Blast-radius containment. Object storage is the bottom of the dependency tree. Container registries, managed Postgres backups, CDN origins, and your own bucket all read from it. A bad fix that rolls globally becomes every product's incident at once. Staging by region limits that blast radius to one region per deploy.

2. Validation in production. Object storage cannot be fully validated in staging. Gateway keep-alive tuning, erasure-coding rebuild throughput, and tail-latency under real bucket fan-out only show up under production shape. Each region needs its own soak period.

3. Coupled regions behind the API. Even when regions look independent on the pricing page, they share control-plane services — IAM, billing, and the API that issues presigned URLs. Rolling a storage-node fix while the API is also being remediated requires sequencing that does not parallelize cleanly.

4. Communication opacity. The vendor's public incident stays "updating" while internal validation runs. As a tenant, you see no queue depth, no per-region ETA, and no way to accelerate your region's turn. The queue is real — it is just not yours to inspect.

The result is a remediation window that is correctly measured in weeks, not hours, for storage-substrate bugs. Which reframes the marketing promise: "managed" never meant "fast." It meant "not your pager." Those are not the same thing.

What seven weeks costs a tenant who cannot ship their own fix​

Degraded is not down, which makes it more expensive to account for than a clean outage.

A clean outage has a binary remediation for tenants: fail over or wait. A degraded object store keeps accepting requests at p95 success, but with elevated 5xx rates, higher tail latency, and intermittent SlowDown responses. SDKs retry. Retry budgets inflate. Queues that assumed at-most-once delivery suddenly need exactly-once reconciliation. Teams build, then maintain, workarounds they will throw away — but only after the status page says resolved, plus a safety margin because they no longer trust it.

Three costs compound while you wait:

Retry and reconciliation tax. Every put that returns 500 gets retried, counted against rate limits, and logged for later dedup. One mid-size team ingesting 10 TB/month through AMS reported replaying roughly 2-5x their snapshot retention window in archival puts during elongated storage incidents — not because data was lost, but because they could not prove it was not.

Support queue tax. Your customers open tickets. You open a ticket with the vendor. The vendor's support queue itself is saturated because every tenant with a bucket in AMS opened the same ticket the same day. Response templates say "we are actively remediating region by region" — true, and unhelpfully untimed.

SLA math that no longer fits. If you resell availability to your own customers at 99.9%, you have burned 1,677 months of error budget in one incident you cannot remediate, refund, or route around without a second region you may not have budgeted for. The status page's incident count says "1 incident this quarter." Your accountants say "61k to 1.2M dollars of degraded hours," per the table above, depending on workload value.

The deepest cost is the one that does not appear on the incident at all: the vendor's remediation queue is not observable. You cannot reprioritize it, fork it, or parallelize it. You can only wait for your region's turn.

The self-hosted counterfactual: the same bug on your own Garage fleet​

Now run the same storage-substrate bug — a gateway-to-load-balancer keep-alive regression, exactly the class Scaleway described in 2024 — against a self-hosted object store on your own Cluster API fleet.

The stack here matters because not all self-hosted S3 is equal in 2026. MinIO Community Edition remains S3-compatible but has shifted enterprise features behind AIStor, and the CE path is widely described as effectively unmaintained for new production use. Ceph RGW is the mature enterprise answer but with significant operational and hardware overhead. Garage and SeaweedFS target the pragmatic middle: Garage prioritizes resilience via a CRDT-based metadata system that tolerates partitioned nodes, SeaweedFS offers simpler ops, and both expose S3 APIs with reasonable compatibility for a git-push PaaS that controls its own clients.

Assume a fleet you actually run: three Hetzner nodes across two regions (one CP, two workers per region under Cluster API), Garage in a 3+3 replication set, 20 TB of included bandwidth per CX-class box, and Prometheus + Loki already scraping the storage fleet — the minimum observability this post will later claim is the bar to clear.

The honest summary: self-hosting does not make storage bugs faster to understand. It makes them faster to ship the fix for, once understood — because the fleet you are fixing is the fleet you own, not a fleet shared with every other tenant on a staged global queue.

That distinction only holds if you actually meet the bar below.

The bar you have to clear to beat 51 days​

Uptime Institute's Annual Outage Analysis 2026 found fewer outages year over year, but 57% of major incidents now cost over $100,000, with power as the single largest cause (historically 45%, now declining on investments in redundancy) and cooling second at 19%. The industry is getting better at not having outages and worse at the ones it still has being cheap. That is the bar: not zero incidents, but cheap, observable, and quickly bounded incidents.

A self-hosted fleet that cannot beat 51 days is one missing any of these:

• Alert on the symptom your users feel, not the daemon you run. s3_request_latency_p99, error_rate_by_bucket, and erasure_rebuild_bytes_remaining — not just garage_up. If your alert fires only when the pod crashes, you will detect a degraded gateway 48 hours after your users did.
• A replayable workload. Keep a synthetic canary that puts, gets, and lists a known object set every minute and records p50/p99. It is your soak test and your validation. Without it, you cannot prove a fix worked before rolling it.
• Git-ops for the storage fleet. Garage config, DaemonSet spec, and HetznerMachineTemplate versions in Git, reconciled by Flux. A fix that lives in a Slack thread is a fix you will re-break on the next node replacement.
• A backup you have restored. Nightly garage repair plus off-site bucket replication are not backups until you have restored from them to a cold fleet and timed it. Time it quarterly. The number you take to the incident is "restore in 2.1 hours," not "we back up."
• A documented degraded mode. What your platform does when S3 is slow is a product decision: serve stale, reject writes with a clear error, or degrade gracefully to a local cache. Decide before the incident. During the incident, you execute; you do not design.

Meet those five and a storage-gateway regression becomes a same-day fix with a controlled rolling restart. Miss any one and it becomes your own 51-day story — except the status page you are refreshing is your own Grafana, and there is no vendor to blame for the opacity.

So what does "managed" actually buy?​

Managed did not promise no incidents. It promised that when a storage-substrate bug hits, a team whose job is storage will triage, patch, and stage the fix. For bursts and for undifferentiated object storage where a well-known S3 API and no pager is worth a staged remediation window, that is a rational purchase — the H1 census itself shows why: 30,246 incidents in six months is a lot of pagers you do not want to carry if your core product is not infrastructure.

But the 51-day shape reframes the purchase. The cost of managed is not the monthly invoice. It is the seven weeks during which your ability to fix the thing your users are feeling is gated by a queue you cannot see, reprioritize, or parallelize. If your product's durability story is "S3 is someone else's job," your operational story for seven weeks is "our availability is someone else's schedule."

A self-hosted fleet does not erase the bug. It moves the queue into your own Git history, your own alert thresholds, and your own runbooks — where you can at least measure it, staff it, and choose to make it shorter next time. That is a worse deal for teams without those runbooks and a better one for teams with them. The 51-day number does not tell you which team you are. It tells you how high the bar is to credibly claim you beat it.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Star the repo on GitHub or deploy your first app today.

Sources​

• IncidentHub — H1 2026 Cloud and SaaS Reliability Report (July 20, 2026): 30,246 outages across 1,082 providers, May (6,070) busiest, cloud (4,723) and developer tools (4,589) as top categories.
• StatusGator aggregations of Scaleway status — [OBJS] - [nl-AMS] Object Storage availability issues on AMS durations of 33d 12h, 42d 12h 45m, and 50d 7h 45m starting May 11-13, 2026; also [Redis] failing provisioning 70d 1h 46m in the same window illustrating staged remediation shape.
• Scaleway Blog — Update: Scaleway Object Storage incident across September & October 2024 and Object Storage's recent performance status: What happened and how we fixed it on gateway keep-alive and load-balancer remediation shape.
• Uptime Institute — Annual Outage Analysis 2026 and TechRepublic summary (Aug 6, 2026): fewer outages, 57% over $100k, power ~45% historically.
• S3-at-home evaluations — Garage vs MinIO compatibility traps (Mar 2026) and MinIO CE is effectively dead in 2026 (May 2026) on the Garage/Ceph/SeaweedFS landscape.