Dora NodaMinIO's GitHub repository went read-only in early 2026 — maintenance mode through most of 2025, then a full archive, ending a project with 60,000 stars and over a billion Docker pulls. Every post-mortem about that shutdown asked the same question: which S3-compatible bucket API replaces it. That's the wrong layer to be asking about if you're running a self-hosted PaaS, because buckets sit on top of something more fundamental — the block storage underneath every tenant's volumes. And in the same window MinIO was winding down, Ceph's Tentacle release quietly changed the economics of that layer in a way that actually matters for a Cluster-API-managed fleet.
The Capacity Math, Worked Out
Ceph has always offered a trade a lot of self-hosted platforms skip: erasure coding instead of straight replication. The default in most Kubernetes storage setups — including Longhorn, the popular choice for Cluster API-managed fleets — is 3x replication: write a block, copy it to two more nodes, and you can lose two copies before you lose data. That's simple and fast, but it costs 3x the raw capacity for every byte you actually keep.
A 6+2 erasure-coded Ceph pool (six data chunks, two parity chunks, tolerating any two failures) does the same job for less. Its overhead factor is (k+m)/k — for 6+2, that's 8/6 ≈ 1.33x. Put a number on that: eight nodes running Hetzner's AX102 line (twin 1.92 TB Gen4 NVMe drives each, the spec sheet a Cluster-API fleet would actually provision) gives you 30.72 TB of raw NVMe. Under 3x replication, that's 10.24 TB usable. Under 6+2 erasure coding, it's 23.04 TB — 2.25x the usable capacity from identical hardware. For tenant volumes on a space-constrained fleet, that's not a rounding error; it's the difference between needing eight nodes and needing eighteen.
The reason this math hasn't mattered more in practice is that erasure coding has historically been slow for exactly the workload a PaaS's tenant volumes generate: small, random block writes. That's what Tentacle's FastEC just fixed.
What FastEC Actually Changed
Before Tentacle (v20.2.0, released November 2025), any write to an erasure-coded pool — even a 4 KB write to a single block — had to read the full stripe, recompute parity across every chunk, and write the whole thing back. For RBD (block) and CephFS (file) workloads, where writes are small and frequent, that read-modify-write tax made EC pools noticeably slower than replicated ones for exactly the I/O pattern tenant volumes produce.
FastEC replaces that with partial reads and partial writes: a write now touches only the affected chunks, computes an XOR delta between old and new data, and applies that delta to parity instead of recomputing it from scratch. Ceph's own benchmarks put the result at 2-3x faster erasure-coded I/O for small operations — the specific case that used to be EC's weak point. Tentacle also switched the default EC plugin from Jerasure to Intel's ISA-L for faster CPU-accelerated encoding, on top of the algorithmic change.
That's the honest upside. The honest balance is that FastEC doesn't touch recovery. When an OSD fails, EC pools still have to read k surviving chunks from k different disks, decode the missing piece, and write it to a replacement — more IOPS and more cross-node traffic than a replicated pool's copy-the-surviving-replica recovery. A 6+2 pool's new-object writes also fan out across all 8 OSDs holding data and parity, versus 3 for a replicated pool. FastEC narrows the day-to-day performance gap; it doesn't erase the wider blast radius an EC pool touches on every write or every rebuild.
The partial-write mechanic is worth being specific about, because "partial" is doing real work in that sentence. For a two-parity-chunk pool (m=2), a small in-stripe update now costs 3 reads and 3 writes per client request — the changed data chunk plus both parity chunks, each updated via an XOR delta rather than recomputed from every chunk in the stripe. For m=1, it drops to 2 reads and 2 writes. That's a fixed, small cost regardless of stripe width, which is exactly why FastEC's win concentrates on small I/O: the bigger the stripe, the more the old full-stripe RMW used to cost, and the more a fixed 3-read/3-write path saves by comparison.
Not a Future Roadmap Item — It's in Rook Now
The obvious follow-up question for anyone running Kubernetes-native storage: does this reach a Cluster-API fleet through Rook, or is it stuck upstream in raw Ceph for another year? It's already there. Rook's own tracking issue to make Tentacle v20 the default Rook version was opened November 20, 2025 — nine days after Tentacle shipped — and closed via two merged pull requests, landing in Rook v1.19. A CephBlockPool resource on a Rook-managed cluster today can set allow_ec_optimizations: true in its spec the same way it always could set any other pool parameter, alongside the erasureCoded: {dataChunks: 6, codingChunks: 2} block that defines the profile itself. One tuning detail worth carrying into that spec: Ceph's own guidance recommends bumping stripe_unit to 16 KB for FastEC-enabled pools, since the flag requires 4 KB-aligned chunks and the wider stripe unit gives the partial-write path more room to avoid touching neighboring data. This is available to provision on a fresh cluster today, not something to wait for on a v5-style roadmap.
The Gotcha: How Not to Enable It
Here's the part a capacity spreadsheet won't show you. On January 11, 2026, at 2:48 AM PST, Clyso — a Ceph consultancy — fielded an emergency support call for a cluster where OSDs were crashing and CephFS had gone fully inaccessible. The root cause, per their public post-mortem: the operator had enabled allow_ec_optimizations on a CephFS erasure-coded pool that had been upgraded from a legacy pre-Tentacle EC pool, without allow_ec_overwrites already set on it beforehand. Objects written under the old EC code carry padding the new fast path doesn't expect. The crash landed in ECTransaction::WritePlanObj, hitting a transaction key that simply didn't exist for those legacy-padded objects.
The damage was real: 9 OSDs down, 340,449 scrub errors, and a cluster-wide CephFS outage before the team recovered it. And there's no undo button — once allow_ec_optimizations is set on a pool, Ceph doesn't let you turn it back off.
The lesson isn't "don't use FastEC." It's specific: enabling it is safe on a pool created fresh after upgrading to Tentacle, and it's a real landmine as a same-day flip on an existing production pool with legacy-EC history behind it. If your fleet's Ceph pools predate November 2025, plan a migration to new EC-optimized pools rather than toggling the flag on what you already have. Rook's Tentacle default (above) means new clusters get this right automatically; upgraded clusters need to get it right on purpose.
The Verdict, for a Cluster-API Fleet's Tenant Volumes
Put the pieces together and the answer to "does erasure-coded Ceph belong as the default over 3x-replicated Longhorn" is yes, for the workload that actually matters here — but with a real condition attached, not a blanket swap.
For space-constrained tenant storage — the common case on a bare-metal or Hetzner-class fleet where every extra node is a real monthly cost, not an autoscaled abstraction — the 2.25x usable-capacity multiplier from 6+2 EC is large enough on its own to justify the added operational complexity, and FastEC closes enough of the historical small-I/O performance gap that RBD-backed tenant volumes no longer pay a punishing tax for it. Longhorn keeps its place for smaller clusters (1-3 nodes, where an 8-OSD erasure-coded pool isn't even a valid topology) and for workloads where write latency matters more than capacity efficiency — its simpler 2-3 replica model has less write fan-out and a more forgiving recovery path. But for a multi-node fleet built to run many tenants' volumes on hardware you're paying for by the terabyte, Rook-managed Ceph with FastEC enabled on freshly created pools is the more honest default in 2026 than it was a year ago — provided nobody flips the flag on an existing pool without reading the pool's own history first.
That's a real reversal of the conventional wisdom around Ceph on small self-hosted fleets. "Ceph is overkill unless you're running hundreds of terabytes" has been true for years, mostly because the 3x-replication tax on capacity and the pre-FastEC EC performance penalty combined to make Longhorn's simplicity the obviously correct trade below a certain scale. FastEC doesn't change that calculus for a single-node or two-node deployment — EC needs k+m failure domains to mean anything, so the floor for a viable 6+2 pool is genuinely eight nodes. But once a fleet crosses that floor, the argument for staying on replicated storage out of caution rather than genuine workload fit gets noticeably weaker than it was in 2025.
It's worth naming the connection back to where this post started. The same year MinIO's Community Edition went dark, Ceph's RGW — built on these same erasure-coded pools — remained one of the few S3-compatible object stores that didn't lose its community edition out from under its users. The block-storage story and the object-storage story aren't separate questions on a self-hosted fleet; they're the same erasure-coded pools serving two different APIs on top.
Bex.co is the open-source, AI-native Render alternative — a Cluster API control plane that provisions and manages the storage layer underneath your apps' volumes, not just the compute. Star the repo on GitHub or deploy your first app today.
Sources:
- Ceph.io — v20.2.0 Tentacle released
- Ceph.io — Fast Erasure Coding for Tentacle Performance Updates
- Ceph Documentation — Erasure Code
- Post Mortem: Tentacle v20.2.0 OSD crashing due to EC Bug — Clyso GmbH
- Rook GitHub Issue #16747 — Make Ceph Tentacle v20 the default Rook version
- Rook GitHub Issue #16788 — allow_ec_optimizations on pool upgrade
- MinIO Is Dead, Long Live MinIO — Pigsty
- Hetzner AX102 Dedicated Server specifications
