Skip to main content

Litestream's Writable VFS: SQLite Mounted From an S3 Bucket as the No-Managed-Database Persistence Path for a Self-Hosted PaaS

10 min readDora NodaDora Noda
Share
On this page

On February 4, 2026, Ben Johnson published a quiet bombshell on the Fly.io blog: Litestream's SQLite VFS — until then a read-only way to query a backup sitting in S3 — became writable. An application can now open a SQLite database whose pages live in an S3-compatible bucket, serve reads by fetching individual pages over HTTP Range requests, buffer writes locally, and sync them back to object storage roughly every second. No database server. No volume that has to follow the app around. The bucket is the database.

Here is the early verdict, so you don't have to dig for it: this pattern fits single-writer, restart-tolerant workloads that can accept a durability window of about one second — internal tools, per-tenant databases, agent state stores, read-heavy sites. It does not replace a real Postgres for concurrent multi-writer OLTP, and Johnson himself is the first to say so. But for a self-hosted platform that deliberately refuses to run managed databases, it is the most interesting persistence primitive to appear in years: tenant state lives in a bucket the tenant controls, and app nodes become genuinely disposable. The rest of this post walks the mechanism, the honest constraints, and a workload-by-workload decision table.

From Replication Sidecar to Mountable Filesystem

Litestream spent its first five years as a Unix-y sidecar: it watched a SQLite database's write-ahead log and shipped WAL segments to object storage, one way, for disaster recovery. Useful, beloved, and strictly a backup tool — restoring meant replaying every WAL frame since the last snapshot, which got painfully slow for write-heavy databases.

The path from that sidecar to a mountable filesystem ran through three releases:

  • v0.3.x (2021–2024) — classic WAL-segment shipping. One-way replication, full-database restores, the "backup/restore system SQLite forgot to ship."
  • v0.5.0 (October 2025) — the LTX file format replaced raw WAL segments with sorted, transaction-aware page changesets, plus LSM-tree-style hierarchical compaction: level-0 files upload every second, then compact into 30-second, 5-minute, and hourly windows. The payoff: point-in-time restore to any moment "using only a dozen or so files on average," instead of replaying millions of page writes. The release also swapped Consul-style external coordination for time-based leases built on S3's conditional writes — the coordination service is the bucket itself.
  • The read-only VFS (2025) — a loadable SQLite extension that skips restore entirely. Each LTX file carries a trailer indexing the offset of every page it contains; by fetching just the trailers (roughly 1% of total file size), the VFS builds a lookup table of every page in the database. A SELECT then translates into targeted Range requests against S3, with an LRU cache keeping hot B-tree branch pages local. It even does time travel: PRAGMA litestream_time = '5 minutes ago'; gives you the database as it existed before your bad deploy.

The writable VFS closes the loop. What started as one-way replication is now a mountable, near-stateless persistence layer: open a connection with vfs=litestream, and SQLite reads and writes against a bucket.

How a Write Actually Reaches the Bucket

The mechanism matters, because every constraint in the next section falls out of it.

Reads work as in the read-only VFS: the page index (those (file, offset, size) tuples reconstructed from LTX trailers) maps a requested page number to a byte range in an LTX file in the bucket. Cold pages cost an S3 round trip; hot pages come from the local LRU cache.

Writes never hit S3 synchronously. They accumulate in a local temporary write buffer, and the VFS syncs that buffer to object storage approximately every second, or on clean shutdown. Johnson calls this "eventual durability," and he is candid about the trade: an unclean crash can lose up to a second of acknowledged writes. The feature was built for Fly.io's Sprite storage stack, where every layer already shares that property — a cold-booted machine must serve reads and writes within milliseconds, long before a full database download could finish.

Hydration bridges cold start and steady state. With LITESTREAM_HYDRATION_ENABLED=true, the VFS serves queries from S3 immediately while a background job pulls the complete database to local disk; once hydration completes, reads flip to the local copy and the S3 round trips disappear. Since v0.5.9, companion .meta files let the hydrated copy persist across restarts instead of being discarded on every exit.

Single-writer enforcement is structural, not advisory. Enabling write mode (LITESTREAM_WRITE_ENABLED=true) disables the background polling that normally detects remote changes — the VFS assumes it is the only writer to that replica path. Johnson's name for distributed multi-writer SQLite is the "Lament Configuration," the puzzle box from Hellraiser that you really should not open. Litestream's lease mechanism, built on S3 conditional writes, exists precisely to guarantee one writer per destination without running Consul.

The full setup is two environment variables and one connection string:

bash
export LITESTREAM_REPLICA_URL="s3://tenant-bucket/app.db?endpoint=s3.us-west-2.amazonaws.com"
export LITESTREAM_WRITE_ENABLED=true
export LITESTREAM_HYDRATION_ENABLED=true
sql
ATTACH DATABASE 'file:app.db?vfs=litestream' AS app;
PRAGMA litestream_hydration_progress;  -- watch the background restore
PRAGMA litestream_lag;                 -- seconds behind the bucket

The extension ships as a loadable library on PyPI, npm, and RubyGems for Linux and macOS, so most stacks load it in one line.

The Honest Constraints

Johnson does not oversell this — his own post ends with "they probably don't make sense for your application. But if for some reason they do, have at it!" Here is what decides whether they make sense for yours:

ConstraintConcrete consequenceMitigation
Single-writer semanticsTwo app instances writing to one replica path will corrupt state; write mode disables remote-change detection entirelyLitestream's S3-conditional-write leases enforce one writer; scale reads with read-only VFS replicas, not writers
~1-second durability windowA kill -9 or node loss can drop up to a second of acknowledged writesClean shutdowns flush the buffer; don't put money movements or audit logs here
Page-fetch latencyEvery cold page is an object-storage round trip — and the round trip depends entirely on where the bucket is: a self-hosted MinIO or Garage instance on the same LAN answers in ~1–5 ms, same-region AWS S3 or Hetzner Object Storage in ~10–50 ms first-byte, cross-region S3 at 100 ms+. A query walking cold B-tree branch pages multiplies thatLRU cache keeps hot pages local; hydration eliminates remote reads entirely once complete; co-locate the bucket with the app
Cache warmingFirst queries after a cold boot pay the remote-read tax; pre-v0.5.9, hydration files were discarded on every exit, so every restart started coldv0.5.9 .meta files persist hydration across restarts; block maps stay small ("low tens of megabytes worst case")
Long-running transactionsThe two-index isolation strategy holds page versions in memory for the durationKeep transactions short — good SQLite hygiene anyway

The latency row deserves emphasis because it interacts with everything else: the same workload that is a poor fit against cross-region S3 can be a perfectly good fit against a Garage cluster one switch hop away. Backend choice moves the fit line as much as workload shape does.

What This Buys a Platform With No Managed Databases

Now the PaaS angle — why this pattern matters beyond Fly.io's internal storage stack.

Every hosted PaaS eventually funnels you into its managed database: Render's Postgres starts around $7/month per instance and climbs, Railway meters you per GB, Heroku's cheapest production-grade Postgres has been a budget line item for a decade. A self-hosted platform that deliberately declines to run managed databases — treating databases as tenant-run workloads, not platform features — has always had to answer: then where does state live?

The writable VFS gives that question a genuinely new answer: state lives in a bucket the tenant already controls.

  • AWS S3 at ~$0.023/GB-month plus ~$0.0004 per thousand GETs — a 2 GB SQLite database with modest traffic costs pennies per month.
  • Hetzner Object Storage at €4.99/month including 1 TB of storage — one flat line item covering the state of dozens of small apps.
  • Self-hosted MinIO or Garage on hardware you already pay for — zero marginal cost, LAN-grade page-fetch latency, and no third party in the data path at all.

Compare that to $7–20/month per managed database instance, multiplied across every side project, staging environment, and per-tenant deployment. The bucket model collapses that column of the invoice to nearly zero.

The operational win is just as large: app nodes become disposable. A redeploy, a node drain, a spot-instance reclaim — none of it requires volume migration or failover choreography, because the node never held the only copy of anything. The replacement pod opens the same LITESTREAM_REPLICA_URL, serves queries against the bucket immediately, and hydrates to full local speed in the background. For a platform built on Kubernetes and Cluster API, where machines are cattle by design, a persistence layer with the same property is the missing piece — stateful apps stop being the exception that pins pods to nodes.

Which Git-Push Workloads Fit — and Which Still Need Postgres

The decision table, with the constraint that drives each verdict:

WorkloadVerdictDeciding constraint
Internal tools, admin panels, dashboardsFitsSingle instance, low write rate; a lost second of writes means re-clicking a button
Per-tenant databases (one SQLite file per customer)Fits wellEach tenant is naturally single-writer; one bucket holds thousands of LTX trees
AI agent state stores (memory, task queues, run logs)FitsAgents are sequential writers; restart-with-full-state-from-bucket is exactly the agent-ops model
Read-heavy sites (blogs, docs, catalogs)Fits, hydration recommendedReads dominate; after hydration, queries are local-disk fast
Cron jobs and batch workersFitsClean shutdown flushes the buffer; no concurrency by construction
Multi-instance web backends (horizontal scaling)Does not fitSingle-writer semantics — the "Lament Configuration" is not on the menu
Payments, ledgers, audit trailsDoes not fitThe ~1 s durability window is disqualifying regardless of latency
Write-heavy OLTP (thousands of writes/sec, many clients)Does not fitBuffer sync cadence and single-writer ceiling; this is Postgres's home turf
Anything needing concurrent writers with row-level lockingDoes not fitRun a real Postgres — as a tenant workload, on the same platform

The pattern's honest scope, in one sentence: the long tail of small, single-writer databases — which is most databases most teams run — moves into a bucket; the few genuinely concurrent, strictly durable systems keep their tenant-run Postgres. The platform doesn't have to manage either one.

The Near-Stateless Endgame

Litestream's arc — sidecar, then LTX, then read-only VFS, then writable VFS — is one of the clearest signals of where self-hosted persistence is heading: object storage as the durability layer, local disk demoted to cache, and compute nodes that hold nothing worth mourning. S3-compatible storage is now the closest thing infrastructure has to a universal interface, from AWS to a €5 Hetzner bucket to a Garage cluster in your rack, and the writable VFS makes that interface speak SQLite.

Johnson's caveat stands: most applications should keep running classic Litestream as a sidecar, and concurrent systems should keep their Postgres. But for platform builders, the more important fact is that this exists at all, is open source, and works against any bucket. The no-managed-database stance stops being an austerity measure and starts being an architecture.


Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Bex deliberately offers no managed databases: patterns like Litestream's writable VFS are exactly how tenant state stays in a bucket the tenant controls while app nodes stay disposable. Star the repo on GitHub or deploy your first app today.


Sources

Related articles

Run this on infrastructure you own

bex is the open-source, AI-native Render alternative — push a git repo and get a running HTTPS service on your own machines.

Get started with bex