RustFS — Components Deep Dive

A study guide to RustFS, an open-source, S3-compatible distributed object store written in Rust: what each component is, the commands and settings you will actually use, when to use it, the traps, and where the docs live.

How to read this guide. Each component has (1) a mental model, (2) the code or config, (3) a when to use / when not to verdict, (4) traps and (5) 📖 doc links. The decision tables are in §2 (which deployment mode) and §11 (should I use RustFS at all). The sibling page Prefect — Components Deep Dive covers an orchestrator that can use an S3-compatible store for results and code. Version note: written 23 Oct 2026 from docs.rustfs.com. The S3 compatibility page was snapshotted by RustFS against a commit dated 9 Aug 2026, so treat counts as a moving target. ⚑ marks items from prior knowledge or inference rather than the pages I read (boto3 usage, exact Prefect wiring). Read §11 before adopting: this is a young project, so verify production readiness yourself.

0. The map: what RustFS is

Analogy: a warehouse of labelled bins with a front desk that speaks Amazon's language. Customers (applications) hand over boxes (objects) with a bin label (bucket) and a shelf label (key). The front desk (the S3 API) uses the same forms as Amazon's warehouse, so existing tools just work. Behind the desk, each box is split into pieces and spread over many shelves (disks) so a few broken shelves do not lose anything.

• What: a distributed object storage system in Rust, Apache 2.0 licensed, S3-compatible, aimed at private cloud, data lakes, AI/ML and cloud-native workloads. It runs on Linux, Windows, macOS, FreeBSD and containers across x86, ARM and RISC-V.
• Positioning from the project: the permissive licence is the headline contrast with AGPL-licensed alternatives; it claims migration and coexistence with S3-compatible platforms such as MinIO and Ceph. The README also claims 2.3x faster than MinIO for 4 KB payloads. That is a vendor benchmark; reproduce it on your hardware and object sizes before it informs a decision.
• Components: the rustfs server (S3 API on port 9000), the web Console (port 9001), the rc command-line client, optional protocol gateways (WebDAV, FTP/FTPS, SFTP, OpenStack Swift), background services (scanner for bit-rot, healing, lifecycle, replication), Kubernetes Helm chart and Operator, and an MCP server for AI assistants.
• Object-store vocabulary: an object is a blob plus metadata, addressed by bucket/key. Keys look like paths but the namespace is flat; 'folders' are just key prefixes.

Addressing. Path-style by default (http://host:9000/my-bucket/path/to/hello.txt). For virtual-host style (https://my-bucket.s3.example.com/hello.txt) set RUSTFS_SERVER_DOMAINS=s3.example.com and create wildcard DNS (*.s3.example.com) pointing at RustFS.

📖 Docs home · S3 protocol · Virtual-host access · GitHub

1. Installing it

1.1 Docker, the fastest start

docker volume create rustfs-data
docker run -d --name rustfs --restart unless-stopped \
  -p 9000:9000 -p 9001:9001 -v rustfs-data:/data \
  -e RUSTFS_ACCESS_KEY='<your-access-key>' -e RUSTFS_SECRET_KEY='<your-secret-key>' \
  -e RUSTFS_ADDRESS=':9000' -e RUSTFS_CONSOLE_ADDRESS=':9001' -e RUSTFS_CONSOLE_ENABLE=true \
  rustfs/rustfs:latest /data
curl --fail http://localhost:9000/health         # S3 API health; Console on :9001

• Change the default credentials immediately. Never use the well-known rustfsadmin value. On Linux set RUSTFS_ACCESS_KEY and RUSTFS_SECRET_KEY in /etc/default/rustfs and restart; with containers pass them at creation (recreate the container if it started without them; the volume survives).
• The container runs as non-root user rustfs, UID 10001. A bind-mounted host directory must be owned by 10001 (chown -R 10001:10001 /path) or you get permission denied. Named volumes need no setup.
• Flags win over env vars when both are given (--access-key, --address, --console-enable).
• Compose: the repository ships a docker-compose.yml with optional Grafana, Prometheus, OpenTelemetry collector and Jaeger under --profile observability; start only the server with docker compose up -d rustfs. An init container fixes volume ownership.
• Multi-node containers need --network host (the default bridge network does not support it) and name resolution (/etc/hosts on every node): RUSTFS_VOLUMES='http://node{1...4}:9000/data/rustfs{0...3}'.
• TLS: mount a cert directory and set RUSTFS_TLS_PATH=/certs. For HTTPS in front, the docs cover Nginx, Traefik, Caddy, Envoy, HAProxy and Apache with separate hostnames for S3 and Console.

📖 Installation · Docker · Linux · Kubernetes · Checklists · TLS · Reverse proxy

2. Deployment modes: which one?

Decision rule: if losing a server or its data would hurt, you need MNMD plus a backup or replication strategy (erasure coding protects against disk failures, not against deletion, corruption by software, or a lost site). SNSD is a development tool, not a storage strategy.

Disk preparation (Linux). Use dedicated data disks, format, mount and verify them before installing (the docs refer to XFS and a verification step); keep identical service environment configuration on every node. Work through the hardware, network, software and security checklists first.

📖 SNSD · SNMD · MNMD · Disk preparation · Hardware selection

3. Erasure coding and capacity

Analogy: a jigsaw with spare pieces. Each object is cut into N-M data pieces plus M extra 'parity' pieces, then spread across N disks. Any N-M of the N pieces rebuild the object, so up to M pieces can be lost.

• Reed-Solomon erasure coding. Each pool is divided into one or more erasure sets. For a set of N drives with M parity shards: data shards = N-M; tolerable unavailable shards = M; capacity efficiency about (N-M)/N. A 16-drive set with EC:4 stores 12 data and 4 parity shards, about 75% efficiency before file-system, metadata and operational overhead.
• Failure-domain caveat (from the docs): parity covers unavailable shards within one set. Node-level tolerance depends on how a set's drives are spread over nodes; do not assume EC:4 survives four whole nodes, because one node may hold several drives of the same set.

# /etc/default/rustfs
RUSTFS_STORAGE_CLASS_STANDARD=          # recommended: leave empty for automatic per-pool parity
# or pin explicitly (must be valid for the NARROWEST pool):
RUSTFS_STORAGE_CLASS_STANDARD=EC:4
RUSTFS_STORAGE_CLASS_RRS=EC:1           # REDUCED_REDUNDANCY writes; default 1 parity on multi-drive sets
RUSTFS_ERASURE_SET_DRIVE_COUNT=16       # pin set width only if you verified the whole layout

• Rules for explicit parity: M <= N/2 in every target set; STANDARD parity >= RRS parity when both non-zero; valid for every pool. An invalid value makes startup fail rather than silently weakening parity.
• Set widths range from 2 to 16 drives, selected automatically from the topology. Validate against the set width, not the cluster's total drive count.
• Capacity estimate: usable is roughly raw x (N-M)/N. Sixteen 10 TiB drives as one 16-drive set with EC:4 gives about 120 TiB, and you must still reserve space for metadata, versioning, incomplete multipart uploads, healing and headroom.
• Verify after start: Console Status page or rc admin info cluster shows layout, parity, drive availability and pool membership.

Trap. More parity buys durability at the price of usable capacity and write cost; do not pin high parity on small sets without checking the efficiency. Leaving the default is the documented recommendation unless you have a validated requirement.

📖 EC configuration · Status check · Storage env vars

4. Configuration surface

The reference page also lists variables for KMS, the scanner, healing and event targets. rustfs has subcommands and server flags with environment equivalents (CLI reference). Keep the same environment file on every node.

📖 Environment variables · CLI reference · CORS

5. The S3 API and compatibility

Analogy: speaking a dialect. RustFS speaks Amazon S3 well enough for most everyday conversation, but not every idiom. The project says so itself: it is a tested subset, not a claim of full coverage.

# AWS CLI against RustFS: just add --endpoint-url
aws --endpoint-url "$RUSTFS_ENDPOINT" s3 cp hello.txt s3://my-bucket/path/to/hello.txt
aws --endpoint-url "$RUSTFS_ENDPOINT" s3 ls s3://my-bucket/
aws --endpoint-url "$RUSTFS_ENDPOINT" s3api head-object --bucket my-bucket --key path/to/hello.txt
aws --endpoint-url "$RUSTFS_ENDPOINT" s3 presign s3://my-bucket/path/to/hello.txt

# Python (boto3)  ⚑
import boto3
s3 = boto3.client('s3', endpoint_url='http://localhost:9000',
                  aws_access_key_id=KEY, aws_secret_access_key=SECRET, region_name='us-east-1')
s3.upload_file('hello.txt', 'my-bucket', 'path/to/hello.txt')

• Gate size (Aug 2026 snapshot): 455 implemented cases, 5 lifecycle cases, 17 planned, 270 excluded, from Ceph s3-tests lists. Read the numbers as 'a lot is covered, but 270 are excluded', not as 'complete'.
• Encrypted objects are not portable. A passing encryption test means RustFS can read what RustFS encrypted; it does not mean it can read an encrypted object copied raw from another implementation.
• S3 Select currently accepts only uncompressed input.
• Other protocols: WebDAV, FTP/FTPS, SFTP and (optional build) OpenStack Swift gateways over the same buckets.

Practical advice: if you depend on ACLs, bucket access logging, or exotic multipart listing behaviour, test those exact calls against RustFS before committing. Use rclone or your SDK's integration test suite as a smoke test.

📖 S3 compatibility matrix · SDKs · Python · SFTP · WebDAV

6. Data management features

📖 Versioning · Object Lock · Lifecycle · Tiering · Quota · Bucket replication · Site replication · Scanner · S3 Tables · Event notifications

7. Security

Analogy: badges, keys and a locked cabinet. IAM decides who may enter which room (badges). Encryption keeps the contents unreadable even if someone steals a disk (locked cabinet). TLS protects the hallway.

• IAM: users, groups and policies; policy attachment and evaluation semantics are documented. Service accounts and STS issue scoped, short-lived credentials for applications. Access keys are managed per identity. Prefer service accounts or STS for applications and keep the root key out of code.
• Bucket policies are applied through the S3 API; public access block exists. ACLs are intentionally unsupported.
• OIDC: Console single sign-on with providers such as Keycloak and GitLab.
• Encryption at rest: SSE-S3 (server-managed keys), SSE-C (customer-provided AES-256 keys) and SSE-KMS with a local or HashiCorp Vault backend; the Operator can configure KMS for tenants. Note: the project README marks its KMS as 'under testing' while the docs describe KMS configuration, so validate your key-management path and recovery procedure before putting regulated data on it.
• TLS with RUSTFS_TLS_PATH; mTLS and cert-manager are documented for Helm deployments.
• Audit targets deliver request-level audit records to external systems. Credential management has its own page; rotate the root credentials.

Traps. Default credentials left in place. Exposing the Console (9001) to the internet. Putting RustFS behind a reverse proxy that rewrites the Host header without enabling the matching virtual-host settings. Losing the KMS key means losing the data.

📖 Security and compliance · IAM · Service accounts and STS · Encryption · OIDC · Audit · Credentials

8. Operations

• The rc client: a RustFS command-line client to connect to, inspect and administer a cluster, including lifecycle rules, replication, heal tasks and rc admin info cluster.
• Health and status: /health on the S3 port; the Console Status page; the status-check workflow covers network access, health endpoints, cluster status and capacity.
• Observability: export metrics, traces and logs over OTLP; the docs list recommended alerts and metric names, and integrations with Prometheus, Grafana, Loki, Tempo and Thanos exist.
• Scaling: expand by adding a storage pool; rebalance to redistribute existing objects over active pools; decommission to drain and retire a pool and verify the data moved. Pools are the unit of growth, so plan pool sizes and symmetry up front.
• Healing and failures: replace a failed drive and let RustFS heal the missing shards onto it; for a lost node prepare a replacement, update its configuration, restore cluster membership and watch healing. Manual heal tasks can be bounded with rc.
• Upgrades: binary, container and Kubernetes procedures that preserve availability. Rehearse on a non-production cluster first.
• Kubernetes: the Helm chart (standalone or distributed, probes, hardening, pool expansion) or the Operator (multi-tenant isolation, pool expansion, KMS).

Runbook items to write before go-live: disk replacement, node replacement, credential rotation, certificate renewal, capacity alert thresholds, upgrade order, restore test from backup or replica.

📖 rc client · Observability · Scaling · Node healing · Drive failures · Node failures · Upgrade · Cluster lifecycle

9. Integrations relevant to a data platform

RustFS documents step-by-step integrations (mostly Docker Compose based) for tools that speak S3. Grouped by purpose:

9.1 Using it with Prefect ⚑

Prefect can keep results and deployment code in S3-compatible storage through the prefect-aws integration. Because RustFS speaks S3, the wiring is the integration's S3 bucket block with a custom endpoint URL pointing at RustFS and a service-account key. I wrote this from how prefect-aws normally works, not from a tested RustFS example; check the prefect-aws docs for the current block and client-parameter names and run a write/read round trip before trusting it.

• Results: point result storage at a bucket such as prefect-results; every worker must reach the endpoint and credentials.
• Code: a pull step can fetch flow code from a bucket at run time.
• Hygiene: a dedicated service account with a bucket-scoped policy, a lifecycle rule to expire old results, TLS on the endpoint.

📖 Integrations index · Airflow · Iceberg · DuckDB · Spark · lakeFS · Terraform state · rclone · prefect-aws

10. Moving data in and out

• Migration and coexistence: the project says it supports running alongside other S3 platforms such as MinIO and Ceph. The practical tools are rclone sync, bucket replication (versioned buckets) and application-level dual writes.
• Validate a migration: compare object counts, sizes and checksums; test presigned URLs, multipart behaviour, listing order and any ACL or logging features you used, because those are the documented gaps.
• Do not copy encrypted objects raw between implementations (not portable); decrypt and re-encrypt through the S3 API.
• Exit plan: because the API is S3, leaving RustFS later is an rclone job, which is a point in its favour. Keep that tested.

11. Should you use RustFS? (the part to challenge)

Questions to answer before adopting: (1) Is the benefit the licence, the cost, performance, or Rust itself, and have you measured it? (2) Who is on call when a node dies at 3 a.m.? (3) Have you rehearsed a disk and node replacement and a restore? (4) What is the backup strategy beyond erasure coding? (5) How mature is the release you pin, and what is the upgrade policy? Pin a version rather than using :latest in production.

12. Trap checklist and exercises

1. Default credentials; Console exposed publicly.
2. Bind-mount directory not owned by UID 10001.
3. Believing EC:4 equals four-node tolerance.
4. Pinning parity that the narrowest pool cannot satisfy (startup fails), or setting a set width that does not divide the topology.
5. SNSD or SNMD treated as production, with no backup.
6. Replication configured without versioning; async replication assumed to be lossless.
7. Relying on ACLs, access logging or encrypted-object portability.
8. Using :latest; no upgrade rehearsal; no restore test.
9. Windows build used in production.

Exercises. (1) Start the Docker command with unique credentials, create a bucket and upload a file with the AWS CLI using --endpoint-url. (2) Enable versioning, overwrite the file, and recover the first version. (3) Compute usable capacity for 8 nodes x 4 x 8 TiB drives if sets are 16 wide with EC:4, and say what a node loss does to a set. (4) Add a lifecycle rule to expire objects under tmp/ after one day and verify. (5) Run a write/read round trip from a Python script with boto3 and a presigned URL.

Sources: docs.rustfs.com: docs home, installation, Docker, EC configuration, S3 compatibility matrix, S3 protocol, docs index; GitHub README (claims). Sections 6-8 summarise the documentation index descriptions rather than every page body; items marked ⚑ are unverified.