Before a scale-up, an upgrade or a storage purchase, a Ceph cluster is judged by measurements, not impressions. As an independent Ceph consultant I audit and design distributed storage platforms: CRUSH maps and failure domains, RGW multi-site S3, RBD for Kubernetes PVCs, CephFS for shared file access, and sizing down to the vendor quote. I also intervene when a production cluster stops behaving predictably.
Proof point: Ceph and MinIO performance and observability on a multi-petabyte private cloud data lake; block storage latency optimization for a satellite imagery platform; multi-site S3 platform upgraded from Ceph Squid to Tentacle. References available under NDA.
Ceph cluster and storage audit
An independent review of an existing cluster: CRUSH map and failure domains, pool layout, erasure coding versus replication where it matters, OSD hardware classes and placement, monitor and manager health, RGW policies, RBD image features, release and security hygiene. The audit ends with a prioritized risk list and costed fixes, not generalities.
Block storage: RBD and Kubernetes PVCs
RBD performance work: latency attribution between network, OSD WAL and client limits, WAL and DB placement on the right media, image feature sets that do not silently cost throughput. For Kubernetes, the choice and operation of RBD as a PVC backend, including RBD mirroring for cross-site DR. The topic is covered in an article on choosing a PVC storage backend.
Object storage: RGW and multi-site S3
RGW multi-site replication design and validation, S3 acceptance tests before go-live, bucket and IAM policies, and upgrade paths that hold a production SLA. The Squid to Tentacle multi-site upgrade and the migration of unmanaged OSDs to managed OSDs document both in detail.
Performance and sizing
Measurements before commitment: network and NVMe profiling, bottleneck identification, erasure coding versus replication trade-offs against your real failure scenarios, 3 and 5 year sizing plans, and independent review of a vendor proposal. Every costed recommendation is confronted with your real workload, not a vendor benchmark.
This is not staff augmentation or development work: for a single technical decision, the flash architecture review is enough.
FAQ
Is Ceph a good backend for virtualization and Kubernetes PVCs?
Yes for block workloads that tolerate shared-nothing latency: RBD is the mainstream on-premises PVC backend, and mirroring covers cross-site DR. It earns its cost when the same cluster also serves object or file workloads; a pure block need is sometimes cheaper on dedicated storage. The trade-off deserves sizing before commitment.
Can a Ceph upgrade be done without downtime?
Yes within a release line: OSDs roll one failure domain at a time while clients keep serving. The constraints appear at major version jumps and in multi-site RGW topologies, where the ordering between sites is part of the plan. An upgrade is rehearsed against the cluster’s own topology, never against a generic runbook.
How much hardware does a petabyte of Ceph need?
It depends on the failure domain, the erasure coding profile and the network, which is why the honest answer comes from a sizing exercise rather than a rule of thumb: the same petabyte can vary by a factor of two in drives alone. I size against the real workload and translate the result into server and drive configurations or cloud instance types.