Layered Ceph isolation model with realms, zonegroups, zones, tenants and accounts

Ceph Tentacle does not introduce RGW tenants or accounts. Tenants have existed since Jewel. Accounts arrived in Squid. Tentacle clarifies the direction of travel by deprecating tenant-level IAM in favor of accounts. That distinction matters if you run a multi-tenant S3 service and plan to change its identity model during the upgrade.

An RGW version upgrade and an identity migration are separate changes. The first changes binaries and service behavior. The second can transfer bucket ownership, change effective permissions and alter how notification topics appear through the API. I would never combine them in one production window.

Start with the correct RGW hierarchy

The terms sound interchangeable until a failure exposes the boundary. They are not interchangeable.

LayerWhat it controlsWhat it does not provide
RealmThe RGW multi-site replication domain and its periodsApplication-level tenant isolation
ZonegroupA group of zones, endpoints and supported multi-site featuresAn IAM or billing boundary
ZoneAn RGW deployment that stores and serves dataA customer account
TenantNamespace separation for users and buckets with the same namesA first-class IAM organization
AccountAccount-owned resources, IAM users and roles, aggregate usage and quotasA replacement for realms, zonegroups or zones
RGW topology and RGW identity solve different problems. A safe design keeps those boundaries explicit.

Ceph documents tenant isolation as a namespace feature introduced in Jewel. Every user and bucket belongs to a tenant, including the empty legacy tenant. Accounts add a different model. Resources created by account users belong to the account, appear to other authorized users in that account and contribute to account-level statistics and quotas.

What changes when a user joins an account

The Ceph account documentation lists consequences that belong in the migration plan, not in release notes that nobody reads during the change window.

  • Bucket ownership moves. Adopting an existing user into an account transfers ownership of all that user’s buckets to the account.
  • Membership is permanent. Ceph states that a user cannot be removed from an account after adoption.
  • Permissions change. Account users start with no permissions by default. You must add identity policy before the migrated user can resume normal API operations.
  • Notification topics need separate work. Existing topics keep working, but ownership does not transfer. The user can lose visibility through the SNS Topic APIs until topics and bucket notification references are migrated.
  • Names are validated differently. IAM user naming rules apply during adoption. A display name that worked for a legacy RGW user can fail account migration.
  • Tenant placement still matters. A tenanted account can contain only users from the same tenant.

These are data-model changes. Replication health alone cannot prove that the new ownership and policy model is correct.

Separate the Tentacle upgrade from account adoption

A queue of abstract shapes waits at a door labeled tenant while the door labeled account stands unused
Tenant and account are different identity doors; users queue at the wrong one unless the model is planned.

I would first complete the binary upgrade using a canary, cross-zone S3 tests and explicit rollout gates. I cover that process in Upgrading a Ceph Multi-Site S3 Platform from Squid to Tentacle. Only after every zone is stable would I begin account adoption as a second project with its own rollback boundary.

Staged multi-site upgrade order with DR and secondary zones before the primary zone
Finish and validate the version rollout before changing the identity model. Mixing the two removes the cleanest diagnostic boundary.

A migration sequence that keeps evidence

  1. Inventory tenants, users, buckets, ACL owners, IAM policies, quotas, notification topics and bucket notification configurations.
  2. Create the target account in the correct tenant. Record its account ID and root user.
  3. Build the identity policies that the adopted user will need. Do not wait for the migration to reveal missing rights.
  4. Rehearse the procedure with a representative non-production user, including bucket listing, object access and notification delivery.
  5. Adopt one low-risk production user. Confirm that bucket ownership moved to the account and that the account can see the expected resources.
  6. Apply the planned identity policies and run client-level S3 acceptance tests.
  7. If notifications are used, migrate topics and bucket references according to Ceph’s procedure. The notification_v2 zone feature must be enabled where required.
  8. Verify metadata and data sync from every zone, then hold the next migration until the observation window closes.

The gates I would put in the runbook

GateEvidence to collectStop condition
InventoryUser, bucket, policy, quota and notification exportAn owner or dependency cannot be mapped
OwnershipBucket owner and ListBuckets results before and after adoptionUnexpected bucket visibility or ACL owner
AuthorizationPositive and negative API tests using real client credentialsA required action fails or a forbidden action succeeds
NotificationsTopic list, bucket configuration and delivered canary eventA topic becomes invisible or delivery duplicates
Multi-siteMetadata sync, data sync and cross-zone client testsLag exceeds baseline or account metadata diverges
Each gate needs client-visible evidence. A healthy Ceph status is necessary, but it is not an identity acceptance test.

The design decision comes before the command

Accounts are the better foundation when you need delegated IAM, account-owned resources and aggregate quotas. Tenants still matter when you need namespace separation. The hard question is how those boundaries map to customers, business units, billing and operational ownership. Decide that mapping before adopting the first user, because account membership is permanent and ownership changes propagate across the service.

If you are planning a Ceph RGW upgrade or account migration, I can review the topology, identity model, acceptance tests and rollout gates through a Flash Architecture Review or Data Platform Audit. Book a 15-min intro call to discuss the platform and the scope.

Primary documentation

Update 2026-09-29. This post is now part of Ceph in Production: RBD, RGW and CephFS, the field guide to Ceph in production.

Related posts

Categories: ceph

0 Comments

Leave a Reply

Avatar placeholder

Your email address will not be published. Required fields are marked *