PepiteData Julien Laurenceau
  • Home
  • Services
  • Audits
  • Expert Call
  • About
  • Blog
  • Contact

performance

Two warehouses linked by conveyors illustrate S3A rename copying objects to new keys and deleting the originals.
apache spark

Spark Output Committers on S3, MinIO and Ceph: Stop Committing by Rename

S3A committers avoid copying Spark output during commit, but they do not publish a dataset atomically. This guide shows how to configure and verify them in upstream Spark, what changes on Amazon EMR, and how to protect readers and concurrent writers.

By Julien Laurenceau, 3 weeks ago
A data file at a switching yard where scan, random-access, object-storage, geospatial and time-series paths diverge
data

Beyond Parquet and Avro: Choosing a Data File Format by Access Pattern

Lance, Vortex, Nimble, FastLanes, GeoParquet, cloud-optimized geo formats and new time-series encodings each optimize one access pattern on one storage tier. A field guide to what their published numbers measure, where tuned Parquet still wins, and how to choose.

By Julien Laurenceau, 4 weeks ago
A large server bank feeds a narrow processing gate, with work queuing before it, under the headline Bigger bill. Same bottleneck.
data

Performance and Cost Engineering for Data Platforms

A field guide to performance and cost engineering: claim-checking with the scaling and queueing laws, measurement discipline, where GPU and CPU budgets leak, and how to turn a benchmark into a hardware decision.

By Julien Laurenceau, 2 months2026-09-30 ago
An open duct in a stone chamber holding a single file of evenly spaced glowing cubes, one entering and one leaving
data

Benchmark Best Practices, Part 2: Little’s Law Ties Throughput, Latency and Concurrency Together

Second in my benchmark best practices series. Little’s law checks that throughput, latency and concurrency describe the same system, exposes the load generator’s ceiling, and turns a latency budget into a sizing requirement.

By Julien Laurenceau, 2 months2026-09-22 ago
Two MinIO sites locked in a closed replication loop: delete markers stamped 405 circulate endlessly between two racks, piling up instead of converging
MinIO

MinIO site replication bug: how 684 million HTTP 405s exposed the delete-marker loop

HEAD probes on delete markers answer 405 and replication counts that as success. 20M stuck versions turned the loop into 684M useless requests in 48 h.

By Julien Laurenceau, 2 months2026-09-06 ago
A balance scale holding a small solid block on one pan and an oversized hollow sphere on the other
data

Benchmark Best Practices, Part 1: Amdahl’s Law Caps Your Scaling Claim

Part 1 of my benchmark best practices series: how I turn a scaling claim into an implied serial fraction with Amdahl’s law, and what the Universal Scalability Law adds. Includes a measured pipeline where assigning more vCPUs per image cut global throughput.

By Julien Laurenceau, 3 months2026-08-25 ago
The same abstract point-cloud terrain packaged as many small tiles or a few larger containers
object storage

Why File Count Matters as Much as File Size on Object Storage

Compression, object population and request count measure different things. Compare point-cloud packaging, range reads, update costs and cleanup with an explicit synthetic sizing example.

By Julien Laurenceau, 3 months2026-08-22 ago
Isometric illustration of a distributed edge computing network of glowing server and sensor nodes, with one node shaped like a retro video game controller, on a dark navy and cyan background
data

Which Message Broker for IoT and Real-Time Control

Choose a message broker for real-time IoT control: start from the latency budget, test ack mode and packet loss, and when to skip the broker for UDP.

By Julien Laurenceau, 3 months2026-08-22 ago
An acknowledged object remains at the source while a disconnected peer has no copy
MinIO

MinIO Site Replication: Sync Mode Will Not Save Your RPO

A successful source PUT does not establish a recoverable peer copy. Separate MinIO site-replication acknowledgment, persisted retry state and catch-up capacity before committing to an RTO or RPO.

By Julien Laurenceau, 3 months2026-08-03 ago
Two panels. Top: a single MinIO bucket overflowing with small files, a worried Tux, and a dead directory tree on a tombstone. Bottom: the same objects split across four prefix-partitioned buckets under separate folders, with a healthy green directory tree.
MinIO

MinIO on XFS: Inode Exhaustion and Prefix Design for Lots of Small Files

MinIO has no NameNode. It puts the namespace on XFS as real directories and xl.meta files. On Lots of Small Files that means you can exhaust inodes while df -h still looks fine, and a flat leaf can stall PUT, LIST, scanner, and ILM together. Here is the on-disk model, the inode math, and a prefix recipe that keeps XFS inside a regime you can operate.

By Julien Laurenceau, 4 months2026-07-27 ago

Posts pagination

1 2 Next
  • Privacy Policy
  • Mentions légales
  • CGV
  • Cookies
Hestia | Developed by ThemeIsle