Data Infrastructure Audits, Due Diligence and Capacity Planning

What triggers a review

Before a commitment

• You are investing in, acquiring or buying into a company whose value depends on a technical platform.
• You are about to commit to a strategic data, AI or infrastructure product and want the claims tested against evidence.
• You need to size a new platform and choose servers, instances or storage before the budget is signed.
• A migration, a re-platforming or an architecture redesign is about to start, and the assumptions have not been challenged from outside.
• A vendor has proposed a configuration and nobody internally can verify whether it is right, generous or short.

Under load

• Traffic, data volume or model training demand is about to increase by a large factor and you need to know what breaks first.
• Infrastructure cost is rising faster than usage and the drivers are spread across several teams.
• Your disaster recovery plan has never been tested under realistic conditions, and the stated RTO and RPO have to survive a real failover.

When the platform is the constraint

• Spark jobs are slow, unstable or expensive.
• Kafka pipelines are hard to operate at scale.
• Ceph, MinIO, Cassandra or the storage layer produce latency or reliability you cannot predict.
• The data platform is holding back AI or analytics work.

How I investigate: every layer, one review

I do not scope a review to a single product, a single software layer or a single team. Depending on the question, the investigation moves through:

• application source code, algorithms and data models;
• distributed processing engines, databases and messaging systems;
• Kubernetes, container orchestration and scheduling;
• networking and data paths;
• object, block and distributed storage;
• operating system and infrastructure configuration;
• server architecture, drive specifications and measured hardware behavior;
• the capacity plan and the vendor proposal behind the purchase.

I can move from reading a Spark job’s source to challenging the drive specification in a vendor quote in the same engagement. That range is the point. Reviews that stop at a layer boundary produce incomplete diagnoses, optimizations that relocate the bottleneck, overprovisioning bought as insurance, redesigns that were never necessary, sizing set by whoever is selling the hardware, and months of trial and error while five teams hold five plausible theories.

What I deliver

Data Platform Audit: From €7,500 excl. VAT

Typical range: €7,500 to €15,000 excl. VAT, depending on scope, urgency and system complexity.

A full review of a complex data, cloud or AI platform where performance, scalability, reliability or cost has business impact. This is the engagement for cross-system problems, for platforms about to take a large step up in load, and for decisions that need more evidence than a single component review can produce.

Typical deliverables: architecture map; bottleneck analysis; reliability and scalability risks; infrastructure cost and capacity model; quick wins; prioritized roadmap; executive debrief for CTO, VP Engineering or Head of Platform.

Technical Due Diligence: From €4,500 excl. VAT

An independent technical read for investors, buyers and leadership teams before investing in a company, acquiring shares, validating a vendor platform or committing to a strategic data and AI infrastructure product.

Typical scope: architecture and codebase review where available; scalability, reliability and infrastructure risk assessment; technical debt and operational maturity; the cost curve implied by the current design at the projected scale; vendor or target-company claims tested against evidence; written risk memo with go and no-go considerations and prioritized follow-up questions.

Best for: investors who need technical diligence before a transaction, executives evaluating a platform-dependent acquisition, and teams committing to a product where performance, lock-in, reliability or cost would materially change the business case.

Capacity Planning and Platform Sizing: From €4,500 excl. VAT

Evidence-based sizing for a new deployment, a capacity uplift, a migration or a re-platforming decision.

This is one of my strongest areas. I size open source components (Kafka, Cassandra, ScyllaDB, Ceph, MinIO, PostgreSQL, Kubernetes, Spark, Pulsar) against the real workload, then translate the result into concrete hardware: the right instance types from a public cloud catalog, or the right server and drive configuration from an OEM catalog such as Dell or HPE. Where a vendor has already proposed a configuration, I check it against what the components actually need and what the hardware actually delivers.

Typical scope: target workload analysis; sizing of compute, storage and network; server and instance selection from cloud or OEM catalogs; capacity headroom recommendations; cost estimate against the chosen infrastructure, on-premise, private cloud or public cloud; written sizing document and debrief session.

Flash Architecture Review: From €4,500 excl. VAT

One critical decision, one bottleneck or one system component, reviewed in depth and answered quickly.

Typical scope: architecture review; technical interviews; metrics, logs and configuration review where available; written diagnosis; prioritized recommendations; debrief session with the CTO, tech lead or platform team.

Resilience and Disaster Recovery Assessment: From €4,500 excl. VAT

A technical validation of whether your platform can actually survive the failures it is designed to survive, and whether it can come back inside the window the business has been promised.

Typical scope: failure scenario and blast radius analysis; high availability and fault tolerance design review; replication topology, consistency and lag under partition; backup integrity and measured restore throughput against real data volumes; dependency ordering during recovery; recovery testing practice; a reachability verdict on the stated RTO and RPO with the gap analysis behind it.

Before implementation, this catches the design choices that put the target out of reach. After implementation, it verifies that the plan on file describes the system that is running. Recovery plans fail on details that only appear when someone tests them: replication that silently fell behind, a restore path that is throughput-bound, a runbook that assumes a control plane which is part of the outage.

Cost Optimization Review: From €4,500 excl. VAT

A technical review of what your platform costs and why, tied to the engineering choices that produce the bill.

Cost in a data platform is rarely owned by one team. Executor sizing, Kubernetes quotas, storage layout and replication factor, retention policy, GPU allocation, idle capacity, data transfer and vendor pricing all contribute. I review the drivers with the technical evidence behind each one, so you can separate waste from headroom you are right to keep, and act on the difference.

Typical scope: cost allocation against workloads; right-sizing opportunities with the measurement behind them; storage layout, replication and retention economics; scheduling and utilization analysis; commitment and vendor pricing review; a ranked list of reductions with effort and risk attached.

Expert Call: €250 excl. VAT / 1h

A focused video session on one architecture, performance, scalability or cost question. Best for validating a direction, challenging a proposal, or getting an external read before a larger decision. It is a standalone product, not a step you have to buy before an audit.

Fractional Advisor: From €3,000 excl. VAT / month

Recurring senior review for teams operating complex data infrastructure. A few hours a month of independent architecture challenge, incident analysis and roadmap input, without hiring a full-time principal engineer. The introductory call for this engagement is free.

Who this is for, and who it is not for

This is for: investors, acquirers, CTOs, CIOs, Heads of Platform Engineering, Heads of Data, VP Engineering and SRE leaders who are evaluating, sizing, scaling or operating production data, cloud and AI infrastructure where performance, reliability, resilience, technical debt or cost has measurable business impact.

This is not for: staff augmentation, single-contributor freelance work, generic cloud strategy, or engagements under €5,000 of scope. If the question fits in an hour, the Expert Call is the right entry point and I will say so on the introductory call.

Systems I review

The review focuses on the systems that usually decide performance, capacity and cost in mature data platforms.

Spark on Kubernetes

I review executor sizing, CPU overcommit, shuffle behavior, autoscaling assumptions, queueing, namespace limits, pod failures, object-store access patterns and job timelines. The goal is to separate Spark tuning problems from Kubernetes scheduling, storage or network bottlenecks.

Object storage, Ceph and MinIO

I look at latency, throughput, S3 operation rates, erasure coding, disk layout, network paths, metadata pressure, small-file behavior, lifecycle rules and failure domains. Object storage often looks healthy at the bucket level while Spark, AI training or analytics workloads suffer from request amplification and unstable tail latency.

Cassandra and ScyllaDB

I review data models, partition size, compaction, tombstones, JVM and GC behavior, repair practices, node balance, latency percentiles and incident history. For operational databases, peak throughput matters less than predictable latency. The important question is whether the platform can keep predictable latency under real production load.

Kafka, private cloud and AI infrastructure

I examine broker sizing, retention, consumer lag, network topology, storage contention, GPU utilization, batch scheduling and capacity allocation. In private-cloud environments, the same physical bottleneck can affect Spark, Kafka, object storage and AI workloads at the same time.

Signals I examine

A useful review starts from evidence. I typically work through job timelines, Prometheus and Grafana dashboards, CPU and memory profiles, JVM metrics, S3 latency and throughput, I/O patterns, network paths, pod restart history, scheduler events, queueing, Cassandra compaction statistics, GC pauses, incident postmortems, capacity plans, vendor quotes and cloud or private-cloud cost allocation.

I also review the team’s own hypotheses. Many platform teams already know part of the answer, but they lack time, external distance or cross-system evidence. The review turns those signals into a clear diagnosis and a prioritized set of decisions.

Deliverables

• Architecture map: the systems, dependencies, data paths and ownership boundaries that matter for performance and reliability.
• Bottleneck analysis: the constraints that explain slow jobs, unstable latency, rising cost or reliability incidents.
• Risk register: reliability, scalability, resilience, operability and vendor-risk items that leadership should track.
• Sizing and cost model: compute, storage, network and capacity assumptions with cost impact where data is available.
• 30/60/90-day roadmap: prioritized fixes, quick wins, decisions to postpone and changes that need deeper engineering work.
• Executive summary: a clear version for CTO, VP Engineering, Head of Platform, investor or buyer discussions.

What this is not

This is not staff augmentation, a generic cloud migration project, vendor implementation, a digital transformation program or a long freelance mission. The work is diagnostic and decision-oriented. It helps you decide what to build, what to buy, what to fix and where engineering time has the highest return.

What I need from you

The best inputs are architecture diagrams, metrics dashboards, configuration excerpts, incident reports, workload samples, cost reports, capacity plans, vendor proposals and interviews with the engineers who operate the platform. Production access is often unnecessary. Good evidence and direct technical conversations are usually enough to find the main constraints.

Selected client work

A non-exhaustive list of past engagements. I name some clients with their permission and anonymize the others.

• BPCE-IT: Cassandra performance and reliability work supporting an API Management platform. “His expertise and skills have helped improve the performance and behavior of our platforms.”
• Multi-petabyte private cloud data lake: independent audit of a large on-premise data lake supporting analytics and AI workloads
• MinIO and Ceph at scale: performance and observability work on production object storage
• Spark on Kubernetes: performance and reliability engineering on private-cloud Spark platforms
• Geospatial / satellite imagery pipelines: data infrastructure design for high-volume image processing

References available on request under NDA.

Why me

I hold a PhD in numerical optimization and high performance computing, and I have more than 20 years of hands-on production experience in distributed systems, data platforms and private cloud.

I have worked on multi-petabyte data lakes, Spark on Kubernetes, Ceph and MinIO object storage, Cassandra and ScyllaDB, satellite imagery pipelines, banking infrastructure, telecommunications and aerospace systems.

I sell no implementation, no licenses and no hardware, and I work alone. You get one senior opinion, accountable and unhedged, with nothing downstream depending on the conclusion.

Typical outcomes

Depending on context, clients use the review to:

• Size a platform against the workload it will actually carry
• Reduce processing time
• Lower infrastructure cost
• Identify storage or compute bottlenecks
• Validate that recovery targets are reachable
• De-risk a migration or a scale-up
• Improve production reliability
• Challenge vendor, target-company or integrator recommendations
• Surface technical risks before an investment, acquisition or strategic platform commitment
• Prioritize engineering work
• Avoid over-engineering and unnecessary cloud spend

Common engagement scenarios

Most teams do not call because one dashboard looks bad. They call when several systems interact badly, when a decision is close and the evidence is thin, or when the internal diagnosis is no longer obvious. These are the patterns I see most often.

A platform has to absorb a large increase in load

Data volume, traffic or training demand is about to grow by a factor nobody has tested. The question is not whether the platform works today. It is which component saturates first, at what point, and whether the answer is configuration, layout, capacity or architecture.

I model the growth against the real workload, find the first three constraints in order, and give you the sizing and the sequence: what to change before the load arrives, what to buy, and what will hold longer than the team expects.

A vendor proposal needs an independent check

You have a quote for servers, storage or a managed platform, and internally nobody can verify it. Vendor sizing is rarely wrong on purpose. It is built from generic assumptions, sized for the vendor’s risk rather than yours, and quoted against specifications that describe the hardware under conditions your workload will not reproduce.

I check the configuration against what the components actually need and what the equipment actually delivers under your access pattern: drive characteristics, controller and network limits, erasure coding overhead, replication factor, headroom. The output is a position you can take into the negotiation.

Spark jobs are slow, unstable or expensive

A Spark problem can come from code, partitions, shuffle, file layout, Kubernetes limits, object storage, autoscaling, queueing or a mix of all of them. I review job timelines, stage metrics, executor behavior, pod scheduling, storage latency and data layout together. This avoids tuning one layer while the real constraint sits somewhere else.

The output is not a generic list of Spark best practices. It is a ranked diagnosis: which jobs create the cost, which bottlenecks are structural, which fixes are quick, and which changes require engineering work or platform decisions.

Object storage works, but analytics workloads suffer

Ceph, MinIO and S3-compatible platforms can look healthy from the storage dashboard while data workloads see unstable performance. Small files, request amplification, erasure coding, disk contention, network paths and metadata pressure can all create a gap between raw storage health and application performance.

I connect storage metrics to workload behavior. That includes S3 operation rates, tail latency, object size distribution, retry patterns, throughput by client, network placement and failure domains. The goal is to identify whether the platform needs layout changes, capacity changes, operational changes or application-side changes.

Cassandra or ScyllaDB latency becomes unpredictable

Latency incidents in Cassandra-like systems often come from a combination of data model drift, tombstones, compaction, repairs, node imbalance, JVM pressure, disk behavior and client access patterns. Average latency rarely tells the story. The important signals are percentiles, distribution, hot partitions, compaction backlog, read amplification and incident timing.

The review separates operational hygiene from architectural risk. Some issues need configuration or repair-process changes. Others point to a data model that no longer matches production traffic. Leadership needs to know which case they face before assigning months of work.

A recovery plan has never been tested

The document exists, the targets are written down, and the architecture diagram shows replication to a second site. Nobody has measured a full restore, failed over under load, or checked whether the recovery sequence depends on a system that would be down at the same time.

I work through the failure scenarios, the replication behavior, the restore throughput against real data volumes and the dependency order, then give a direct verdict on whether the stated RTO and RPO are reachable and what it would cost to close the gap.

Infrastructure cost rises without a clear owner

Cost problems in data platforms are rarely owned by one team. Spark sizing, Kubernetes quotas, storage layout, replication, retention, GPU allocation, idle capacity and vendor pricing all contribute. I review cost drivers with the technical evidence behind them, so the team can distinguish waste from necessary headroom.

The result is a cost model tied to engineering choices. That may show an immediate right-sizing opportunity, a storage-layout problem, a workload scheduling issue or a platform decision that needs executive attention.

A strategic decision needs independent technical due diligence

Investors, buyers and leadership teams often need a fast technical read before a platform commitment, acquisition, vendor selection or major roadmap decision. I focus on the evidence that changes the decision: scalability claims, operational maturity, hidden platform risk, dependency risk, cost exposure and the credibility of proposed fixes.

The deliverable is written for decision-makers, but grounded in technical review. It gives leadership a clear view of what is healthy, what is risky, what needs follow-up and which claims should be challenged before money or roadmap time is committed.

How I separate symptoms from root causes

I start by mapping the data path, not by assuming the problem sits in the component with the loudest alert. A slow Spark job may expose object storage latency. An object storage incident may come from network placement. A Cassandra latency spike may come from compaction pressure created by a product change. A GPU cost issue may come from pipeline scheduling, not from GPUs.

I look for cross-system evidence: timestamps that line up, workload changes, capacity limits, retries, queuing, tail latency, saturation, noisy neighbors and operational events. This makes the review useful even when the team already has strong engineers. The value is the independent synthesis across platform layers.

The final recommendations are ranked by business impact, implementation effort and risk. Some fixes are immediate. Some require a design decision. Some should be rejected because they add complexity without enough return. A good review should help you spend less time debating symptoms and more time making the right technical decisions.

What leaders can decide afterward

The engagement is built to support concrete decisions, not to produce a long report that nobody owns. Typical decisions include how to size the next platform, whether to accept a vendor configuration, whether to resize Spark workloads, whether to change object-storage layout, whether to adjust Kubernetes quotas, whether to postpone a migration, whether to fund a resilience workstream before adding new product scope, and how to split a platform roadmap.

For investors and buyers, the output highlights the technical risks that can affect valuation, integration cost, delivery timelines or operating margin. For CTOs and platform leaders, it clarifies where engineering time should go first and which ideas are expensive distractions.

How the work is scoped

A good scope is narrow enough to produce a clear answer and broad enough to avoid local optimization. I usually start with the business question: a platform about to scale, a sizing decision, a vendor proposal, a recovery target, a slow reporting chain, an unstable AI workload, an expensive Spark platform, a storage layer that behaves unpredictably, or technical due diligence before a transaction.

From there, I define the systems in scope, the evidence available, the people to interview, the decision deadline and the expected output. If the useful answer fits in an Expert Call or a Flash Review, I will say so. If the question is cross-system, the full audit gives enough time to connect workload behavior, infrastructure metrics, operational history and cost impact.

This keeps the engagement efficient. The goal is not to inspect every dashboard. The goal is to find the evidence that changes the decision.

How it works

1. Scope: Clarify the question, system boundaries, deadline and expected business impact.
2. Investigate: Review architecture, code, metrics, configuration, capacity plans, incidents, team hypotheses and constraints.
3. Prioritize: Rank constraints, risks, quick wins and high-return fixes.
4. Debrief: Deliver a written diagnosis and a prioritized technical roadmap, then walk your team through it.

Related technical notes

These notes show the kind of production issues a review can surface and prioritize:

• Spark on Kubernetes CPU overcommit and operator patterns
• MinIO performance with large numbers of small files (LOSF)
• MinIO site replication and what it actually does for RTO and RPO
• GPU infrastructure cost and overprovisioning strategies
• Data lakehouse operating models at scale

FAQ

Do you need production access?

Usually no. Metrics, architecture diagrams, configuration excerpts and technical interviews are often enough to start.

Can this be done remotely?

Yes. Most engagements are remote-first.

Do you work with clients outside France?

Yes. Engagements are remote-first and run in English or French. I work with clients across Europe and North America, and I schedule interviews and debriefs around the client’s time zone.

Can you review a vendor proposal or a capacity plan?

Yes. This is a frequent request. I check the proposed configuration against the workload it has to carry and against what the hardware actually delivers, and I tell you where it is short, where it is generous, and what evidence is missing.

Is this only for large companies?

No, but the decision or the problem must be large enough to justify a premium engagement.

Can you help implement the recommendations?

Yes, but the review is designed to stand alone. Implementation can be scoped separately.

Why not just hire a freelance for a few months?

Because you may not need more hands. You may need an independent diagnosis before spending months in the wrong direction.