Your ML Platform Is Serving Itself

Is your ML platform becoming a bottleneck instead of an accelerator?

You set a budget for it, formed a special team to work on it, assigned it a name, created a roadmap, and provided it with a Confluence space containing seventeen pages of onboarding documentation. Yet at some point between the first internal demonstration and the fourth quarterly planning cycle, your ML platform simply ceased to serve the people for whom it had been built and began serving itself instead.

This isn't a skill problem. The teams building internal ML platforms are usually the strongest engineers in the company. The issue is structural in nature and gradually worsens until a senior data scientist decides to resign, a product deadline is delayed by three sprints, or finally someone speaks up and says out loud what all the others have been thinking: "It's quicker just to carry out this task outside the platform."

The Inversion Nobody Audits

The situation worth looking at is that internal ML platforms are generally tailored to the team that developed them rather than to those who use them. This is not intentional — it's more like a natural tendency. Platform engineers tend to design systems in accordance with their own mental models, operational limitations, and their idea of what is "correct". As a result, the platform ends up reflecting their priorities.

What starts as a well-intentioned abstraction layer — "we'll handle the infrastructure so data scientists can focus on modeling" — gradually accumulates:

• Mandatory approval gates for model registration that require platform team review
• Opinionated SDK wrappers that abstract away just enough to make debugging painful
• Standardized container templates that work perfectly for 80% of use cases and create week-long blockers for the other 20%
• Centralized feature stores with ingestion pipelines that only the platform team fully understands

The more enterprise-grade the platform becomes, the more it begins to look like the traditional IT ticketing systems that MLOps was meant to replace; you've simply exchanged one form of bureaucracy for a more attractive one.

The right question isn't "Does our platform have good coverage?" Rather, it is: Where does cognitive load go when a data scientist encounters a problem? If the answer is "onto the practitioner, not onto the platform", you have an inversion problem.

What the Metrics Won't Tell You

Usually platform teams assess adoption (the number of models registered), reliability (uptime and pipeline success rates), and velocity (deployment frequency). Such metrics are reasonable but also dangerously incomplete.

Consider what they miss:

• Time-to-first-experiment: How long does it take a new machine learning practitioner to carry out their first meaningful experiment on your platform? If the response requires reading a 40-page onboarding document, setting up three CLI tools, and waiting for the IAM permissions to propagate, then you're measuring the wrong thing.
• Workaround rate: The proportion of teams that carry out experiments or serve models outside of the platform since the platform's process is too slow or too inflexible is rarely monitored, and from my experience it is almost always greater than leaders suppose.
• Escalation frequency: The number of Slack messages per week that the platform team gets which begin with "quick question" but are in fact blockers is a leading indicator of abstraction failure.

A fintech engineering organization I spoke with found that their data science team was maintaining two parallel environments — the official one, used for anything that had to undergo a compliance review, and a more loosely controlled AWS account for everything else. The platform team had no knowledge of the shadow environment, and the data science team had ceased requesting new features, as they no longer expected them to arrive in time to be relevant.

It is worth being precise about what that story actually is. A shadow environment is a design signal, but it is also an audit finding. If any of that work touched regulated data or fed a model that made decisions about customers, the platform team's lack of visibility was itself the risk, however reasonable the data science team's motives were.

The point to establish before you begin eliminating friction is that part of what practitioners perceive as bureaucracy is actually model risk management, and this has come about because a regulator, an auditor, or a previous incident required it. The approval checkpoints concerning the registration of models in lending or fraud situations do not mean that the platform team is exercising control. A useful way to assess this is to consider whether a given step results in a record that someone outside your organization could request to see; if it does, then your responsibility is to ensure that this record is produced quickly, automatically, and in a clear format, not to make the step optional. If it does not, the step should be considered for removal and treated as such.

The Cognitive Load Redistribution Trap

A good platform engineering approach helps reduce the cognitive load. What usually takes place, however, is a redistribution — the complexity does not go away; it just shifts.

A self-service feature pipeline might eliminate the need for data scientists to write Spark jobs. But if the pipeline's configuration requires an understanding of a custom YAML schema which has seventeen optional fields, three of which interact in non-obvious ways, you're shifting the complexity from the code over to the configuration. That isn't simplification; it's merely translation.

The same thing happens with model serving. It is reasonable to abstract Kubernetes from the data scientists. However, if your internal serving abstraction forces them to understand your platform's resource quota model, your specific autoscaling annotations, and a deployment manifest format that differs from both the native Kubernetes format and any publicly available documentation — then you have established a proprietary knowledge silo that deepens over time.

# What your platform team designed:
serving:
  model_ref: "registry://fraud-detection/v3.2"
  scaling_profile: "adaptive-p95"
  resource_tier: "ml-standard-4"
  canary_weight: 10
  rollback_policy: "auto-p99-threshold"

# What the data scientist needed to understand to write this:
# - Your internal registry URI scheme
# - Four scaling profiles and when to use each
# - Six resource tiers and their actual CPU/memory mappings
# - Your canary traffic routing implementation
# - Your custom rollback threshold logic

Every piece of institutional knowledge your platform requires is a tax on everyone outside the platform team.

How to Audit Your Own Platform

Before you have your next look at the platform roadmap, carry out this exercise; it will take two hours and will tell you more than your adoption metrics ever will.

Step 1 — The Stranger Test: Ask a person who became a member of the organization within the last six months to take a model from a Jupyter notebook to a production endpoint using only the provided platform documentation. Watch them without interfering and record each time they pause, check Slack, or ask a question.

Step 2 — The Escape Hatch Inventory: Ask three senior data scientists to provide a list of all the instances in the past quarter when they had to work around the platform rather than using it. Don't make this seem punitive — instead present it as part of a design audit. The answers will be enlightening.

Step 3 — The Ticket Taxonomy: Retrieve the support requests from the last 90 days and classify them. Whenever more than 30% of the requests are different versions of "how do I do X even though it should be basic", then your abstraction layer has a comprehension gap.

Step 4 — The Deprecation Question: Find out from your platform team what features they would get rid of if they had the chance. Features that nobody uses but everyone has to work around are an example of accumulated debt that does not appear in technical debt reviews.

This audit does not result in any blame being assigned; rather, it sends out a prioritization signal. What friction points are causing the greatest diversion of energy away from real ML work?

From Gatekeeper Back to Accelerator

It's worthwhile, before the fixes are implemented, explaining why this problem persists. Inversion is seldom a failure of intention; rather, it is the result of the incentives in place. The platform teams are financed and assessed based on delivering their roadmaps: the features being released, the migrations finished, and the workloads being brought on board. Almost none of these criteria take into account how long it takes a practitioner to get their first model into production, and none of them pick up on work that has simply disappeared from the platform. Until a practitioner's experience is reflected in the platform team's performance metrics, the audit I referred to above will show what the issue is but will make no difference. This is a matter of leadership, not of engineering.

Getting back isn't about doing a platform rewrite; it's about having a philosophical reset.

The most effective ML platforms I've seen share a few characteristics that have nothing to do with the technology stack:

• The company regards data scientists as their main customer rather than as a secondary stakeholder. The platform roadmaps are based on practitioners' difficulties, not on the platform team's intuitions. This seems obvious but is seldom put into practice.
• They intentionally include ways of escaping. Instead of making all workloads go through a single abstraction, they set up a "golden path" for typical cases and ensure it is really easy to move down to lower-level primitives when necessary. The aim is to make the correct approach easy, not to make the incorrect one impossible.
• They measure time-to-value, not just uptime. A platform that is 99.9% available but still takes three weeks to onboard a new model type is not high-performing. While reliability is necessary, it by itself is not enough.
• They have embedded office hours, not just providing documentation. Although asynchronous documentation is valuable, it is through synchronous feedback sessions — in which platform engineers sit with practitioners and observe them working — that abstraction failures can be detected before they become workarounds.

The aim has always been to get better models to market more quickly while experiencing fewer production incidents. The platform was merely a method of achieving this; if it's starting to become the main objective, that should be discussed during your next leadership meeting.

What I'd like to leave you with is this: when was the last time you sat down with a data scientist and watched them try to use your platform without any assistance? Not a demonstration and not a guided tour — just watch them.

If it has gone past a quarter, then that should be your starting point.

What is the greatest source of annoyance that your ML platform causes for your practitioners at the moment — and how did you work it out?