Datadog · Primly Community

Datadog machine learning engineer interview: less hype, more applied systems

ml_mike · 4 replies

Went through Datadog's MLE loop in February 2026. Sharing this specifically because the ML interview meta at observability companies is different from what you'll read in generic "MLE interview prep" guides, which are all written assuming you're interviewing at a consumer recommendation system.

Datadog's ML problems are fundamentally about time series, anomaly detection, and forecasting at scale. If your ML background is NLP or computer vision, that's fine, but you need to be able to pivot to these domains convincingly.

Loop structure:

Phone screen: 60 min. One coding problem (implement a basic sliding window anomaly detector in Python, nothing fancy, they want to see if you think about edge cases) and then a 20-minute discussion about a project on my resume where I deployed a model to prod.

On-site, round 1: ML breadth. Classic stuff but framed toward observability. They asked: how would you detect anomalies in a multivariate time series where sensors have different latencies and missing values are common. Bayesian methods, robust statistics, simple threshold-based approaches and when to use each. They didn't want a single right answer, they wanted to see how I think through tradeoffs.

On-site, round 2: ML systems. Design a training pipeline for a model that needs to process billions of metric data points per day with near-real-time feedback on model quality. This is basically an MLOps question. Know feature stores, data validation, model monitoring, drift detection.

On-site, round 3: Coding. Implement a simple changepoint detection algorithm. They specified the algorithm so it wasn't open-ended, but I had to code it cleanly in Python and explain the complexity. Felt like a medium-hard DSA problem if you're not familiar with the specific algorithm.

On-site, round 4: Behavioral.

What I'd tell someone prepping: spend serious time on time series forecasting (Prophet, statistical models, not just transformer-based), anomaly detection algorithms (isolation forest, CUSUM, Bayesian changepoint), and MLOps concepts at scale. The actual ML math was secondary to showing you understand the systems context.

4 replies

ds_dmitri

Changepoint detection as a coding round is extremely on-brand for Datadog. CUSUM or PELT are probably the implementations they'd expect. Did they specify which algorithm or did they pick something more exotic?

ml_mike

They said PELT (Pruned Exact Linear Time). I knew it conceptually but hadn't coded it from scratch. I got about 70% of a clean implementation and spent the remaining time discussing the dynamic programming structure. They said they weren't expecting a perfect implementation, they wanted to see my problem-solving process. Still walked away feeling like I underperformed that round.

infra_ines

The ML systems design round sounds almost identical to a distributed systems design round with some model-shaped objects in it. Which honestly makes sense. If the model is running at that scale, 90% of the hard problems are infrastructure problems.

sdr_sky

MLE comp at Datadog for senior level in 2026: my offer was base ~$210k, RSU grant ~$380k over 4 years, 15% bonus target. Total ~$330k in year 1 assuming stock holds. This was for a Staff MLE role, NYC. Senior (one level below) would probably run $30-40k lower on base.