Anthropic · Primly Community

Anthropic machine learning engineer interview: what they actually tested

ml_mike · 6 replies

let me cut through the vague stuff I've seen posted about Anthropic MLE interviews. I just went through the full loop for an MLE role on their alignment research infrastructure side. here's what they actually asked.

phone screen: one ML conceptual question, one coding problem. the ML question was about training instabilities: you're running a large language model training run and you notice loss spikes every ~500 steps. walk me through how you'd diagnose it. they weren't looking for one right answer, they wanted to see systematic debugging: learning rate scheduler? gradient clipping? data quality in that batch window? mixed precision overflow?

the coding problem was on the simpler end. clean implementation of a custom loss function in PyTorch, make sure it handles edge cases and is numerically stable. they cared about whether I'd think about float precision without being prompted.

onsite, 5 rounds (yes, 5): ML depth: deep on transformers. not just "explain attention" but: what changes when you scale context length, why does KV cache memory scale quadratically, how do you think about grouped-query attention as a tradeoff. if you're not up to date on the architecture space, this will sting. systems for ML: distributed training design. they gave me a training job that doesn't fit on one node and asked how I'd approach data parallelism vs. model parallelism vs. pipeline parallelism. also: what fails first as you add nodes. coding: more involved than the phone screen. implement something meaningful, not just a code kata. mine involved building a small evaluation harness for a text generation model. research communication: explain a paper you've read recently and what you took from it. I chose a mechanistic interpretability paper. they engaged substantively, which was a good sign. mission/values: the standard Anthropic round. for MLE it had an ML safety flavor. how do you think about evaluation of models for harmful outputs? what's the difference between capability and safety evaluation?

the bar is genuinely high and they care about the AI safety angle more than any other MLE interview I've done. if you're applying here because it's a hot AI company and not because you have any interest in the safety mission, it'll show.

offer was around 240-290k TC depending on your equity assumptions. haven't decided whether to take it yet.

6 replies

corp_refugee

the KV cache memory question is a pretty good filter for people who've actually worked with large context vs. people who just talk about it. were they asking about wall clock memory or the asymptotic complexity?

ml_mike

both, but they were more interested in the practical implications. like: at 128k context length with this batch size and this precision, you'd need X memory. they wanted to see that I could connect the formula to real constraints.

ds_dmitri

did they ask about RLHF / preference learning at all? that seems like it would be on-topic for an alignment-adjacent MLE role.

ml_mike

not explicitly in the technical rounds. it came up in the mission/values round naturally when we talked about how safety evals differ from capability evals. I'd read the RLHF paper and the Constitutional AI paper before going in regardless.

infra_ines

five rounds is a lot. how long was the total onsite? half day or full day?

quietquit_quincy

240-290k TC for MLE. what's the breakdown? just curious how much of that is the pre-IPO equity component.