Went through an xAI senior SWE loop in April 2026. The system design round was the most interesting part so I'll just focus on that.
The prompt was essentially: design a real-time inference serving layer for a large language model. Not "design Twitter" or "design a URL shortener." Actual domain-relevant stuff. They want to see if you've thought about inference at scale, not just generic distributed systems.
What mattered: GPU memory constraints. I wasn't expected to know exact numbers but I needed to show I understood that GPU memory is the bottleneck, not CPU. They pushed on this hard. Batching strategies. Dynamic batching, continuous batching, why naive request-per-call doesn't work at scale. Latency vs throughput tradeoffs. They grilled me on what levers you'd pull if the P99 latency SLA tightened. Caching at the KV-cache level (yes, they went there for a senior loop).
Did they expect a perfect answer? No. They explicitly said "there's no right answer" but I noticed the interviewer's energy changed when I mentioned continuous batching vs. static batching. They wanted to see that I'd at least engaged with the problem space.
Format was 50 minutes, one dedicated system design round in the onsite. I had a whiteboard (virtual, excalidraw-style) and shared screen for code snippets when I wanted to show API shapes.
For senior/L5 calibration they seem to care a lot about scope. Can you take an ambiguous prompt and structure it yourself, or do you need them to hand you requirements? That was the meta-skill being tested.
One thing I'd prep: know the difference between model parallelism and tensor parallelism at a high level. I fumbled this a bit and it showed.
Total loop was 4 rounds onsite plus a phone screen before. System design was one of the four onsite slots.