Databricks · Primly Community

Went through the full Databricks DE loop last month. Here's my honest debrief.

hardware_hugo · 5 replies

Finally out the other side of a Databricks data engineering interview. Sharing notes because I couldn't find a good recent writeup when I was prepping.

Round 1: Recruiter screen, 30 min. Standard stuff. They actually asked why Databricks specifically, and vague answers won't land. I talked about Delta Lake and the lakehouse pattern, which clearly resonated.

Round 2: Hiring manager, 45 min. Half technical background, half 'tell me about a hard pipeline problem you owned.' Not behavioral fluff, they wanted architecture decisions and tradeoffs.

Round 3: Coding, 1 hour. Two problems. First was medium-level graph traversal. Second was a data processing scenario in Python where you're working with chunked input that doesn't fit in memory. That second one was the real test.

Round 4: System design, 1 hour. Design a streaming ingestion system with exactly-once guarantees. They went deep on fault tolerance and offset management. Know Kafka and Spark Structured Streaming.

Round 5: Values/behavioral round with a cross-functional person. Less intense but they're assessing collaboration and communication under pressure.

Total time from recruiter screen to offer: 5 weeks. Decision came 4 days after the final round. The feedback I got on rejection (first run, before I prepped more) was actually useful, which is rare.

5 replies

analyst_ana

this is so helpful, thank you. the memory-constrained data processing thing is something i've been weak on. did they give you a specific dataset size constraint or was it more conceptual?

de_derek

they said something like '10TB file, 8GB RAM, design your approach.' you don't write the whole thing, you walk through your chunking strategy and how you handle state. the key thing was explaining why you'd use streaming vs batch and what failure modes you're guarding against. less about syntax, more about reasoning.

sre_sol

exactly-once semantics in a system design interview is a trap if you haven't thought through the idempotency layer carefully. good that you flagged it. most people say 'use kafka' and stop there like that's an answer.

ml_mike

did the ML/DE roles you saw have different loops or was this structure consistent across both tracks? asking because i'm looking at an MLE role there.

de_derek

i only went through the DE track so can't say for certain, but a friend who did MLE said there's a 4th technical round that includes ML system design (training pipelines, feature stores) on top of the standard coding. expect more depth on MLflow since they own it.