went through IBM's data engineer loop for a senior DE role on one of their IBM Cloud data platform teams. sharing this because the DE loop differs from the DS loop in ways that aren't obvious from the job description.
what the loop looked like: recruiter screen (30 min) technical phone screen with a DE on the team (45 min, mostly SQL and a systems question) take-home: design a data ingestion pipeline (48 hours, delivered as a design doc, not code) final round: panel with two engineers and a manager (2.5 hours total)
SQL portion: heavier on analytical SQL than you'd think for a 'pipeline' role. got asked about window functions, handling slowly changing dimensions (SCD Type 2 specifically), and one question about writing a query to detect duplicate records with a business-key-based deduplication strategy. this is the classic 'here's a messy source, make it clean' problem.
pipeline design / systems: IBM's world is very enterprise, so 'pipeline' often means ELT from mainframe sources, legacy COBOL-output files, or SAP. the take-home prompt i got involved ingesting from an on-prem Oracle database to a cloud data warehouse. they want to see: incremental load strategy, schema evolution handling, error recovery, observability.
if your pipeline experience is all kafka/spark on cloud-native stacks, you can still answer these but you need to translate. the soul of the answer is the same (idempotency, at-least-once vs. exactly-once delivery, monitoring), even if the tooling differs.
tech stack they mentioned: IBM DataStage came up in conversation, also IBM Data Fabric. knowing these names at least conceptually helps. i hadn't used DataStage but explained how it maps to things i'd built with Airflow/dbt and they seemed fine with that.
final panel behavioral: same IBM themes as other roles. stakeholder management, handling ambiguity, time you failed and what you learned. have real stories. the DE-specific behavioral question i got: 'tell me about a time a data pipeline you built caused a downstream problem for a business team. how did you handle it.' classic 3am scenario. have a story ready.