EH-1521- Data Engineer

Job Title

Data Engineer

Location

Hyderabad

Experience

8 – 12 Years

Employment Type

Full-Time

Shift

Overlap with US Team (evening IST preferred)

Staff Data Engineer

Data Platform & Analytics Engineering · Hyderabad (On-site, Softobiz office)

About the Role

We are looking for a Staff Data Engineer to own our end-to-end data platform — from raw event ingestion, through the warehouse, to the analytics marts that the business, product and marketing teams rely on every day. You will be the person who makes sure the numbers we make decisions on are accurate, fresh, reproducible and trusted.

This is a hands-on individual-contributor role. You will partner daily with engineering, analytics, product, growth and external data partners. Success looks like: one canonical patient or customer key across every system, a well-modelled warehouse that answers business questions in SQL rather than spreadsheets, and a data platform that product, marketing and leadership genuinely trust.

Scope note: this role owns warehouse architecture, pipelines and data modelling. The sister MarTech Engineering role owns event capture at the edge and the delivery of conversion signals to ad platforms. Both roles share one source of truth (the warehouse) and collaborate on the hand-off between event streams and warehouse tables.

What You'll Own

Warehouse architecture and modelling — BigQuery (or equivalent) as the single source of truth, designed to serve both analytics and activation use cases.

dbt project ownership — model layering (staging → intermediate → marts), naming conventions, tests, documentation, exposures, and the promotion workflow from dev to production.

Ingestion pipelines — event streams from the CDP (Segment or equivalent) landing cleanly in the warehouse, plus structured loads from the product database, billing / payments, ad platforms and third-party SaaS systems.

Identity resolution in the warehouse — stitching anonymous, edge-level and identified user records into one canonical key, and publishing that key to downstream consumers.

Core marts — revenue, orders, subscriptions, cohorts, LTV, funnel conversion, channel performance, clinical or operational metrics as the business requires.

Reverse-ETL / activation models — warehouse tables purpose-built to feed ad-platform conversions APIs, lifecycle marketing, audiences and experimentation tools.

Data quality and observability — dbt tests, freshness monitors, row-count alerts, reconciliation checks between systems (for example: warehouse orders vs payment processor, warehouse conversions vs ad-platform reports).

Data governance — PII / PHI handling, column-level masking, row-level security where required, access policies, and the documentation that makes those policies auditable.

Orchestration and CI/CD — scheduled runs, retries, incremental strategies, pull-request-based dbt reviews, automated tests in CI.

Partnership with external data partners and the sister MarTech Engineering function — clear ownership lines, shared conventions, joint on-call for pipeline incidents.

Required Experience

8–12 years in data engineering or analytics engineering at consumer-facing digital businesses — ideally including at least one direct-to-consumer, e-commerce, subscription or marketplace product.

Deep, hands-on experience designing and maintaining a warehouse on BigQuery, Snowflake or Redshift. You can explain partitioning, clustering, materialised views, and cost / performance trade-offs from memory.

Production experience with dbt — Core or Cloud — including model layering, incremental strategies, snapshots, tests, documentation, exposures, and managing a multi-contributor repo through pull requests.

Fluent SQL — advanced window functions, CTEs, pivoting, session stitching, slowly-changing-dimension patterns. You can read a 200-line query and tell us what is wrong with it.

Experience designing identity resolution in the warehouse — merging anonymous, cookie-level, logged-in and server-generated identifiers into one canonical customer key, and publishing that key for downstream systems to join on.

Hands-on experience building marts for revenue, orders, subscriptions, cohorts, LTV and funnel analysis — and reconciling those marts against authoritative systems (payment processors, billing, ad platforms).

Python for data work — transformations, orchestration DAGs, small internal tools, and scripting around the warehouse. You do not need to be a backend engineer.

Working knowledge of at least one workflow orchestrator — Airflow, Prefect, Dagster, or equivalent — and the operational discipline that goes with it (idempotency, backfills, retries, alerting).

Comfort integrating warehouse output into reverse-ETL tooling (Polytomic, Hightouch or Census) so that audiences, conversion events and customer traits can be activated in ad platforms, lifecycle tools and experimentation systems.

A bias for data contracts and tests over heroics — you treat a broken dashboard as a platform failure to be prevented, not a one-off to be patched.

Experience operating under ambiguity — a platform migration, re-platforming, or a major tracking overhaul where the "right" schema had to be invented mid-flight.

Tools and Technologies

The following stack describes what you will work with day-to-day. You do not need hands-on experience with every single tool — depth in 60–70% of this list is what we are looking for, along with the pattern-matching to pick up the rest quickly.

Category Tools

Data Warehouse BigQuery (required), Snowflake or Redshift. Strong grasp of partitioning, clustering, cost controls.

Transformation dbt Core or dbt Cloud (required), dbt tests, snapshots, exposures, packages.

Orchestration Airflow, Prefect, Dagster, or an equivalent DAG runner. dbt Cloud scheduling for dbt-only pipelines.

Ingestion Segment (CDP → warehouse), Fivetran or Airbyte (SaaS → warehouse), custom Python ingestion where needed.

Reverse-ETL / Activation Polytomic (preferred), Hightouch, Census — mappin