Senior Data Engineer

We are looking for a hands-on Data Engineer with strong expertise in SQL, PySpark, Apache Spark and Databricks, along with awareness of GenAI and AI-assisted coding tools.

The candidate should be capable of building scalable data pipelines, troubleshooting performance issues, and optimizing data processing for both performance and cost.

Key Responsibilities

• Design, develop and maintain scalable data pipelines using SQL, PySpark, Spark and Databricks.
• Build reliable batch/incremental data-processing solutions and data products.
• Troubleshoot and optimize Spark workloads for performance, scalability and cost efficiency.
• Work with Delta Lake and modern data-platform architecture.
• Use AI coding agents to accelerate development, debugging, testing and documentation.
• Collaborate with Application teams, Consultants and Managers.

Required Skills

• Expert SQL and PySpark development.
• Strong understanding of Apache Spark internals — distributed processing, partitioning, shuffle, data skew and execution model.
• Strong Databricks experience with Delta Lake, optimization, compute management and cost optimization.
• Strong data-engineering fundamentals: ETL/ELT, data modelling, incremental processing and data quality.
• Experience troubleshooting and optimizing large-scale data workloads.

GenAI \& AI-Assisted Development

• LLMs, RAG, Embeddings, Agents and Agentic AI
• AI-assisted coding tools such as Claude Code, Codex, Cursor, Antigravity or similar
• Ability to review, validate and improve AI-generated code.

Good to Have

• DevOps knowledge: Git, CI/CD, Docker and deployment practices.
• Exposure to NoSQL databases and their use cases.
• Data Engineer/Analytics Certificate in any of cloud (AWS, Azure or GCP).

Candidate Profile

• Strong problem-solving and analytical skills.
• Hands-on, engineering-oriented mindset.
• Ability to understand why a technical approach is selected, not just implement it.
• Clear and structured communication.
• Comfortable working independently and learning new technologies.

Pay

₹509,000.00 - ₹1,880,000.00 per year

Benefits

• Flexible schedule

Work Location

In person