Key Responsibilities
- Lead and mentor data engineering, data science, and analytics team members.
- Architect and implement scalable end-to-end data pipelines and distributed data processing.
- Design and optimize ETL/ELT workflows using Airflow, Spark/PySpark, and related tools.
- Develop and manage REST APIs and microservices and integrate external systems and tools.
- Build and maintain cloud infrastructure with strong focus on AWS services.
- Manage containerization and orchestration using Docker and advanced Kubernetes.
- Implement and manage enterprise data warehousing solutions, preferably Snowflake.
- Support research and implementation of GenAI-based solutions.
- Define and enforce data standards, governance, and data quality frameworks.
- Collaborate with business and product teams to deliver data-driven insights.
Education and Experience
- Bachelor’s degree in Computer Science, Engineering, or related field.
- Minimum 7+ years of relevant experience in data engineering, data science, or software development.
- 5+ years hands-on experience with Python and SQL.
- 5+ years experience in AWS cloud services.
- Strong Docker expertise (5+ years) and advanced Kubernetes experience.
- Airflow experience (5+ years) for workflow orchestration.
- Hands-on experience with Spark / PySpark.
- Data warehousing experience (Snowflake preferred).
- Basic understanding of ELK stack.
- Experience with system integrations (Slack, Glean AI, OpenMetadata preferred).
- Intermediate experience with GenAI / LLM-based implementations.
Analytical and Personal Skills
- Stakeholder management and strong verbal & written communication skills are a must.
- Strong leadership, analytical thinking, and problem-solving capabilities.
- Self-driven, motivated and detail oriented.
- Patient, persistent and focused
