Location: Bedford (Hybrid)

Experience:10+ years

Job Type: Full Time

Job Summary:

We are seeking a seasoned Lead Data Engineer to architect and lead end-to-end implementation of modern data solutions in a hybrid cloud environment. This role requires strong expertise in Snowflake, AWS Glue, SQL, Python, Spark, and Big Data concepts. The ideal candidate will be a hands-on technical leader who can manage complex projects, mentor engineering teams, and collaborate cross-functionally to build scalable, secure, and high-performing data platforms.

Key Responsibilities:

• Architect, design, and lead the implementation of robust data pipelines using AWS (Glue, EMR, S3) and Snowflake
• Translate business needs into technical specifications and scalable solutions
• Build and maintain optimized, reusable data models for analytics and reporting
• Lead performance tuning, data quality checks, and governance controls including RBAC and row/column-level security
• Mentor and guide a team of 8–10+ engineers, ensuring high delivery standards and adherence to best practices
• Handle production deployments, release planning, and post-deployment support (L2/L3)
• Implement CI/CD pipelines, job scheduling, and automation of ingestion and transformation tasks
• Leverage knowledge of insurance data (claims and loss) to enhance decision-making insights
• Champion continuous improvement, code refactoring, and operational reliability

Must-Have Skills:

• Hands-on Snowflake experience with strong architectural understanding
• Expertise in ETL design, data ingestion, CDC, incremental/batch processing, and job orchestration
• Strong command of SQL, Python, and PySpark
• Deep understanding of data warehousing principles, query optimization, and data quality frameworks
• Proficiency in Apache Iceberg, AWS Glue, Aurora, RDS, and other AWS services
• Experience with RBAC, data masking, profiling, and integrating data from multiple sources
• Ability to lead technical discussions, manage project timelines, and resolve complex delivery challenges
• Demonstrated success in end-to-end data platform implementation, including training, documentation, and change management
• Strong problem-solving aptitude and the ability to independently drive technical initiatives

Preferred Experience:

• 10+ years of experience in data engineering, with proven leadership in project delivery
• Deep experience in big data ecosystems and cloud-based solutions
• Background in the insurance industry, especially with claims and loss data
• Familiarity with Agile practices, Scrum ceremonies, and stakeholder engagement in fast-paced teams