Lead Data Engineer

We are on the lookout for an experienced Lead Data Engineer to join our customer's innovative team on a contract basis.

Key Responsibilities:

• Provide strategic leadership to the Data Engineering team, fostering best practices in software development to create robust AWS Glue data pipelines.

• Oversee the secure management of the data engineering lifecycle, ensuring all pipelines are designed, built, deployed, and maintained with security in mind.

• Take an active role in developing data pipelines by writing, testing, and deploying scalable and efficient code.

• Construct CI pipelines and implement Infrastructure as Code for deployment across AWS environments.

• Establish and configure development environments and services, including databases and various AWS components.

• Advocate for Agile and DevOps methodologies to streamline software development processes and enhance delivery speed.

• Facilitate daily stand-up meetings to assess team progress, address challenges, and ensure alignment with project goals.

• Conduct thorough code reviews and maintain comprehensive technical documentation.

• Collaborate with stakeholders to gain insights into existing data workflows and support development efforts.

• Mentor and support team members through guidance, pair programming, and code review sessions to help them achieve their potential.

Location: Hybrid (Greater Manchester - 2 days per week)

Ideal Candidate:

To thrive in this role, you will need:

• Extensive experience in leading the development of AWS Glue data pipelines, with a strong background in similar roles within medium-sized organizations.

• A minimum of four years’ experience in data engineering, showcasing your ability to write clean, testable, and maintainable code.

• Proficiency in AWS technologies, including Glue, Lambda, S3, as well as programming languages such as Python and SQL, and familiarity with PostgreSQL and PySpark.

• Knowledge of building CI pipelines, preferably using Microsoft Azure DevOps, alongside a solid understanding of the software development lifecycle and Agile methodologies (Scrum and Kanban).

• Experience with DevOps principles and practices, as well as lean development methodologies.

• A strong grasp of information security requirements, particularly regarding GDPR compliance and secure development lifecycles.

Skills and Competencies:

• Exceptional verbal and written communication skills to effectively engage with technical and non-technical stakeholders at all levels.

• Strong people management abilities, creating a positive team environment, resolving conflicts, and being responsive to team needs.

• Demonstrated decision-making capabilities and the ability to quickly identify requirements to deliver effective solutions without unnecessary complexity.

• A detail-oriented mindset with excellent problem-solving skills, able to prioritize tasks under pressure and maintain a focus on quality.

• A commitment to continuous learning and adapting to new technologies, methodologies, and best practices.

📩 Apply Today for further information.