Security Data Engineer

Security Data Engineer

Location: Hybrid (2 days in London office expected per week)

Contract Length: 6 months from start date

Rate: £430/day Inside IR35

About the Role:

The Control Tower is on an exciting journey to provide data-led security insights, and we are looking for an experienced Security Data Engineer to join the team. As a key member of this initiative, you will design, develop, and implement data solutions using the Azure cloud platform, contributing to the creation of scalable data pipelines and insights that empower leadership and colleagues across the organization.

Key Responsibilities:
• Design and Implement Data Architectures: Create scalable data pipelines using Azure Data Factory and Databricks to ingest, transform, and load data from multiple sources.
• Data Processing & Transformation: Develop and maintain integration processes, ensuring seamless data flow leveraging Databricks and Synapse Analytics.
• Data Storage Management: Select and manage storage solutions like Azure SQL Database, Azure Data Lake, and Azure Blob Storage.
• Data Warehousing: Support the building of scalable Azure Synapse Analytics solutions for large datasets.
• Real-time Data Processing: Optimize streaming pipelines with Azure Stream Analytics.
• Data Governance & Security: Implement data governance practices and ensure data quality within Azure.
• DevOps & Automation: Automate data pipelines and manage deployments using Azure DevOps for CI/CD.
• Performance Optimization: Fine-tune data pipelines, queries, and storage solutions for optimal performance.
• Collaboration: Work closely with cross-functional teams (data scientists, software engineers, business analysts) and senior stakeholders to deliver end-to-end data solutions.
• Troubleshooting & Continuous Improvement: Diagnose data-related issues and stay updated on emerging technologies in data engineering and banking.

Required Skills & Experience:
• Proficiency in Azure Data Factory, Databricks, and Synapse.
• Hands-on experience with Python, SQL, and Spark for data processing.
• Strong ability to work with unstructured datasets and design high-quality code.
• Experience in automating tasks and deploying production-level code.
• Familiarity with visualization tools and building insightful dashboards.
• Excellent communication and collaboration skills in a team environment.
• Azure certification or related technologies (e.g., Microsoft Certified: Azure Data Engineer Associate).
• Understanding of security tools, risk frameworks, and change management principles.
• Experience in Agile development methodologies and data management principles.

Desirable Skills:
• Knowledge of retail banking channels and products, especially related to data-driven insights.
• Familiarity with industry standards, roadmaps, and best practices for data engineering and cloud computing.