Data Engineer – Azure Data Factory

Data Engineer – Azure Data Factory

Job Summary:

We are seeking a highly skilled Data Engineer with expertise in Azure Data Factory (ADF) to design, build, and maintain data pipelines and ETL workflows in Microsoft Azure. The ideal candidate will have experience in data ingestion, transformation, and integration using ADF, Azure Synapse, Data Lake, and SQL-based solutions.

Key Responsibilities:ETL & Data Pipeline Development:
• Design and develop scalable ETL pipelines using Azure Data Factory (ADF).
• Implement data ingestion, transformation, and orchestration processes.
• Optimize data workflows for performance, cost efficiency, and reliability.

Azure Data Services:
• Work with Azure Synapse Analytics, Azure SQL Database, and Azure Data Lake Storage (ADLS).
• Develop data integration solutions using Dataflows, Azure Functions, and Logic Apps.
• Implement event-driven and batch data processing architectures.

Data Modeling & Transformation:
• Design and maintain data models for relational and big data environments.
• Implement data quality and governance best practices.
• Work with structured and unstructured data from various sources.

DevOps & Automation:
• Automate data pipeline deployments using CI/CD with Azure DevOps and GitHub Actions.
• Utilize ARM templates, Terraform, or Bicep for infrastructure automation.
• Implement monitoring and logging using Azure Monitor, Log Analytics, and Application Insights.

Collaboration & Best Practices:
• Work closely with data analysts, BI teams, and cloud engineers.
• Ensure compliance with security, privacy, and data governance policies.
• Stay updated with Azure advancements and data engineering best practices.

Required Skills & Qualifications:
• 5+ years of experience in data engineering and ETL development.
• Strong expertise in Azure Data Factory (ADF) and Azure Synapse Analytics.
• Proficiency in SQL, Python, or Spark for data processing.
• Experience with Azure Data Lake Storage (ADLS) and Blob Storage.
• Knowledge of data warehousing concepts and data modeling techniques.
• Strong troubleshooting and performance optimization skills.

Preferred Qualifications:
• Azure Data Engineer Associate (DP-203) certification.
• Experience with Databricks, Power BI, and Event Hub/Kafka.
• Knowledge of machine learning or real-time data processing in Azure.