Azure Databricks DevOps Engineer -Terraform/Bicep/ARM Templates

Azure Databricks DevOPS Engineer -Terraform/Bicep/ARM Templates

Location: Remote work

Duration: 12 Months+

We are seeking a highly skilled and motivated Databricks Cloud Engineer to join our growing team. The ideal candidate will have a strong background in cloud infrastructure, with a proven ability to deploy and manage Databricks clusters on Azure using infrastructure-as-code tools like Terraform or Bicep. Experience with ARM Templates is also a required

Responsibilities:
• Design, build, and maintain Databricks clusters on Azure using infrastructure-as-code principles.
• Develop and implement automated deployment pipelines for Databricks resources.
• Troubleshoot and resolve infrastructure-related issues within the Databricks environment.
• Collaborate with data engineers, data scientists, and other stakeholders to ensure smooth and efficient data processing.
• Optimize Databricks cluster configurations for performance and cost-effectiveness.
• Stay abreast of the latest advancements in cloud computing, Databricks, and related technologies.
• Contribute to the development and maintenance of best practices and standards for cloud infrastructure management.
• Participate in code reviews and ensure adherence to security and compliance best practices.

Qualifications:

Must Have:
• 3+ years of experience in cloud computing, with a strong focus on Azure.
• Proven experience deploying and managing Databricks clusters on Azure.
• Expertise in infrastructure-as-code tools such as Terraform or Bicep.
• Strong understanding of Azure resources and services (e.g., Virtual Networks, Storage, Key Vault).
• Experience with ARM Templates for managing Azure resources.
• Proficiency in scripting languages like Python or Bash.
• Excellent problem-solving and troubleshooting skills.
• Strong communication and collaboration skills.
• Nice to Have:Experience with containerization technologies (Docker, Kubernetes).
• Experience with data pipelines and ETL/ELT processes.
• Knowledge of security best practices for cloud environments.
• Experience with CI/CD pipelines (e.g., Azure DevOps, Jenkins).
• Relevant certifications (e.g., Azure Certified Solutions Architect, Databricks Certified Professional).

Education:
• Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent experience.