Azure Databricks Architect (Salesforce)

Title: Azure Databricks Architect

Location: Remote

Duration: Long-term Contract

Note: Client is looking someone hands on experience with Python or Scala and Salesforce, Healthcare Background.

Overall Responsibility:

Responsible for developing, implementing, and operating stable, scalable, low-cost solutions to source data from client systems into the Data Lake, data warehouse and end-user facing BI applications. Responsible for ingestion, transformation and integration of data to provide a platform that supports data analysis and enrichment as well as making data operationally available for analysis. The data Engineer will be a data pipeline builder, data wrangler and support application developers, database architects, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture and is consistent throughout our ongoing projects.

Essential Functions:
• Build and maintain scalable automated data pipelines. Support critical data pipelines with a highly scalable distributed architecture
• Have Knowledge on Saleforce API to pull data from Salesforce system to Databricks Delta tables
• including data ingestion from Salesforce, data integration, data curation.
• Need to build scalable and metadata driven framework to reuse notebooks/pipelines to move data from Salesforce to Databricks. Solution should work for future ingestions from Salesforce to Databricks data movement with minimal
• Deploy, automate, maintain and manage Azure cloud-based production system, to ensure the availability, performance, scalability and security of productions systems.
• Good architectural understanding to build and ensure customer success when building new solutions and migrating existing data applications on Azure platform.
• Conduct full technical discovery, identifying pain points, business and technical requirements, "as is" and "to be" scenarios.
• Design and arrangement of scalable, highly attainable, and fault tolerant systems on Azure platform.
• Ownership and responsibility for end-to-end design and development, testing, release of key components.
• Understand and implement best practices in management of data, including master data, reference data, metadata, data quality and lineage. Experience with code versioning tools and a command of configuration management concepts and tools, CI-CD including DevOps. Other duties as assigned.

Experience:
• Expert level SQL knowledge and experience.
• Expert level experience with Python/Pyspark/Scala and Object-Oriented programming.
• Experience with streaming integration and cloud-based data processing systems such as; Kafka, and Databricks.
• Hands-on knowledge of cloud-based data warehouse solutions like Azure & Snowflake.
• Experience with Azure cloud architecture and solutions.
• Experience with Azure Data Lake store, Blob Storage, VMs, Data Factory, SQL Data Warehouse, Azure Databricks, HDInsight, etc. Experience with data pipeline and workflow management tools: Azure Data Factory, Airflow, etc.
• Experience with Oracle, Microsoft SQL Server Database system.
• Experience working within Agile methodologies.
• Experience with Microsoft Windows and Linux virtual servers. Moderate skill in Power BI.

Other Skills:
• Experience with healthcare data modeling standards, such as HL7 or FHIR is preferred.
• Experience in processing healthcare data sets (medical records, claims, clinical data etc.) is preferred.
• Strong project management skills.
• Strong problem-solving, decision-making and analytical skills.
• Excellent interpersonal and organizational skills a team player who can effectively partner with all levels of the company.
• Detail oriented and organized.
• Ability to handle numerous assignments simultaneously.

About Logisoft:

We represent Logisoft Technologies, Inc. with pride; We present ourselves as a premiere Technology, Consulting, Product Development, and Software Services Company.

Our Head Office is located in South Plainfield, NJ, and Our Offices location is in Hyderabad, INDIA & Accra, GHANA.

We are Microsoft Official Partners - A Microsoft Certified Partner that helps customers with a range of IT projects and specific IT solutions.

We started our journey in the year 2008. In the past 18 years, we have acquired the trust of many IT market leaders.

LOGISOFT Technologies is committed to a policy of Equal Employment Opportunity and will not discriminate against an applicant or employee on the basis of age, sex, sexual orientation, race, color, creed, national origin, ancestry, disability, marital status, or any other legally protected basis under federal, state or local law.

Interested candidates can apply directly to this posting or they can reach Venkatesh.ch@logisofttechinc.com