Data Engineer - GCP
Exciting contract opportunity for a Data Engineer in Alpharetta, GA. In this role you will work 3 days/week in the office (Monday, Tuesday, Thursday). As a Data Engineer you will have the opportunity to design and execute vital projects such as re-platforming our data services on Cloud and on-prem and delivering real-time streaming capabilities to our business applications. You will bring a clear point of view on data processing optimization, data modeling, data pipeline architecture, data SLA management. Our client’s mission is design & implement a data and analytics platform/infrastructure that enables a future-state analytics lifecycle, data monetization opportunities, data acquisition, analysis & feature engineering, model training, impact analysis, reporting, predictive and quantitative analysis, & monitoring.
What You Get to Do:
• An ideal candidate is intellectually curious, has a solution-oriented attitude, and enjoys learning new tools and techniques.
• You will have the opportunity to design and execute vital projects such as re-platforming our data services on Cloud and on-prem, and delivering real-time streaming capabilities to our business applications.
• Brings a clear point of view on data processing optimization, data modeling, data pipeline architecture, data SLA management.
• Holds accountability for the quality, usability, and performance of the solutions.
• Leads design sessions and code reviews to elevate the quality of engineering across the organization.
• Design, develop data foundation on cloud data platform using GCP tools and techniques e.g.: Google Cloud Platform, Pub/Sub, Big Query, Cloud SQL, BigTable, BigLake DataForm, DataFlow, DataStream, Google cloud storage, Cloud Composer/DAG, Cloud Run, Cloud RESTAPI, ADO GITREPO, CI/CD Pipelines, Secret Manager, Cloud IAM Terraform/YAML etc.
• ETL pipeline using Python builds and scalable solutions.
• Multi-level Data Curation and modeling.
• Data design and architecture.
• Hands on experience in building complete CI/CD Pipeline creation and maintenance using Azure DevOps and Terraform/Terragrunt.
• Increase the efficiency and speed of complicated data processing systems.
• Collaborating with our Architecture group, recommend and ensure the optimal data architecture.
• Analyzing data gathered during tests to identify strengths and weaknesses of ML Models
• increase the efficiency and speed of complicated data processing systems.
• Collaborate across all functional areas to translate complex business problems into optimal data modeling and analytical solutions that drive business value.
• Lead the improvements and advancement of reporting and data capabilities across the company, including analytics skills, data literacy, visualization, and storytelling.
• Develop a certified vs. self-service analytics framework for the organization.
• Collaborating with our Architecture group, recommend and ensure the optimal data architecture.
• Highly skilled on RDMS (Oracle, SQL server), NoSQL Database, and Messaging services (Publish / Subscribe) systems.
• Extensive knowledge/coding skills of Python including understanding of data modeling and data engineering.
What You Bring to the Table:
• Bachelor’s degree in computer science, Engineering, Mathematics, Sciences, or related field of study from an accredited college or university; will consider a combination of experience and/or education.
• Ideally 5+ years of experience in developing data and analytics solutions and approximately 4+ years data modeling and architecture.
• Expertise in programming languages including Python and SQL.
• Familiarity with certain software development methodologies such as Agile, or Scrum.
• Critical thinking.
• Leveraging cloud-native services for data processing and storage.
• Storage – BigQuery, GCS, Cloud SQL, BigTable, BigLake
• Event processing – Pub/Sub, EventArc
• Data pipeline and analytics – Dataflow, DataForm, Cloud Run, Cloud Run Function, DataStream, Cloud Scheduler, Workflows, Composer, Dataplex, ADO GITREPO, CI/CD Pipelines, Terraform/YAML
• Security – Secret Manager, Cloud IAM
• Others – Artifact Registry, Cloud Logging, Cloud Monitoring
• Work with distributed data processing frameworks like Spark.
• Strong knowledge of database systems, and data modeling techniques.
• Ability to adapt to evolving technologies and business requirements.
• Ability to explain technical concepts to nontechnical business leaders.
• Monitor system performance and troubleshoot issues.
• Ensure data security.
• Proficiency in technical skills, cloud tools and technologies.
Technical Skills:
• Must Have
• GCP (Google Cloud Platform)
• ETL pipeline using Python
• Expertise in programming languages including Python and SQL.
Got Extra to Bring?
• GCP – Professional Data Engineer Certification
• Documenting all steps in the development process
• Manage the data collection process providing interpretation and recommendations to management
Exciting contract opportunity for a Data Engineer in Alpharetta, GA. In this role you will work 3 days/week in the office (Monday, Tuesday, Thursday). As a Data Engineer you will have the opportunity to design and execute vital projects such as re-platforming our data services on Cloud and on-prem and delivering real-time streaming capabilities to our business applications. You will bring a clear point of view on data processing optimization, data modeling, data pipeline architecture, data SLA management. Our client’s mission is design & implement a data and analytics platform/infrastructure that enables a future-state analytics lifecycle, data monetization opportunities, data acquisition, analysis & feature engineering, model training, impact analysis, reporting, predictive and quantitative analysis, & monitoring.
What You Get to Do:
• An ideal candidate is intellectually curious, has a solution-oriented attitude, and enjoys learning new tools and techniques.
• You will have the opportunity to design and execute vital projects such as re-platforming our data services on Cloud and on-prem, and delivering real-time streaming capabilities to our business applications.
• Brings a clear point of view on data processing optimization, data modeling, data pipeline architecture, data SLA management.
• Holds accountability for the quality, usability, and performance of the solutions.
• Leads design sessions and code reviews to elevate the quality of engineering across the organization.
• Design, develop data foundation on cloud data platform using GCP tools and techniques e.g.: Google Cloud Platform, Pub/Sub, Big Query, Cloud SQL, BigTable, BigLake DataForm, DataFlow, DataStream, Google cloud storage, Cloud Composer/DAG, Cloud Run, Cloud RESTAPI, ADO GITREPO, CI/CD Pipelines, Secret Manager, Cloud IAM Terraform/YAML etc.
• ETL pipeline using Python builds and scalable solutions.
• Multi-level Data Curation and modeling.
• Data design and architecture.
• Hands on experience in building complete CI/CD Pipeline creation and maintenance using Azure DevOps and Terraform/Terragrunt.
• Increase the efficiency and speed of complicated data processing systems.
• Collaborating with our Architecture group, recommend and ensure the optimal data architecture.
• Analyzing data gathered during tests to identify strengths and weaknesses of ML Models
• increase the efficiency and speed of complicated data processing systems.
• Collaborate across all functional areas to translate complex business problems into optimal data modeling and analytical solutions that drive business value.
• Lead the improvements and advancement of reporting and data capabilities across the company, including analytics skills, data literacy, visualization, and storytelling.
• Develop a certified vs. self-service analytics framework for the organization.
• Collaborating with our Architecture group, recommend and ensure the optimal data architecture.
• Highly skilled on RDMS (Oracle, SQL server), NoSQL Database, and Messaging services (Publish / Subscribe) systems.
• Extensive knowledge/coding skills of Python including understanding of data modeling and data engineering.
What You Bring to the Table:
• Bachelor’s degree in computer science, Engineering, Mathematics, Sciences, or related field of study from an accredited college or university; will consider a combination of experience and/or education.
• Ideally 5+ years of experience in developing data and analytics solutions and approximately 4+ years data modeling and architecture.
• Expertise in programming languages including Python and SQL.
• Familiarity with certain software development methodologies such as Agile, or Scrum.
• Critical thinking.
• Leveraging cloud-native services for data processing and storage.
• Storage – BigQuery, GCS, Cloud SQL, BigTable, BigLake
• Event processing – Pub/Sub, EventArc
• Data pipeline and analytics – Dataflow, DataForm, Cloud Run, Cloud Run Function, DataStream, Cloud Scheduler, Workflows, Composer, Dataplex, ADO GITREPO, CI/CD Pipelines, Terraform/YAML
• Security – Secret Manager, Cloud IAM
• Others – Artifact Registry, Cloud Logging, Cloud Monitoring
• Work with distributed data processing frameworks like Spark.
• Strong knowledge of database systems, and data modeling techniques.
• Ability to adapt to evolving technologies and business requirements.
• Ability to explain technical concepts to nontechnical business leaders.
• Monitor system performance and troubleshoot issues.
• Ensure data security.
• Proficiency in technical skills, cloud tools and technologies.
Technical Skills:
• Must Have
• GCP (Google Cloud Platform)
• ETL pipeline using Python
• Expertise in programming languages including Python and SQL.
Got Extra to Bring?
• GCP – Professional Data Engineer Certification
• Documenting all steps in the development process
• Manage the data collection process providing interpretation and recommendations to management
