Senior Data Engineer
•
•
•
• We are not in a position to sponsor candidates for employment for this position nor can we work with anyone in a corp-to-corp arrangement. W2 only! No exceptions!
•
•
•
• Senior Data Engineer
The Senior Data Engineer is a key development resource responsible for designing, coding, testing, implementing, documenting, and maintaining NextGen solutions for GCP Cloud enterprise data initiatives. This role involves close collaboration with data teams, often within a matrixed environment as part of a larger project team. Given the rapidly evolving nature of GCP/Hadoop technology, staying informed about technological advancements and effectively applying new innovations is essential.
Key Responsibilities:
• Analyze business requirements, perform design tasks, construct, test, and implement solutions with minimal supervision.
• Build and support a GCP-based ecosystem for enterprise-wide analysis of structured, semi-structured, and unstructured data.
• Integrate new data sources into GCP, transform and load data into databases, and manage data transfers between clusters.
• Develop a deep understanding of relevant product areas, codebases, and systems.
• Demonstrate proficiency in data analysis, programming, and software engineering.
• Collaborate with the Lead Architect and Product Owner to define, design, and build new features and improve existing products.
• Produce high-quality code with good test coverage, using modern abstractions and frameworks.
• Work independently, completing tasks on schedule by exercising strong judgment and problem-solving skills.
• Collaborate closely with team members to execute development initiatives using Agile practices and principles.
• Participate in the deployment, change, configuration, management, administration, and maintenance of deployment processes and systems.
• Prioritize workload effectively to meet deadlines and work objectives in a rapidly changing business environment.
• Work collaboratively with Data Scientists and business and IT leaders to understand their needs and use cases.
• Engage in technical group discussions and adopt new technologies to improve development and operations.
Experience:
• 5+ years of relevant IT work experience.
• 3+ years of experience as a Data Engineer.
Qualifications:
• Bachelor’s degree in Computer Science or a related discipline.
Required Skills:
• Strong understanding of best practices and standards for GCP data process design and implementation.
• 2+ years of hands-on experience with the GCP platform and familiarity with components such as:
• Cloud Run, GKE, Cloud Functions
• Spark Streaming, Kafka, Pub/Sub
• Bigtable, Firestore, Cloud SQL, Cloud Spanner
• HL7, FHIR, JSON, Avro, Parquet
• Python, Java, Terraform
• BigQuery, Dataflow, Data Fusion
• Cloud Composer, DataProc, CI/CD, Cloud Logging
• Vertex AI, NLP, GitHub
• Ability to multitask and balance competing priorities.
• Strong problem-solving skills and the ability to impose order in a fast-changing environment.
• Excellent verbal, written, and interpersonal skills, with a desire to work in a highly matrixed, team-oriented environment.
Preferred Skills:
• Experience in the healthcare domain and with patient data.
• Experience with AWS or Azure cloud platforms.
• Familiarity with hardware/operating systems such as Linux, UNIX, and distributed, highly scalable processing environments.
• Knowledge of databases including RDBMS (MS SQL Server, Teradata, Oracle) and NoSQL (Hbase, Cassandra, MongoDB, in-memory, columnar, and other emerging technologies).
• Experience with build systems (TFS, Maven, Ant), source control systems (Git, Mercurial), and continuous integration systems (Jenkins, Bamboo).
Certifications (a plus, but not required):
• GCP Cloud Professional Data Engineer
Other Requirements:
• Prolonged sitting at a computer.
• Ability to participate in after-hours support.
•
•
•
• We are not in a position to sponsor candidates for employment for this position nor can we work with anyone in a corp-to-corp arrangement. W2 only! No exceptions!
•
•
•
• Senior Data Engineer
The Senior Data Engineer is a key development resource responsible for designing, coding, testing, implementing, documenting, and maintaining NextGen solutions for GCP Cloud enterprise data initiatives. This role involves close collaboration with data teams, often within a matrixed environment as part of a larger project team. Given the rapidly evolving nature of GCP/Hadoop technology, staying informed about technological advancements and effectively applying new innovations is essential.
Key Responsibilities:
• Analyze business requirements, perform design tasks, construct, test, and implement solutions with minimal supervision.
• Build and support a GCP-based ecosystem for enterprise-wide analysis of structured, semi-structured, and unstructured data.
• Integrate new data sources into GCP, transform and load data into databases, and manage data transfers between clusters.
• Develop a deep understanding of relevant product areas, codebases, and systems.
• Demonstrate proficiency in data analysis, programming, and software engineering.
• Collaborate with the Lead Architect and Product Owner to define, design, and build new features and improve existing products.
• Produce high-quality code with good test coverage, using modern abstractions and frameworks.
• Work independently, completing tasks on schedule by exercising strong judgment and problem-solving skills.
• Collaborate closely with team members to execute development initiatives using Agile practices and principles.
• Participate in the deployment, change, configuration, management, administration, and maintenance of deployment processes and systems.
• Prioritize workload effectively to meet deadlines and work objectives in a rapidly changing business environment.
• Work collaboratively with Data Scientists and business and IT leaders to understand their needs and use cases.
• Engage in technical group discussions and adopt new technologies to improve development and operations.
Experience:
• 5+ years of relevant IT work experience.
• 3+ years of experience as a Data Engineer.
Qualifications:
• Bachelor’s degree in Computer Science or a related discipline.
Required Skills:
• Strong understanding of best practices and standards for GCP data process design and implementation.
• 2+ years of hands-on experience with the GCP platform and familiarity with components such as:
• Cloud Run, GKE, Cloud Functions
• Spark Streaming, Kafka, Pub/Sub
• Bigtable, Firestore, Cloud SQL, Cloud Spanner
• HL7, FHIR, JSON, Avro, Parquet
• Python, Java, Terraform
• BigQuery, Dataflow, Data Fusion
• Cloud Composer, DataProc, CI/CD, Cloud Logging
• Vertex AI, NLP, GitHub
• Ability to multitask and balance competing priorities.
• Strong problem-solving skills and the ability to impose order in a fast-changing environment.
• Excellent verbal, written, and interpersonal skills, with a desire to work in a highly matrixed, team-oriented environment.
Preferred Skills:
• Experience in the healthcare domain and with patient data.
• Experience with AWS or Azure cloud platforms.
• Familiarity with hardware/operating systems such as Linux, UNIX, and distributed, highly scalable processing environments.
• Knowledge of databases including RDBMS (MS SQL Server, Teradata, Oracle) and NoSQL (Hbase, Cassandra, MongoDB, in-memory, columnar, and other emerging technologies).
• Experience with build systems (TFS, Maven, Ant), source control systems (Git, Mercurial), and continuous integration systems (Jenkins, Bamboo).
Certifications (a plus, but not required):
• GCP Cloud Professional Data Engineer
Other Requirements:
• Prolonged sitting at a computer.
• Ability to participate in after-hours support.
