Principal Data Engineer with Apache Flink

Skills:

Apache Spark, DynamoDB, MongoDB, NoSQL, Java, Scala, big data pipelines,

Job Title : Principal Data Engineer with Apache Flink/Apache Spark

Location California based team)

Duration : Long-term

Job Description

What were looking for:

Youre a talented, creative, and motivated engineer who loves developing powerful, stable, and intuitive apps and youre excited to work with a team of individuals with that same passion. Youve accumulated years of experience, and youre excited about taking your mastery of Big Data and Java to a new level. You enjoy challenging projects involving big data sets and are cool under pressure. Youre no stranger to fast-paced environments and agile development methodologies in fact, you embrace them. With your strong analytical skills, your unwavering commitment to quality, your excellent technical skills, and your collaborative work ethic, youll do great things here at client location.

What Youll Do

As a Principal Data Engineer, youll be responsible for designing and building high performance, scalable data solutions that meet the needs of millions of agents, brokers, home buyers, and sellers. Youll design, develop, and test robust, scalable data platform components. Youll work with a variety of teams and individuals, including product engineers to understand their data pipeline needs and come up with innovative solutions. Youll work with a team of talented engineers and collaborate with product managers and designers to help define new data products and features.

Skills, Accomplishments, Interests You Should Have

BS/MS in Computer Science, Engineering, or related technical discipline or equivalent combination of training and experience.

8+ years core Scala/Java experience: building business logic layers and high-volume/low latency/big data pipelines.

5+ years of experience in large scale real-time stream processing using Apache Flink or Apache Spark with messaging infrastructure like Kafka/Pulsar.

5+ years of experience on Data Pipeline design, Data architecture, ETL and processing of structured and unstructured data.

3+ years of experience using NoSQL systems like MongoDB, DynamoDB and Relational SQL Database systems (PostgreSQL) and Athena.

Excellent understanding of Architecture, system Design including nonfunctional requirements and experience evaluating tech stack/product to decide build vs. buy.

Experience with technologies like Lambda, API Gateway, AWS Fargate, ECS, CloudWatch, S3, DataDog.

Experience defining and owning technical/data direction of the pipelines.

Excellent written and verbal communication skills in English.

Strong work ethic and entrepreneurial spirit.