Data Platform Engineer - OLAP

Описание вакансии

This is Adyen

Adyen provides payments, data, and financial products in a single solution for customers like Meta, Uber, H&M, and Microsoft - making us the financial technology platform of choice. At Adyen, everything we do is engineered for ambition. 

For our teams, we create an environment with opportunities for our people to succeed, backed by the culture and support to ensure they are enabled to truly own their careers. We are motivated individuals who tackle unique technical challenges at scale and solve them as a team. Together, we deliver innovative and ethical solutions that help businesses achieve their ambitions faster.

Sitting at the intersection of Data Engineering, Backend Engineering, and Systems Engineering, Data Platform Engineers at Adyen build the foundational layer of tooling and processes for our on-premise Analytical Data Platforms. These tools support 10s of products, 100s of developers, and 1000s of daily jobs that add to Adyen’s strong portfolio of capabilities.

We’re looking for an expert with deep knowledge in distributed systems, to focus on our internal Online Analytical Processing (OLAP) ecosystem. You’ll collaborate with Data and ML Engineers to continuously improve this ecosystem. You’ll also collaborate with other platform engineers to position this ecosystem properly within the larger Data/AI/ML Platform capabilities, powered by Hadoop, Kubernetes, Spark, Trino, Flink, and Ray.

What you’ll do

Your responsibilities will focus on building new tools and improving existing tooling within the Data Platform ecosystem. Key focus areas include:

• Performance at Scale: Develop and maintain high-performance OLAP systems, supporting multi-tenant query workloads and ingestion pipelines with real Big Data scale.
• Reliability: Work with system reliability in mind, ensuring high availability for business-critical analytical products through observability and engineering excellence.
• Productize the Platform: Build self-service tooling that enables Data Engineers and Analysts to independently manage their data assets and diagnose issues.
• Data Quality: Engineer automated frameworks to validate data integrity and leverage metadata-driven tools to enhance data discoverability, lineage, and cataloging across the ecosystem.
• Ecosystem Integration: Architect seamless integrations of the OLAP ecosystem with adjacent distributed systems (e.g. storage, messaging, and batch / stream processing systems)
• Efficiency & Governance: Monitor and optimize cluster resource-efficiency while making sure the platform adheres to global security and data privacy standards.

Who you are

• Fluency in Python and/or Java. Golang or Rust are also appreciated.
• Team player with strong communication skills
• Ability to work closely with diverse stakeholders you enable (analysts, data scientists, data engineers, etc.) and depend upon (infrastructure, security, etc).
• Experience in OLAP technologies, like Druid, Clickhouse, Pinot, Doris, Starrocks, etc
• Experience in CI/CD pipelines, for code and infrastructure automation.
• Experience in Kubernetes
• Experience in infrastructure and large-scale private cloud systems
• Additional experience developing and maintaining:

• Other distributed data and compute systems like Spark, Trino, etc
• Data modelling for databases
• Real-time and batch data pipelines (via Kafka, Spark streaming) with an eye for frameworks, and emphasis on user friendliness and quality..

Our Diversity, Equity and Inclusion commitments

Our unique approach is a product of our diverse perspectives. This diversity of backgrounds and cultures is essential in helping us maintain our momentum. Our business and technical challenges are unique, and we need as many different voices as possible to join us in solving them - voices like yours. No matter who you are or where you’re from, we welcome you to be your true self at Adyen. 

Studies show that women and members of underrepresented communities apply for jobs only if they meet 100% of the qualifications. Does this sound like you? If so, Adyen encourages you to reconsider and apply. We look forward to your application!

What’s next?

Ensuring a smooth and enjoyable candidate experience is critical for us. We aim to get back to you regarding your application within 5 business days. Our interview process tends to take about 4 weeks to complete, but may fluctuate depending on the role. Learn more about our hiring process here. Don’t be afraid to let us know if you need more flexibility.

This role is based out of our Amsterdam office. We are an office-first company and value in-person collaboration; we do not offer remote-only roles.