ML Data Engineer

Описание вакансии

ML Data Engineer

Локация: Удалённо

Компания: Snoonu

ЗП: обсуждается на собеседовании

Занятость: Полная

We look for experienced engineer for position of dedicated DE for ML team. Main responsibilities will include creating end extending existing feature marts and realtime marts, including investigation of feature drift and performance optimizations. What we also consider as important is having enough skills to take responsibility for full ML-DE interaction, including new pipelines creation in existing DE infrastructure. Main tools used for feature engineering are Dbt, BigQuery, Databricks and Spark structured streaming, having AWS streaming services and infrastructure as a code for DE part.

🛠 What You’ll Get Your Hands On:

• Feature engineering - support and creating feature marts for ML team, including tests and validation
• Realtime Feature engineering - creating realtime pipelines, extending realtime feature store with new features
• Integrating new sources into existing data pipelines for ML needs, ensuring data quality and consistency
• Monitor and automate ML pipelines processes, including data drift, pipelines performance, alerting

🧙‍♂️ The Magic You Bring:

• Experience in several programming languages or Python frameworks
• PySpark knowledge i.e. Spark structured streaming, data distribution, optimizations
• Strong SQL skills, ability to apply subqueries, window functions, CTEs, math specific database functions
• Experience with modern ETL processes, collection, ingestion, storage, cleaning, transformation, analysis of data
• Software design skills - software patterns, application, data system design
• Math statistics knowhelege, i.e. stdev, quartiles, ability to apply them for drift or outlier detection
• Infrastructure as a code is a plus
• Experience with databricks as a plus

🌐 Резюме отправлять: Откликнуться

–––

Бесплатный постинг вакансий: @freeIT_job

Забирай 📚 Базу Знаний