For one of our clients, we are looking for a Senior Data Engineer to design data architectures and build distributed pipelines across cloud and on-premises environments. Your work will make data ready for operational use, analytics and machine learning.
- Design and maintain scalable data architectures and pipelines across cloud and on-premises environments.
- Integrate diverse data sources and ensure processing is reliable and reproducible.
- Test and monitor distributed pipelines and estimate performance and costs for different use cases.
- Collaborate with data scientists to prepare datasets for machine learning and build dashboards that make processed data accessible to users.
- Degree in computer science, mathematics or a related field.
- Several years of experience in technical roles, with strong Python and SQL skills and a focus on maintainable software.
- Sound knowledge of big data infrastructure, scalable databases and distributed processing.
- Good understanding of public cloud services such as AWS, Azure or Google Cloud.
- Experience with or interest in tools such as Spark, Hadoop, Hive, Airflow or Kubernetes, and familiarity with relational or NoSQL databases and DevOps practices.
- Fluent German and English.