Beschreibung
Working in an international team of developers for big data analytics services (time series).Develop batch- and streaming data pipelines in MS Azure from various data sources.
Create and test data pipelines of structured/unstructured data from internal and external sources in a variety of formats with Data Factory.
Set up CI/CD pipelines with Azure DevOps.
Pre-process data, data cleaning and gap-filling with Python, Apache Spark and Databricks.
Data modelling in Adture Delta Lake, parquet and SQL.from internal and external sources in a variety of formats
Good team player in an international virtual team, but great at independent work. Solution oriented. Analytical thinking. Data & analytics enthusiast.
Master or Ph.D. in Computer Science, Mathematics, or related technical discipline.
Minimum 3 years working experience as a hands-on data engineer, working with below technologies:
Coding in Python and Spark.
Azure Delta Lake, Azure Data Factory, Azure DevOps, Databricks.
Data modeling & database design for time series data.
Source control management systems (git) and CI/CD flows.