Beschreibung
Aufgaben:·The Data Engineer is a vital role in delivering data-driven solutions to our clients
·The engineer will be responsible for various aspects of data and AI projects, such as data ingestion, transformation, storage, processing, analysis, and visualization
Must-Have-Skills:
·A deep understanding of data engineering concepts and methodologies, such as ETL/ELT, data quality, data governance, data modeling
·An extensive experience of building and maintaining data pipelines using different technologies, such as Python, SQL, Scala, Java, etc.
·An expert level of proficiency in working with various data platforms and services, such as Data Lake, Lakehouse, Data Warehouse, Cloud, etc.
·A familiarity with Apache ecosystem and open source technologies, such as Kafka, Spark, Airflow, Hadoop, Hive, etc.
·Experience with Hadoop (especially HDFS)
·An ability to design and implement scalable, reliable, and secure data solutions using best practices and industry standards
·A demonstrated experience with data analysis and visualization tools, such as Power BI, Tableau, Qlik, etc.
·Experience in integrating open-source solutions on an enterprise scale with adequat monitoring, security and compliance
·Familiarity with streaming technologies (eg. GCP PubSub)
·Experience in integrating Apache Connect solutions for OT/IoT/IT data
·Familiarity with IoT and OT protocols for data streaming
·A familiarity with data science and machine learning concepts and frameworks, such as TensorFlow, PyTorch, Scikit-learn, etc.
Start: 06.05.2024
Dauer: 8 Monate
Einsatzort: Remote 100%
Branche: EDV-Dienstleistungen
Auslastung: 100%