Data Engineer - ETL - Freelance

Berlin  ‐ Vor Ort

Schlagworte

Continuous Integration ETL Airflow Amazon S3 Continuous Delivery Information Engineering Datenqualität Github Hadoop Distributed File System Json Job Scheduling Extensible Markup Language Parquet Avro

Beschreibung

If you are a Data Engineer with solid experience with ETL then i have a fantastic opportunity for you

Fully Remote

12 month contract

Starting the beginning of July

English speaking project

We are seeking a skilled and motivated Data Engineer to join our team. The ideal candidate will have proven expertise in onboarding new data sources, writing ETL processes, and scheduling jobs in Airflow. You will work with a variety of storage forms and data formats, ensuring data integrity and efficiency.Key Responsibilities:

  • Identify and integrate new data sources into existing data pipelines.
  • Ensure data quality and consistency during the onboarding process.
Required Skills:

  • Technical Expertise:
    • Proven experience working with Spark and Scala.
    • Expertise in using Airflow for job scheduling and management.
    • Familiarity with various storage forms such as HDFS, Object Storage, and S3.
    • Proficiency in working with data formats like AVRO, Parquet, JSON, and XML.
  • CI/CD Proficiency:
    • Experience with Continuous Integration/Continuous Deployment (CI/CD) practices.
    • Proficiency in using tools such as Airflow and Github.


If you are passionate about data engineering and have the required skills, we encourage you to apply for this exciting opportunity. Join our team and contribute to building robust and efficient data pipelines that drive our business forward.

Darwin Recruitment is acting as an Employment Business in relation to this vacancy.
Start
07/2024
Dauer
12 months +
(Verlängerung möglich)
Von
Darwin Recruitment
Eingestellt
11.06.2024
Projekt-ID:
2760721
Vertragsart
Freiberuflich
Um sich auf dieses Projekt zu bewerben müssen Sie sich einloggen.
Registrieren