Beschreibung
#Basic data• Role: Data Intelligence Engineer
• Period: approx.
• Consultant days: approx. 60 BT (with the option of extension)
• Location: Mainz (remote 100%)
• Availability: 100% full time - 5 days / week - ASAP
• Languages ??German
• Industry: Transport
#Task description:
Development of a data pipeline for the evaluation of data. Every day, larger amounts of data are generated that also need to be analyzed. The data is in JSON format and has a heterogeneous structure made up of several interconnected levels. If certain conditions are met during the analysis, tickets are also created on Jira. In addition, part of this data is also displayed using dashboards. All components are currently provided on a Docker cluster. It is intended that the data storage and query are provided on a data lake.
# Must requirements
• Python
• Experience with Spark, Celery or any similar distributed systems
• Docker
# Target requirements
• AWS S3
• Apache Airflow
• RedHat OpenShift (Kubernetes)
• QlikSense
• Celery
• MongoDB
• Amazon Redshift / PostgreSQL
Seniority Level
Mid-Senior level
Industry
Staffing and Recruiting
Employment Type
Full-time
Job Functions