Beschreibung
Freelance Data Engineer (Python, SQL, AWS)asap – 31.12.2022
Remote / Frankfurt 10%
German speaking
# Project description
In an interdisciplinary team, agile methods are developed in the areas of big data analytics, mathematical optimization and simulation. We use these procedures in all business areas of the company in order to make planning and operating processes more efficient and to better tailor the offer to the needs of end customers. For this purpose, we operate a data platform on-premises and in the AWS Cloud. The data preparation in the data platform contains various technical logic and data links. We are looking for support for operation, maintenance, further and new development of loading lines. The data is stored and provided on PostgreSQL. Python and SQL are used for the implementation of the loading distances. The runtime environments are Linux servers and containers. Cloud services use AWS RDS, AWS Batch, AWS ECS, and AWS ECR, among others. The CI/CD pipelines and code management are done via GitLab. Confluence is used for knowledge management and agile project management is supported by Jira. The cooperation takes place with other data engineers and domain experts, so that an adequate induction is guaranteed.
# Service description
• Proactive familiarization with the technical context of the project
• Advising the project team on the maintenance and operation of existing loading lines
• Advising the project team on the maintenance and operation of infrastructure on-premises and in the AWS Cloud
• Further and new development of charging routes in Python, SQL, Docker, GitLab and AWS in cooperation with the project team
• Communication with users when implementing requirements
• Dokumentation in Confluence
• Code management in GitLab
• Support in the conceptual development of the data platform
# Must requirements:*
• At least 3 years of experience in the field of data engineering in the enterprise environment
• At least 1 year of experience in the design and code-based development of ETL lines
• Very good KNOWLEDGE OF SQL
• Very good python skills
• Very good knowledge of Linux
• Experience in code management via Git
• Knowledge of CI/CD, ideally with GitLab
• Experience with containers, e.g. with Docker
• Experience with enterprise network environments (firewalling, security)