09.10.2024 aktualisiert
NG
100 % verfügbar
Senior Data Engineer with Expertise in Python and Cloud Infrastructure
Berlin, Deutschland
Weltweit
BSc. Computer Science and Behavioural NeuroscienceSkills
Amazon Web ServicesMicrosoft AzureBusiness IntelligenceGoogle BigQueryCloud ComputingDatenbankenInformation EngineeringETLDatenmanagementDatenvisualisierungDynamoPythonPostgreSQLMachine LearningDataOps
Python Development
Extensive experience in Python programming, including creating libraries for statistical analysis and implementing business intelligence solutions.
Cloud Infrastructure
Proficiency in working with various cloud platforms including GCP, AWS, and Azure for infrastructure deployment and data management.
Data Engineering
Strong skills in data engineering, including experience with Databricks, Spark, and implementing ETL processes for large-scale data operations.
Database Management
Knowledge of various database systems including PostgreSQL, Dynamo, and BigQuery for efficient data storage and retrieval.
Containerization
Familiarity with Docker for containerization, enabling efficient deployment and scaling of applications.
Data Visualization
Experience in creating data visualizations and dashboards using tools like Plotly.
Machine Learning Operations
Skills in MLOps tools such as Airflow and Kubernetes for orchestrating and managing machine learning workflows.
Extensive experience in Python programming, including creating libraries for statistical analysis and implementing business intelligence solutions.
Cloud Infrastructure
Proficiency in working with various cloud platforms including GCP, AWS, and Azure for infrastructure deployment and data management.
Data Engineering
Strong skills in data engineering, including experience with Databricks, Spark, and implementing ETL processes for large-scale data operations.
Database Management
Knowledge of various database systems including PostgreSQL, Dynamo, and BigQuery for efficient data storage and retrieval.
Containerization
Familiarity with Docker for containerization, enabling efficient deployment and scaling of applications.
Data Visualization
Experience in creating data visualizations and dashboards using tools like Plotly.
Machine Learning Operations
Skills in MLOps tools such as Airflow and Kubernetes for orchestrating and managing machine learning workflows.
Erfahrung
Berufserfahrung5 - 10 Jahre
Freelancer seit2024
Sprachen
EnglischMutterspracheFranzösischverhandlungssicher
Projekthistorie
Developed global experimentation tools, migrated to automated infrastructure deployment using Terraform with Atlantis on GCP, created Python libraries for statistical analysis, implemented custom logging with Datadog, and deployed data clean up jobs on EKS cluster.
Implemented OCR parsers for financial bank statements and fraud detection, leveraged Databricks on Azure to implement a 100GB+ ETL pyspark solution, and worked closely with data scientists and software engineers.
Implemented and designed business intelligence dashboard using Plotly dash in Python and managed structure queries from the user with appropriate fault tolerance.