Senior Big Data Engineer
Siemens AG (DI CS DE&DS DSM MAC)
Skills
Azure, Kubernetes, Hive, Spark, PostgreSQL, Camel, Knative, Big Data, Hadoop, MapReduce, Presto, Pyspark, Python, Numpy, Pandas, Hbase, AWS, cloud, IOT, S3, SQS, EMR, Redshift, Docker, Git, analytics, CloudFormation, ETL tools, Talend, Alteryx, AWS Glue, App development, data management, ETL, database, SQL-Server, MySQL, OrientDB, Excel, Microsoft Excel, VBA, PHP, Outsystems, Programming, JAVA applications, VBA applications, MS-Excel, web client, visualization, MS-Access
Projekthistorie
09/2018
-
06/2021
* Preparation, consolidation, and transformation of large (un) structured
data by using modern big data technologies such as Hadoop,
MapReduce, Presto, Pyspark, Python, Numpy, Pandas, Hive, Hbase, Livy,
Jupyterlab)
* Mainly responsible for the independent design, creation, deployment, and
management of big data pipelines within the AWS cloud infrastructure
(IOT Core, Kinesis, S3, Lambda function, SQS, Glue, EMR, Athena,
Redshift, ELK, Kubernetes, Docker, Git)
* Build data lake as a centralized repository to store structured and
unstructured data for advanced analytics - data Ingestion, big data
processing, real-time analytics (S3, Glue data catalog, Athena, EMR,
Redshift, delta lake, Databricks)
* Automatic setup for EMR clusters as well as continuous performance
improvement using AWS CloudFormation
* Develop AI Model for customer-oriented projects (focusing on xgboost)
* Responsible for 15+ POCs (Customers from automotive industry) as
Senior Data Engineer
* Automation of data preparation using ETL tools (Talend, Alteryx, AWS
Glue)
* App development using low-code framework Mendix
data by using modern big data technologies such as Hadoop,
MapReduce, Presto, Pyspark, Python, Numpy, Pandas, Hive, Hbase, Livy,
Jupyterlab)
* Mainly responsible for the independent design, creation, deployment, and
management of big data pipelines within the AWS cloud infrastructure
(IOT Core, Kinesis, S3, Lambda function, SQS, Glue, EMR, Athena,
Redshift, ELK, Kubernetes, Docker, Git)
* Build data lake as a centralized repository to store structured and
unstructured data for advanced analytics - data Ingestion, big data
processing, real-time analytics (S3, Glue data catalog, Athena, EMR,
Redshift, delta lake, Databricks)
* Automatic setup for EMR clusters as well as continuous performance
improvement using AWS CloudFormation
* Develop AI Model for customer-oriented projects (focusing on xgboost)
* Responsible for 15+ POCs (Customers from automotive industry) as
Senior Data Engineer
* Automation of data preparation using ETL tools (Talend, Alteryx, AWS
Glue)
* App development using low-code framework Mendix
09/2012
-
08/2018
Data Engineer
Cynatics Consulting GmbH
"process and data management" at Cynatics Consulting GmbH /
Siemens DI S CIC (Siemens external Employee)
* Automation of ETL Process using Python, Pyspark, Presto, Hive, AWS
Glue, Talend, Alteryx
* Development and maintenance of database applications based on the
tools Microsoft SQL-Server / MySQL and OrientDB
* Maintenance and further development of the internal database application
(MM-BIB)
* Development and maintenance of terminology-based data management
templates (Excel) for the creation of structured product master data using
Microsoft Excel-VBA and SQL-Server
* Development and maintenance of a web spider for downloading and for
storing product data using PHP
* App development using low-code framework Outsystems
Siemens DI S CIC (Siemens external Employee)
* Automation of ETL Process using Python, Pyspark, Presto, Hive, AWS
Glue, Talend, Alteryx
* Development and maintenance of database applications based on the
tools Microsoft SQL-Server / MySQL and OrientDB
* Maintenance and further development of the internal database application
(MM-BIB)
* Development and maintenance of terminology-based data management
templates (Excel) for the creation of structured product master data using
Microsoft Excel-VBA and SQL-Server
* Development and maintenance of a web spider for downloading and for
storing product data using PHP
* App development using low-code framework Outsystems
Reisebereitschaft
Weltweit verfügbar
