Beschreibung
A Big Data Architect (m/w/d) for our data science consultancy client needed! You will be working for an end-client in the transport industry.The project duration is 3.5 months and it will be remote. The project language is German.
Project duration: 17.08.2020 (or 01.09.2020) till 06.11.2020 (or till 15.12.2020)
Project description
This particular project is focusing on the development and expansion of a data storage, processing and analysis platform (Datalake). This mainly involves machine or sensor data of trains. The aim is to view and evaluate the existing architecture and to show optimisation potential, especially with regard to performance, stability, compatibility, costs and sustainability.
Service description
- Assessment of the current Datalake architecture according to defined, comprehensible criteria with the involvement of other internal experts and users of the platform
- Deduction of recommendations for the further development of the Datalake platform by naming concrete technologies/tools
- Elaboration and written documentation of the advantages, possible disadvantages and challenges of using the recommended technology stack (target image). This also includes a differentiation from technologies that are not favoured but are customary on the market in this context.
- Derivation of a potential migration path from the current technology stack to the target image. Indication of special challenges/pitfalls that could arise in the context of migration
- Organisation and implementation of workshops, especially for the transfer of knowledge
Must-Have skills:
- Structured approach
- Strong analytical and conceptual skills
- Architecture and implementation knowledge of the AWS stack (e.g. ECS, EMR, Athena, Glue, Redshift, Kinesis or MSK, Lambda, Python)
- Architecture and implementation knowledge of the Hadoop Hortonworks stack (e.g. Spark, Kafka, HDFS, Yarn, Hive)
- Development and implementation of concepts for data preparation and data integration (engineering) in the AWS environment
- Presentation know-how
Nice-to-have skills:
- Architecture and implementation knowledge of relational databases, SQL, ETL and BI tools
- project management expertise
I am looking forward to hear from you :)
Katarzyna Bie?ko
Resource Consultant
Harvey Nash GmbH
Phone:
Email: