Beschreibung
Our client are looking for a Contract Data Engineer that comes from a Software Engineering background.
This person will develop new functionalities/solutions in the following areas:
- Data Governance
- Knowledge Graph
- Customer facing interfaces which allow the user to find and use data
- Real Time ingestion services
The Data Engineer should have the following:
- Knowledge of concepts and processes related to data ingestion ETL, the transformation and storage of data, and the use/abstraction of data.
- Good experience developing cloud big data products/solutions
- Experience using Kafka & Spark to build events based and distributed systems.
- Very good knowledge of the Apache Kafka platform
- Good knowledge of typical databases: SQL, Postgres etc
- Proficiency in at least one relevant programming language
They should also have the following DevOps skills:
- Very good Linux experience
- Very good cloud storage and containerisation using things like Docker, Kubernetes etc
- Experience at CI/CD
- Knows about Security concepts and principals
- Good knowledge of Infrastructure
- Good knowledge of Orchestration
- Good knowledge of Distributed Systems
The consultant should feel comfortable speaking to the business about relevant data engineering topics/technologies. They will be expected to propose solutions to the business and be able to clearly communicate solution pros and cons/capabilities and limitations. The engineer will keep up to date with new data engineering technologies and data features such as: streaming; storage; security; management; processing. They will help develop and improve the clients Big Data stack/platform/live environment through increased performance, better reliability, and clever automation. They will help address platform issues/bugs and implement changes to the environment when required. They will contribute to technical documents and use different media channels to share platform relevant news and best practices.