Beschreibung
Our Client is looking for a DevOps engineer with a background in big data engineering.
Candidate will support an excellent and sociable team of data scientists and data engineers by creating and automating solutions in the Clients' Classifieds Group Openstack cloud. This is a great opportunity for a talented engineer to shape the architecture for new data-driven products within one of Germanys top websites.
Key Qualifications:
* Experience and proficiency in data automation and management, particularly Hadoop, HDFS and Spark
* Experience with Elasticsearch, Apache Cassandra and Apache Kafka as search and data backends
* Experience configuring and spawning cloud instances (Openstack, AWS, Azure)
* Experience using ansible or saltstack environments for the cloud is a big plus
* Experience in managing and using data with relational databases, eg MySQL and Sqoop
* Experience with Cloudera Hadoop Distribution is a big plus
* Experience using git and preferably github are a big plus
* Experience with Java applications (Streams, Akka) are a big plus
* Knowledge of Talend software or other data ingestion pipelines is a plus
Description:
Located within the site operations group, the engineer will support a separate data team by bringing their use-cases and workloads to an Openstack cloud. We rely on dual datacentre and switchover capabilities, so that a general understanding of these challenges is desired and additional engineering might be needed. Git/Github, Saltstack and Jenkins are used for automated system integration. Sensu is used for monitoring. The Cloudera Hadoop cluster is provided as a service by another group, so that the engineer can concentrate on the architectural part of the product development itself including production-ready stability and performance analysis. Data pipelines might have to be designed properly from the beginning to the end. Solutions should in general follow the KISS principle, but we value proactive and innovative colleagues by providing trust and freedom of action.