Beschreibung
We are currently seeking individuals that have the experience required to join our Company as Big Data Engineer Data Management & Processing collaborating to build one of the most advanced big data platforms in the world.
Essential Duties and Responsibilities:
- Design, build, deploy and maintain distributed data pipelines and infrastructure for batch and near Real Time processing, with focus on scalability and data quality
- Work with data scientists, data analysts, Back End engineers and product managers to leverage the data we produce and process to the fullest extent
- Optimize and migrate existing data flows into the big data platform, add various new data sources
- Solve challenging problems in areas such as distributed systems, integration of machine learning algorithms and data processing
- Degree in Computer Science, Software Engineering, or equivalent
- At least 2 years' experience of building and optimizing scalable data systems as a Data Engineer, with possible background as a Software Engineer
- Good programming skills in Java or Scala (Scala is a plus)
- Hands-on experience with big data technologies such as Kafka, Spark, Hadoop FS, (Hive, Impala, Drill or similar), planning the capacity and estimating hardware requirements
- Good knowledge of Linux and Shell Scripting
- Solid experience writing and tuning SQL queries and working with relational databases like PostgreSQL
- Solid understanding of Key value Stores and unstructured data
- Familiarity with workflow frameworks like Airflow and Apache NiFi/StreamSet
- Nice to have: experience with AWS, Kubernetes and overall DevOps attitude (you build it, you run it)
- You are a great team player and you communicate fluently in English
- Ability to work with encrypted or privacy related data (GDPR)
- Exceptional English communication skills, both written and verbally
- Remarkable people skills and ability to adapt to a dynamic environment
- Enjoy working in a fast-paced start-up
- Able to work from our office in Berlin
Required knowledge, skills and abilities:
We offer:
- An international and smart team which is eager to revolutionize the way of monetarize digital assets
- A transparent company culture with highly motivated colleagues, which work on our ambitious goals
- Team Events on a regular basis
- An honest and open feedback culture
- First two weeks on-site, 100% remote thereafter
Contract Conditions:
Daily Rate: €325p/d
Contract Length: 3 months + extensions
Location: Berlin, DE