Data Engineer

Beschreibung

Data Engineer Hadoop / Spark / Scala developer


Required skills:

- At-least 5 years of Hadoop development and implementation experience - Experience with MapReduce and tools based on MR such as Pig etc. - Experience with Apache Spark, Hive, HDFS, - Experience with Cloudera, Sentry, Hue, Impala - Test prototypes and oversee handover to operational teams - Propose best practices/standards. - Loading from disparate data sets. - Pre-processing using Hive and Pig. - Translate complex functional and technical requirements into detailed design. - Perform analysis of vast data stores and uncover insights. - Maintain security and data privacy. - High-speed querying - must be able to implement Spark Scripts in Scala (maintain, extend) -

Nice to have skills:
- Managing and deploying HBase. - Create scalable and high-performance web services for data tracking.