Profilbild von Roland Johann Data Architect Stream Processing/Big Data, Data Engineer, Software Engineer aus Koeln

Roland Johann

verfügbar

Letztes Update: 12.04.2024

Data Architect Stream Processing/Big Data, Data Engineer, Software Engineer

Firma: one focus GmbH
Abschluss: nicht angegeben
Stunden-/Tagessatz: anzeigen
Sprachkenntnisse: deutsch (Muttersprache) | englisch (gut)

Skills

Stream Processing, DevOps, Apache Kafka, Kafka Streams, Ansible, Azure Cloud, IoT, Apache Spark, Apache Cassandra, Scala, cloud, data storage, hadoop, Apache, Cassandra, Big Data, kubernetes, SOA, Analytics, AWS, ETL, Alteryx, Spark, HDF, Hive, Amazon Web Services, Software as a Service, AWX, Terraform, Docker, OpenStack, Streams, Spring Framework, bug fixes, Cloud implementations, Apache Zeppelin, Hashicorp, Brocolli, persistent storage, Cloudera, Kerberos, API, Code Review, Spring Boot, HDFS, Prometheus, Grafana, Message Broker, Event Sourcing, CQRS, functional programming, stateful, Flume, Impala, sensor, Data Warehouse, Java, PostgreSQL, Jenkins, Google Cloud, CI/CD, Microservices, Node.js, AWS EMR, R Studio, S3, Apache Hadoop, Apache Parquet, JSON, JSON schema, Streaming, Parquet, data integration, development environment, VPC, EC2, AutoScaling, EMR, Akka Streams, SSTable streaming, Cassandra Cluster, Webshop, Google Analytics, SQL, data ingestion, Apache Kudu, Apache Impala, Apache Hive, Kudu, virtualization, MapR, Logstash, Elasticsearch, cluster layout, automated deployment, REST API, InfluxDB, Machine Learning, ReactJS, Time series analysis, React, JS, Apache Flume, Apache Avro, Apache HBase, HBase, Dependency Injection, Google, Guice, SaaS, Apache YARN, Hashicorp Nomad, YARN, Service Discovery, AngularJS, SBT, Angular, Avro, HTTP API, Reverse Engineering, libs, MySQL, Spring MVC, REST application, REST APIs, frontend, Puppet, Ubuntu 14.04, OS, Load Balancing setup, Play Framework, Hibernate, Spring MVC Application, PHP, Wordpress, backend, WebSocket, Doobie, Shapeless, TypeScript, GraphQL, OAuth2, CentOS, Github, web application, JDBC, data handling, Stream Analytics, Kibana, WebSockets, Spotify, ES6, BEM, SASS, Docker Compose, database, Lucene

Projekthistorie

12/2018 - 12/2020
Consultant, Architect, DevOps Engineer, Data/Software Engineer

Customer Industry Retail, E-Commerce
Project Consultant Stream Processing Architecture, Big Data Introduction for User Tracking Implementation
and BI
Roles Consultant, Architect, DevOps Engineer, Data/Software Engineer
Focus Workshop, Architecture, Implementation
Team Languages German/English
Toolset Hortonworks HDP/HDF, Apache Spark, Apache Hive, Apache Kafka, Apache Cassandra, Ansible, Amazon Web
Services, Scala, Akka Streams, Kafka Streams

Description of project
* Workshops about Hadoop and its use cases
* Concept and Implementation of Big Data processing platform
* Automated deployment of Hortonworks HDP/HDF
* ETL and aggregation pipeline design
* Implementation of pipelines
* Setup Spark on Kubernetes with Hive Metastore

08/2018 - 12/2020
Architect, Software Engineer
Itellity services Ltd

Customer Itellity services Ltd
Project Crawler Reimplementation
Roles Architect, Software Engineer
Focus Concept, Implementation
Team Languages German
Toolset Akka Streams, Akka HTTP
Description of project
Reimplementation of existing application due to external API changes.
Implementation has been done using Akka Streams to overcome problems and complexity of existing,
mostly side effecting, imperative style implementation.

05/2018 - 12/2020
Consultant, Architect, Data Engineer
Vaillant GmbH

Customer Vaillant GmbH
Project Realtime Streaming Architectures for IoT with Kafka - Consultancy
Roles Consultant, Architect, Data Engineer
Focus Workshop, Architecture and Code Review
Team Languages German
Toolset Apache Kafka, Spring Boot, Hadoop HDFS, Apache Spark, Akka, Akka Stream, Kafka Streams, Docker,
Kubernetes, Prometheus, Grafana

Description of project
* Usage as Message Broker as basis for realtime streaming architectures
* Kafka Internals
* Deployment automation
* Kubernetes
* Plain VM via Ansible and Terraform
* Operations
* Patterns/Solutions, Event Sourcing/CQRS
* Hadoop Data Lake integration
* Intro into functional programming
* High available, distributed, stateful Stream Processing

01/2017 - 12/2020
Data Engineer, Full Stack Software Developer

Project Spotify Playlist Finder
Roles Data Engineer, Full Stack Software Developer
Focus Concept, Implementation
Toolset Implementation Backend
Scala, Akka Streams, Play Framework, PostgreSQL, Cassandra, Elasticsearch
Implementation Frontend
ReactJS, ES6, Webpack2, BEM, SASS
DevOps
Ansible, Docker, Docker Compose


Description of project
Implementation of Akka Streams based Crawler to collect all Playlists of Spotify which will be
stored at database. During project different storage solutions had been evaluated, such as
PostgreSQL, Elasticsearch, vanilla Cassandra and Cassandra with secondary index backed by Lucene.
Frontend implementation to allow search of playlists by specific songs, what's the purpose of this
application as Spotify didn't support that feature.

04/2016 - 12/2020
Consultant, Data Engineer, Software Developer

Customer Media Saturn
Project Webshop User Behaviour Analytics via Google Analytics exports
Roles Consultant, Data Engineer, Software Developer
Focus Concept, Implementation
Team Languages German
Toolset Scala, Spark, Apache Spark SQL, Apache Hadoop, Apache Parquet
Description of project * Ingest of Google Analytics BigQuery Exports into Hadoop Datalake
* Implementation of Spark SQL Jobs to analyze specific events and user behavior

06/2018 - 03/2019
Consultant, Architect, DevOps Engineer, Software Engineer
Allianz SE

Customer Allianz SE
Project Software as a Service Development
Roles Consultant, Architect, DevOps Engineer, Software Engineer
Focus Workshop, Architecture, Implementation
Team Languages German/English
Toolset Apache Kafka, Apache Zeppelin, Hashicorp Nomad, Hashicorp Consul, Docker, Cluster Brocolli, Scala,
Play Framework

Description of project
* advise on Kafka, it's deployment requirements and usage scenarios
* persistent storage concept of Kafka on Nomad
* implementation of Kafka on Nomad including Brocolli integration
* Zeppelin to Cloudera Hive connectivity concept including security with Kerberos
* Implementation of Zeppelin to kerberized Hive

03/2018 - 02/2019
Consultant, Data Engineer
TÜV Rheinland Services GmbH

Customer TÜV Rheinland Services GmbH
Project Condition Monitoring Hadoop Migration
Roles Consultant, Data Engineer
Focus Concept, Implementation
Team Languages German
Toolset Implementation
HDFS, Flume, Parquet, Hive/Impala, Scala, Spark

Description of project * Migrating existing legacy Application to Hadoop Cluster utilizing Big Data Technologies
* ETL of sensor data using Flume and Spark
* Data Lake/Data Warehouse via Hive/Impala

01/2017 - 12/2018
Data Engineer, Software Developer

Project Fintec related real time Stream Analytics
Roles Data Engineer, Software Developer
Focus Concept, Implementation
Toolset Implementation
Scala, Akka Streams, PostgreSQL
DevOps
CentOS, RPM, Grafana, Prometheus, Elasticsearch, Kibana, Logstash, InfluxDB

Description of project Multiple implementations of different stream analytics use cases utilizing Akka Streams via
WebSockets.

06/2018 - 12/2018
Consultant, Architect, DevOps Engineer, Software Engineer

Customer Industry Retail, E-Commerce
Project Software as a Service Development, Private Cloud Implementation, Consultant Stream Processing
Architecture
Roles Consultant, Architect, DevOps Engineer, Software Engineer
Focus Workshop, Architecture, Implementation
Team Languages German/English
Toolset Apache Kafka, Apache Cassandra, Ansible, AWX, Terraform, Docker, OpenStack, Amazon Web Services,
Scala, Akka Streams, Kafka Streams, Spring Framework

Description of project
* advise on Cassandra, it's deployment requirements and usage scenarios
* Implementation of automated Deployment and monitoring of Cassandra including zero Downtime
capable updates
* advice on Kafka, usage patterns in stream processing and service-oriented architectures with
Kafka
* architectural bug fixes of Kafka usage and configuration
* Workshop AWS, introduction into Cloud implementations, cloud ecosystem and tooling
* PoC OpenStack implementation and Workshops

05/2018 - 07/2018
Consultant, Data Architect, Data Engineer
Adidas

Customer Adidas
Project Digital Trading Analytics
Roles Consultant, Data Architect, Data Engineer
Focus Processing Architecture, Tooling
Team Languages German/English
Toolset Apache Spark, Scala, AWS

Description of project * Concept and implementation of ETL and Processing pipeline migration from Alteryx to Apache
Spark

11/2017 - 06/2018
Data Engineer, DevOps
REWE Digital

Customer REWE Digital
Project Recommendations/DSGVO
Roles Data Engineer, DevOps
Focus Concept, Implementation, Training
Team Languages German
Toolset Implementation
Java, Spring Framework, Scala, Akka Streams, Spark, PostgreSQL

DevOps
Docker, Jenkins, Kubernetes, Google Cloud
Description of project
* Offer personalization service concept and implementation
* Implementation of CI/CD pipeline utilizing Jenkins, Docker, Kubernetes
* Training in how to develop and implement CI/CD with Docker, Jenkins, Kubernetes
* Training in Big Data technologies (Apache Spark, Scala, Akka)
* DSGVO: Stream oriented Microservices for handling delete requests targeting several DBMS/Data
Lakes
* Reimplementation of several Node.js ETL applications with Scala and Akka Stream

03/2017 - 10/2017
Data Engineer, Software Developer, Consultant, DevOps
European Space Observatory (ESO)

Customer European Space Observatory (ESO)
Project High-Throughput streaming ingestion of sensor data from Apache Kafka into Apache Cassandra
Roles Data Engineer, Software Developer, Consultant, DevOps
Focus Concept, Implementation
Team Languages German, English
Toolset Implementation
Scala, Apache Spark, Apache Cassandra, Apache Cassandra, Akka Streams
DevOps
AWS, Terraform, Ansible

Description of project
* streaming ingestion of sensor data into apache Cassandra per SSTable streaming
* provisioning of infrastructure and processing components (Spark Cluster, Cassandra Cluster,
Kafka Cluster, Monitoring)

03/2017 - 10/2017
Consultant, Solution Architect, DevOps

Customer KfW
Project Hadoop Cluster Deployment
Roles Consultant, Solution Architect, DevOps
Focus Implementation, Training
Team Languages German
Toolset Hortonworks, Cloudera, MapR, Ansible

Description of project
* Deployment of multiple Hadoop Distributions at tightly secured infrastructure for evaluation
of performance and operational criterias
* Consultation and training to operate Hadoop clusters

12/2016 - 10/2017
Consultant, DevOps
Customer Adidas

Customer Adidas
Project Mini LAB Implementation
Roles Consultant, DevOps
Focus Concept, Implementation
Team Languages German, English
Toolset DevOps
AWS, Terraform, Ansible

Description of project Automated setup of temporary LAB Clusters (AWS EMR, R Studio, S3 Datalake) für Analysten
* AWS infrastructure provisioning via Terraform
* Deployment via Ansible

12/2016 - 10/2017
Consultant, Solution Architect, Data Engineer, Software Developer, DevOps
Adidas

Customer Adidas
Project Generic Streaming Integration of Third-Party-Services as MVP
Roles Consultant, Solution Architect, Data Engineer, Software Developer, DevOps
Focus Concept, Implementation
Team Languages English
Toolset Implementation
Scala, Play Framework, Apache Kafka, Apache Spark, Apache Hadoop, Apache Parquet, AWS EMR

DevOps
AWS, Terraform, Ansible

Description of project
* Concept of platform to ingest JSON records into Hadoop data lake, strictly typed by JSON
schema
* Implementation via Play Framework, Apache Kafka, Apache Spark Streaming, Parquet on HDFS

12/2016 - 10/2017
Data Engineer, Software Developer, Consultant, DevOps
Adidas

Customer Adidas
Project Reference Architecture of data integration platform as MVP
Roles Data Engineer, Software Developer, Consultant, DevOps
Focus Implementation
Team Languages German, English
Toolset DevOps
AWS, Terraform, Ansible

Description of project
First phase of MVP implementation by implementing basic provisioning of infrastructure, which will
be reused by different projects inside Adidas Group
DevOps
Automatic provisioning of AWS resources via Terraform
* Implementation of non production environment clone as development environment (dedicated VPC,
Subnets, Route Tables)
* provisioning of basis infrastructure and -components including EC2 instances, AutoScaling
Groups, EMR Clusters with auto scaling

12/2016 - 10/2017
Consultant, Solution Architect, Data Engineer, Software Developer, DevOps
Adidas

Customer Adidas
Project Concept of transactional data ingestion into data lake
Roles Consultant, Solution Architect, Data Engineer, Software Developer, DevOps
Focus Concept, Implementation, Evaluation of Technologies
Team Languages English
Toolset Implementation
Scala, Akka, Apache Spark, Apache Kudu, Apache Impala, Apache Hadoop, Apache Parquet, Apache Hive,
AWS EMR

DevOps
AWS, Ansible, Terraform

Description of project
* Concept to replace current implementation via parquet files
* Evaluation of different solutions (architectural, technological)
* Evaluation of Kudu as Big Data Storage in comparison to parquet on HDFS and other custom
implemented solution following lambda architecture principles
* Deployment of Cloudera Cluster and Vanilla Kudu via Ansible on provisioned AWS clusters
automated by Terraform

12/2016 - 10/2017
Consultant, Solution Architect
Adidas

Customer Adidas
Project Concept of data integration platform reference architecture
Roles Consultant, Solution Architect
Focus Concept
Team Languages German, English
Description of project Concept for integration of global distributed data sources into data lake including data
virtualization layers and near-realtime access scenarios

11/2016 - 12/2016
Consultant, Data Engineer, DevOps
Institut für Materialforschung

Customer Institut für Materialforschung
Project Automated deployment of Hadoop Clusters
Roles Consultant, Data Engineer, DevOps
Focus Concept, Implementation
Team Languages German
Toolset Ansible, Scala, Flume, Logstash, Elasticsearch
Description of project
* Concept of Hadoop cluster layout of 10 nodes and distribution of ist components
* Deployment of Hadoop distribution Hortonworks via Ansible
* Concept and implementation o fingest pipelines via Flume and Logstash
* Implementation of deployment and configuration automation of edge services:
* Elasticsearch
* Kafka
* PostgreSQL

06/2016 - 12/2016
Consultant, Solution Architect, Data Engineer, DevOps

Customer Metro Systems
Project Ansible/Cassandra Deployment Automation
Roles Consultant, Solution Architect, Data Engineer, DevOps
Focus Consultation, Training, Concept, Implementation
Team Languages German
Toolset Ansible, Scala, Akka, Play Framework, Apache Cassandra, PostgreSQL, Docker
Description of project
* Review of existing deployment processes, in specific Cassandra
* Training of Ansible, process automation
* Concept and implementation of micro service architecture with Akka and Play Framework for
automated deployment of clustered applications per REST API
* Advised on monitoring Cassandra with InfluxDB and Grafana

06/2016 - 10/2016
Solution Architect, Data Engineer, Full Stack Software Developer

Customer Media Saturn
Project PoC Sales Forecast
Roles Solution Architect, Data Engineer, Full Stack Software Developer
Focus Concept, Implementation
Team Languages German
Toolset Scala, Spark, Play Framework, Elasticsearch, Machine Learning, ReactJS
Description of project
* Analysis of required data sources and entities
* Ingestion of external data sources
* Stream/Batch Processing via Spark
* Implementation of machine learning at stream processing: Time series analysis of sales data
* Implementation of REST API with Play Framework for downstream consumers
* React JS Frontend
* Dashboard implementation to visualize sales data including the forecasts
* prepared concept and implementation of PIM integration into datalake and analytics pipelines

06/2016 - 10/2016
Solution Architect, Data Engineer, Software Developer, DevOps

Customer Media Saturn
Project Data Integration Platform
Roles Solution Architect, Data Engineer, Software Developer, DevOps
Focus Concept, Implementation
Team Languages German, English
Toolset Scala, Apache Kafka, Apache Spark Streaming, Apache Hadoop, Apache Flume, Apache Avro, Apache HBase
Description of project
* Concept of Hadoop Cluster and its components
* Automated deployment of hadoop cluster
* Automated deployment of Apache Flume and its components
* Concept and implementation of ingest pipeline automation
* Concept and implementation of data lake using Hadoop HDFS and HBase via Hive
* Training
* Implementation of additional ingest pipelines with Apache Flume
* Usage of Apache Zeppelin and Apache Spark
* Implementation of Spark Jobs
* Dependency Injection mit Spark Jobs via Google Guice

06/2016 - 10/2016
Data Engineer, Software Developer, DevOps
Customer Allianz

Customer Allianz
Project SaaS Infrastructure Implementation
Roles Data Engineer, Software Developer, DevOps
Focus Concept, Implementation
Team Languages German, English
Toolset Scala, Apache Spark, Apache Hadoop, Apache YARN, Hashicorp Nomad, Docker
Description of project
* Automated setup of cluster development environments
* Automated setup of cluster development via Ansible
* Apache Spark Standalone
* Apache YARN
* Apache Spark on YARN
* Apache Hadoop HDFS
* Apache Zeppelin
* Concept and implementation of Cluster/Container Manager Hashicorp Nomad
* Concept and implementation of Service Discovery via Hashicorp Consul

06/2016 - 07/2016
Solution Architect, Data Engineer, Full Stack Software Developer
T-Online

Customer T-Online
Project Competitor Monitor MVP
Roles Solution Architect, Data Engineer, Full Stack Software Developer
Focus Concept, Implementation
Team Languages German
Toolset Scala, Apache Spark, Akka, Elasticsearch, Machine Learning, Docker, AngularJS
Description of project
Concept and implementation of scaling, high throughput message driven microservice architecture.
* Akka (Microservices)
* Apache Kafka (Message Broker)
* Apache Cassandra (Persistence)
* Kamon (Monitoring Scala API via aspectj), InfluxDB und Grafana
* Docker/Docker-Compose/SBT Docker (Development Environment)
* Play Framework (REST API)
* Angular JS (Frontend, SPA)

01/2016 - 06/2016
Solution Architect, Data Engineer, Software Developer, DevOps
T-Online

Customer T-Online
Project Date integration of external services data
Roles Solution Architect, Data Engineer, Software Developer, DevOps
Focus Concept, Implementation
Team Languages German
Toolset Scala, Apache Spark, Akka, Play Framework, Apache Flume, Apache Hadoop, Apache Avro
Description of project
* Deployment of cross DC Cloudera Clusters
* Implementation of Akka Job Schedulers with downstream Actor System
* Stream based Ingestion of external Services into Hadoop Datalake as Avro Files and into
Elasticsearch
* HTTP API for delivering aggregations to downstream systems

01/2016 - 02/2016
Software Developer
Itellity GmbH

Customer Itellity GmbH
Project Automated Invoice download of Vodafone Invoice Portal
Roles Software Developer
Focus Concept, Implementation
Team Languages German
Toolset Scala, Akka, Play Framework
Description of project * Reverse Engineering of invoice portal HTTP API
* Implementation of Akka Actor System with Play JSON libs

05/2015 - 12/2015
DevOps, Full Stack Software Developer
Itravel

Customer Itravel
Project Implementation of cloud infrastructure and migration of existing services from on-premise DC
Roles DevOps, Full Stack Software Developer
Focus Concept, Implementation
Team Languages German
Toolset Scala, AWS, Puppet, Jenkins
Description of project
* Analysis of existing infrastructure and it's problems
* Concept of cloud based, automated, high available infrastructure at AWS
* Implementation
* Nodes with Ubuntu 14.04 LTS OS
* Provisioning via Puppet
* Load Balancing setup
* Auto Scaling of relevant services
* Setup of CI/CD via Jenkins
* Implementation of CD
* Port of several services onto cloud based infrastructure

03/2015 - 11/2015
Full Stack Software Developer

Customer itravel
Project Maintenance/Implementation of new features of customer portal and HTTP API
Roles Full Stack Software Developer
Focus Concept, Implementation
Team Languages German
Toolset Scala, Java, Play Framework, Spring MVC, Hibernate, MySQL
Description of project * Concept and implementation into existing Spring MVC Application
* bugfixes
* manual deployments
* Maintenance

02/2015 - 06/2015
Full Stack Software Developer
Itravel; Focus

Customer Itravel
Project Reimplementation of internal Sales/Product Tool
Roles Full Stack Software Developer
Focus Konzept, Implementierung
Team Languages Deutsch
Toolset Scala, Akka, Play Framework, MySQL, Java
Description of project
* Definition of featuresets based on existing implementation and user interviews
* Creation of fronted mocks to visualize possible workflow automation
* Integration of legacy Spring MVC application into Play Framework REST application
* Implementation of several Akka applications
* Concept and implementation of several REST APIs
* Concept and implementation frontend
* Setup of CI and CD
* Concept and implementation of server infrastructure migration to AWS

01/2015 - 03/2015
Full Stack Software Developer
Focus

Customer itravel
Project Online Competition
Roles Full Stack Software Developer
Focus Konzept, Implementierung
Team Languages German
Toolset Scala, Akka, Play Framework, Angular JS, PHP
Description of project
* Concept with Itravel PR Team
* Implementation at Wordpress
* Integration of own product to gather social metrics and topic-/category identification
* Monitoring of competition and implementation of statistic tools

Reisebereitschaft

Verfügbar in den Ländern Deutschland
Profilbild von Roland Johann Data Architect Stream Processing/Big Data, Data Engineer, Software Engineer aus Koeln Data Architect Stream Processing/Big Data, Data Engineer, Software Engineer
Registrieren