Data Engineering Technologies

• Data Integration:

  Apache Airflow, Azkaban Apache, NiFi

• Data Processing:

  Apache Hadoop, Apache Kafka, Apache Spark, Amazon Kinesis, Amazon Athena, Amazon EMR, Azure Data Lake Analytics, Azure Databricks, Azure HDInsight

• Database & Data Warehouse:

  Apache Cassandra, MongoDB, Apache Hive, Apache HBase, Amazon DynamoDB, Amazon Redshift Spectrum, Azure CosmosDB

• Distributed Storage:

  HDFS, Apache Kudu, Amazon S3, Amazon EBS, Azure Blob Storage, Azure Data Lake Storage