Data Engineering Technologies

  • Data Integration and Orchestration:

    Apache Airflow, AWS Lambda, AWS Step Functions, Azure Functions, Azure Data Factory

  • Streaming and Batch Processing:

    Apache Kafka, Apache Spark Amazon Kinesis, Amazon MSK, AWS Batch, AWS Glue, Amazon EMR Azure Event Hubs, Azure Batch, Azure Databricks, Azure HDInsight

  • Databases and Distributed Storage:

    PostgreSQL, Microsoft SQL Server, MongoDB, Apache Cassandra, Cloudera Distributed Hadoop Amazon DynamoDB, Amazon RDS, Amazon S3 Azure CosmosDB, Azure SQL Database, Azure Blob Storage, Azure Data Lake

  • Logging and Monitoring:

    Prometheus, Grafana, Amazon CloudWatch, Azure Monitor

My Chatbot