Pinned Loading

1. apache-airflow-study apache-airflow-study Public

   Python code that implement simple etl on Apache Airflow

   Python

2. Hive_emr_python Hive_emr_python Public

   Hive streaming using Python and Hive transform function

   Python

3. kafka-stream-poc kafka-stream-poc Public

   POC on how to use kafka-stream that read AVRO from Kafka topic and filter only the desire value to print to console.

   Java

4. kafka_connect_rds_to_s3_json kafka_connect_rds_to_s3_json Public

   Retrieve the data from Posgresql on RDS (non CDC) and ingest to AWS S3 as Json String.

   HCL

5. kafka_publisher_json_to_azure_event_hub kafka_publisher_json_to_azure_event_hub Public

   This demo is for test kafka publisher that publish json string to Azure Event Hub (enable Kafka support)

   Java

6. poc_streaming_twitter_to_kafka_to_spark_to_hdfs poc_streaming_twitter_to_kafka_to_spark_to_hdfs Public

   I try to build the data pipeline that read the twitter stream and store tweet data into HDFS

   Python