Skip to content

Latest commit

 

History

History
44 lines (33 loc) · 1.65 KB

README.md

File metadata and controls

44 lines (33 loc) · 1.65 KB

Educational Project: National Highway Traffic Data Analysis Project

Overview

This project, part of an educational program, aims to de-congest national highways by analyzing road traffic data from various toll plazas. Using Kafka, we stream data like vehicle_id, vehicle_type, toll_plaza_id, and timestamp as vehicles pass through toll plazas. This repository contains scripts and instructions for setting up a data pipeline to collect this streaming data and load it into a database for further analysis.

Prerequisites

  • Apache Kafka
  • Zookeeper
  • Python 3.x
  • Relevant Python libraries: (list any specific libraries used in the scripts)

Installation & Setup

  1. Zookeeper Setup:

    • Start the Zookeeper server, a prerequisite for Kafka.

  2. Kafka Setup:

    • Start the Kafka server after initializing the Zookeeper server.
    • Create a Kafka topic named toll for streaming the toll data.

Components

  1. Toll Traffic Generator (toll_traffic_generator.py):

    • Simulates traffic data at toll plazas.
    • Streams data to the Kafka topic.

  2. Streaming Data Reader (streaming_data_reader.py):

    • Reads streaming data from Kafka.
    • Processes and loads data into a database.

Usage

  1. Running the Toll Traffic Generator:

    • Run python3 toll_traffic_generator.py.
    • Ensure Kafka is consuming data from the toll topic.

  2. Running the Streaming Data Reader:

    • Run python3 streaming_data_reader.py.
    • Verify data loading into the database.

Contributing

This project is part of an educational program. Contributions adhering to project standards are welcome.

License

MIT License

Copyright (c) 2024 Farhad Vadiee