Real-time Activity Tracking with Kafka

Website user activity tracking as a set of real-time publish-subscribe feeds

Use Case Description

Hogwarts is in deep waters lately. After Dolores Umbridge took over the headmastership, not even a pixie could flutter without her knowing. She made sure that school’s webmaster, Prof. Quirinus Quirrell is also dancing under her imperius. All he had to do is to monitor the web activities for the news articles and find which are gaining momentum. Original website looks as follows, with 0 read counts.

Proposed Solution

Quirrell deligates this task to one of the Slytherin students, Gregory Goyle. As the solution for this, Goyle has decided to use an ESB to integrate this web server with a Kafka server to provide real-time activity tracking. When the audience queries for the full news article, the request is used to populate an internal statistics engine to process read counts. The requirements can be broken down into 3 key segments.

• Retrieve the full news article and record the impression
• Persist the records, passing data into Hadoop or data warehousing systems for real-time processing and reporting
• Update the read count back on the server