Scaling User Activity with Configurable Campaign Management System

In one of my recent projects, I contributed to building a configurable system aimed at increasing the active user base for both customers and merchants. The system enables non-technical teams to define logic and rules (like database queries) to offer incentives such as money, bonus points, or coupons to users, driving user engagement.

Business Perspective

The core idea behind this project was to grow the active user base by allowing business teams to easily configure campaign strategies. Whether it's incentivizing inactive users or rewarding active users, the system is designed to enable flexible and automated campaign management.

Non-tech teams can configure business logic through the system, defining who qualifies for campaigns and how rewards should be disbursed—whether through money, bonus points, or other incentives.

System Architecture

This system supports two distinct user groups: customer users and merchant users, with separate frontends but a unified backend. The architecture was designed as a modular monolith, incorporating microservices to handle specific business functions.

Key Microservices

1. Frontend

   • Developed using React, Redux, and Material UI for both customer and merchant UIs.

2. Authentication and Authorization (EDA-Auth)

   • MSAL for SSO (Single Sign-On) and LDAP (previously used).
   • Component-based authorization stored in PostgreSQL (later migrated to MySQL), with a Node.js application utilizing Sequelize ORM.

3. API Backend (bmw)

   • A Spring Boot application responsible for generating campaign modalities and producing them in Kafka topics.
   • Campaign summary PDFs are generated and sent to third-party services for statistical reporting.

4. Audience Generation Backend (amw)

   • Another Spring Boot service that triggers Hive workflows to generate campaign audiences. The audience data is stored in HDFS, OracleDB, and ScyllaDB.

5. Communication Backend (cmw)

   • Handles campaign communication, sending messages to different MNOs (Mobile Network Operators) based on the campaign settings.

6. Disbursement Backend (dmw)

   • Manages disbursements of rewards (money, points, or coupons) to eligible customers.

7. Real-Time Transaction Evaluation (RT-Engine)

   • Built as a Java Streams application, this Kafka-based engine processes real-time transactions, checking whether a customer belongs to a campaign audience and whether their transactions meet the reward criteria.
   • The application interacts with Kafka Clusters, ScyllaDB, OracleDB, and Redis Cache for high-performance filtering.

Cloud and Data Infrastructure

• AWS: We utilized various AWS services, including EC2, Lambda, and DynamoDB. For instance, Lambda was employed to create short links from the CMS, while DynamoDB stored the data, which was then cached in Redis for faster access.

• Cloudera: This platform played a vital role in audience generation. Based on campaign modalities, Cloudera queries customer data and ingests it into HDFS and OracleDB for further processing. The audience generation includes both fixed and dynamic bases, with dynamic audiences updated daily.

Databases

The project utilized a diverse range of databases, including:

• OracleDB for high-performance, relational data storage.
• ScyllaDB for distributed NoSQL storage to handle massive amounts of audience and campaign data.
• MySQL and PostgreSQL for various backend services.
• AWS DynamoDB for quick access and integration with AWS Lambda.

Technology Stack

• Frontend: React, Redux, Material UI
• Backend: Spring Boot (API backend, audience generation, communication, disbursement), Java Streams for RT-Engine
• Kafka: For messaging and real-time transaction processing
• Databases: OracleDB, ScyllaDB, MySQL, DynamoDB
• Cloud: AWS (EC2, Lambda, DynamoDB), Redis for caching
• Big Data: Cloudera (Hive, HDFS)

Campaign Workflow

1. Campaign Configuration: Non-technical teams define the audience and campaign modalities using the system’s interface.
2. Audience Generation: The audience is generated either statically (once) or dynamically (daily updates), depending on campaign requirements.
3. Real-Time Transaction Monitoring: Transactions are continuously evaluated by the Kafka-based RT-Engine to determine campaign eligibility.
4. Reward Disbursement: Eligible users receive rewards through the disbursement service, ensuring timely and accurate incentives.

Outcome

This project empowered business teams to drive user engagement through configurable campaigns without needing technical expertise. It provided flexibility to experiment with different strategies, ultimately helping the organization grow its active user base efficiently.

This project gave me an opportunity to work on a complex, modular system while collaborating with multiple teams to ensure both technical and business objectives were met. Feel free to reach out if you’d like more details about the technical architecture or the business impact!