CARAMEL: A Microservice-Based Infrastructure for Scalable Big Social Data Management
Paulo Freitas Silva Júnior1, Tiago Cruz de França2 and Jonice Oliveira1
1 PPGI, Universidade Federal do Rio de Janeiro (UFRJ), Rio de Janeiro, RJ, Brazil (Emails: , )
2 DECOMP, Universidade Federal Rural do Rio de Janeiro (UFRRJ), Seropédica, RJ, Brazil (Email: )
DOI: https://doi.org/10.48545/advance2026-shortpapers-6_3
Abstract
Social media platforms propagate massive volumes of semantically rich data at high speed; however, researchers often face significant technical impediments in data collection and a lack of artifact reuse. Current analysis tools generally rely on monolithic architectures that hinder collaboration, limit independent scalability, and compromise data provenance. To address these challenges, we propose CARAMEL, a microservices-based infrastructure designed for the scalable and collaborative management of Big Social Data. The framework utilizes containerization and asynchronous messaging to achieve temporal decoupling between high-speed data ingestion and multi-stage analysis. We describe the infrastructure’s architecture, emphasizing its ability to integrate heterogeneous sources and automate metadata curation to ensure data quality and traceability. Evaluation through large-scale case studies conducted during the 2022 and 2024 Brazilian elections demonstrates the viability of the infrastructure, processing millions of data points across multiple social channels while maintaining resilience under high demand. The proposed CARAMEL infrastructure contributes a flexible, standards-based environment that allows research communities to share, reuse, and scale social data workflows effectively.
Keywords
Social Data Management, Workflow Management, Big Data Infrastructure, Scalability, Reproducibility