RedTAO: A Trillion-edge High-throughput Graph Store

  • Shihao Zhou
  • , Qi Mao*
  • , Yi Cheng
  • , Hongcheng Qi
  • , Yilun Huang
  • , Peng Cai*
  • , Jun Peng Zhu
  • *Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

With the explosive growth of daily active users, the social graph data of Xiaohongshu has scaled to trillions of edges, imposing high pressure on our storage system. Current state-of-the-art systems struggle to address the issue, primarily due to: (1) Traditional relational databases as the back-end storage require frequent scaling, incurring high cost and stability risks. (2) Most graph databases focus on complex multi-hop queries. Redundant components in these systems make them difficult to take advantage when processing our workloads dominated by one-hop queries. (3) Using cache systems like Redis or Memcache often struggles to ensure consistency between the cache and storage. In this paper, we propose RedTAO, which has a scalable and efficient graph cache layer optimized for social scenarios. Over 90.7% of queries are served directly by the cache, enabling us to focus on scaling it as traffic increases. RedTAO employs cross-cloud, multi-active deployment, synchronizing replicas through the storage layer. The cache layer directly accesses local storage, avoiding costly cross-region requests. Additionally, the data transmission service (DTS) component asynchronously corrects cache data, ensuring cache consistency. RedTAO has been successfully deployed in Xiaohongshu, achieving a 1.8× throughput improvement and at least 21.3% reduction in resource usage compared to the previously used MySQL architecture.

Original languageEnglish
Title of host publicationSIGMOD-Companion 2025 - Companion of the 2025 International Conference on Management of Data
EditorsAmol Deshpande, Ashraf Aboulnaga, Babak Salimi, Badrish Chandramouli, Bill Howe, Boon Thau Loo, Boris Glavic, Carlo Curino, Daisy Zhe Wang, Dan Suciu, Daniel Abadi, Divesh Srivastava, Eugene Wu, Faisal Nawab, Ihab Ilyas, Jeffrey Naughton, Jennie Rogers, Jignesh Patel, Joy Arulraj, Jun Yang, Karima Echihabi, Kenneth Ross, Khuzaima Daudjee, Laks Lakshmanan, Minos Garofalakis, Mirek Riedewald, Mohamed Mokbel, Mourad Ouzzani, Oliver Kennedy, Oliver Kennedy, Paolo Papotti, Peter Alvaro, Peter Bailis, Renee Miller, Senjuti Basu Roy, Sergey Melnik, Stratos Idreos, Sudeepa Roy, Theodoros Rekatsinas, Viktor Leis, Wenchao Zhou, Wolfgang Gatterbauer, Zack Ives
PublisherAssociation for Computing Machinery
Pages716-728
Number of pages13
ISBN (Electronic)9798400715648
DOIs
StatePublished - 22 Jun 2025
Externally publishedYes
Event2025 ACM SIGMOD/PODS International Conference on Management of Data, SIGMOD-Companion 2025 - Berlin, Germany
Duration: 22 Jun 202527 Jun 2025

Publication series

NameProceedings of the ACM SIGMOD International Conference on Management of Data
ISSN (Print)0730-8078

Conference

Conference2025 ACM SIGMOD/PODS International Conference on Management of Data, SIGMOD-Companion 2025
Country/TerritoryGermany
CityBerlin
Period22/06/2527/06/25

Keywords

  • database
  • graph store
  • social graph

Fingerprint

Dive into the research topics of 'RedTAO: A Trillion-edge High-throughput Graph Store'. Together they form a unique fingerprint.

Cite this