A highly scalable key-value storage system for latency sensitive applications

  • Qiang Wang*
  • , Feilong Tang
  • *Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Scopus citations

Abstract

This paper presents the design of a highly scalable key-value storage system for building latency sensitive applications. To be able to scale up to an extremely large system, the design adopts a decentralized and symmetrical architecture and uses consistent hashing for partitioning. Every node keeps a full membership of all the nodes in the system and can route a lookup in one hop. Server and network failures occur at any time and the storage system is dynamic for nodes join and leave the system continually. We design a hierarchical gossip based membership protocol to capture the accurate membership information of the system with acceptable memory and communication overhead. The system also replicates data items on multiple nodes to achieve high availability and high durability.

Original languageEnglish
Title of host publicationProceedings of the International Conference on Complex, Intelligent and Software Intensive Systems, CISIS 2009
Pages537-542
Number of pages6
DOIs
StatePublished - 2009
Externally publishedYes
EventInternational Conference on Complex, Intelligent and Software Intensive Systems, CISIS 2009 - Fukuoka, Japan
Duration: 16 Mar 200919 Mar 2009

Publication series

NameProceedings of the International Conference on Complex, Intelligent and Software Intensive Systems, CISIS 2009

Conference

ConferenceInternational Conference on Complex, Intelligent and Software Intensive Systems, CISIS 2009
Country/TerritoryJapan
CityFukuoka
Period16/03/0919/03/09

Fingerprint

Dive into the research topics of 'A highly scalable key-value storage system for latency sensitive applications'. Together they form a unique fingerprint.

Cite this