Exploring small-world-like topologies via SplitProber: Turning power laws into an advantage in unstructured overlays

Recent unstructured Peer-to-Peer systems, represented by Gnutella and Freenet, offer an administration-free and fault-tolerant application-level overlay network. While elegant from a theoretical perspective, these systems have some serious disadvantages. First, due to knowing very little about the nature of the network topology, the search algorithms operating on these networks result in fatal scaling problems. Second, these systems rely on application-level routing, which may be inefficient with respect to network delays and bandwidth consumption. In this paper, we propose a novel search algorithm, called Split-Prober, to explore the small-world-like topologies of these networks efficiently and scalablely, by turning the power-law degree distributions in these networks to an advantage, and by making discriminative use of nodes according to their different roles in the network. As a result, we are able to reconcile the conflict of remedying the mismatch between the overlay topology and its projection on the underlying physical network, while at the same time navigating these networks with a guaranteed high efficiency and using only local, knowledge as cues. Our simulation results indicate that the proposed algorithm outperforms several other well-known methods with significant performance gains.