Peer-to-Peer (P2P) computing has gained a lot of attention from both the scientific and the large Internet user community. Popular applications utilizing this new technology offer many attractive features to a growing number of users while taking up a large amount of everyday network traffic. This talk presents bandwidth-efficient and adaptive algorithms to facilitate data location and processing for massive data management applications that operate on P2P overlays. The basis of these schemes is their ability to learn from past interactions, increasing their performance with time.

In the first part of the talk, previous work in efficient content location and distribution for Unstructured Peer-to-Peer overlays is described. The Adaptive Probabilistic Search (APS) scheme utilizes directed walkers to forward queries on a hop-by-hop basis. Peers store success probabilities for each of their neighbors in order to efficiently route towards object holders. The stored state can be used to extend the protocol to support efficient data dissemination and replication based on the demand incentive. In the GrouPeer project, we apply many of these techniques in order to identify and group peers with similar schemas in an interconnected network of autonomous databases.

In the second part of the talk I will present some of my current work which focuses on indexing methods for data and query-intensive applications over DHTs. HiPPIS and PASSION are systems that utilize adaptive algorithms that automatically adjust the level of indexing (for hierarchically organized data or ranges respectively) according to the granularity of the incoming queries, without assuming any prior knowledge of the workload. In both cases, special consideration is given to the minimization of network traffic, as we assume local only knowledge for each network host.