Dr. Etienne Rivière (Université de Neuchâtel)
Building robust and self-organizing large-scale distributed systems: an introduction to the gossip paradigm
Large-scale distributed systems, such as peer-to-peer and edge cloud systems, are complex to operate and manage. Their scale prevents from building global and accurate knowledge of their constituents and their interactions, and they are typically subject to high levels of dynamism. In particular, they exhibit high level of churn, which is a measure of the number of nodes leaving and joining the system over time.
The gossip-based computing paradigm is based on periodic interactions between pairs of nodes in a large-scale distributed system. While remaining simple and elegant, it provides strong high-level properties. The most important of these properties is the ability for systems to self-organize. A self-organizing system is able to recover from any incorrect state (e.g., after a period of high churn) back to a correct state, without requiring explicit repair mechanisms or human intervention.
In these two lectures, I will introduce a series of gossip-based protocols for key operations in large-scale distributed systems.
In the first lecture, I will start by the seminal example of robust data dissemination. I will then detail the principles and construction of gossip-based peer sampling, the basis for self-organizing properties of all other gossip-based protocols. We will finally cover how gossip-based principles allow computing global functions over a set of peers using aggregation.
In the second lecture, we will focus on emerging structure in large-scale systems using gossip-based protocols. We will detail how simple node-centric selection functions, together with a peer sampling layer, allow emerging self-organizing distributed structures. We will then put all the covered protocols together and detail the construction of a self-organizing topic-based publish/subscribe system based on a combination of these protocols.
Back to EBSIS Events section.