How does Neo4j Causal Clustering work

In this article, we will explore Causal Clustering in Neo4j.

Purpose of Clustering

When operating a database, you must choose between running a single instance or setting up a cluster using Causal Clustering.

Single Instance Case

Running a single instance is straightforward. Since all read/write requests are handled by a single instance, there are no concerns about consistency.

However, in a production environment, reliability and scalability issues will quickly arise. No matter how much you scale up a single instance vertically, there is a limit to the number of requests per second (RPS) it can handle. Additionally, if the instance becomes unavailable due to hardware or network failures, downtime will occur.

Clustering Case

In production environments, clustering is typically implemented. In Neo4j, the Leader node handles all write requests, while Replica nodes scale out to handle read requests.

For reliability, if the Leader node fails, the Raft Protocol is used to promote one of the Replica nodes to the new Leader, ensuring continuous operation without downtime.

For scalability, additional Replica nodes can be added to meet read request demands. However, write request scaling requires vertical scaling of the Writer node.

Consistency

When clustering is implemented, consistency becomes a key concern.

If data replication occurs asynchronously, there will be a lag between the Leader and Replica nodes. For example, if a write operation is performed on the Leader and a read request is sent to a Replica at the same time, the Replica may return outdated data due to replication delays.

Read-your-write Consistency

To address this issue, the Read-your-write consistency model ensures that a user will always read their own writes consistently.

For example, in the following case, a user sends a write request to the Leader node. After the data is updated in the Leader, it is replicated to the Replica nodes. If a read request is sent after synchronization is complete, the user receives the expected result.

However, in the following scenario, the user receives outdated data. If the read request reaches the Replica before synchronization is complete, the outdated data is returned.

It is important to note that Read-your-write consistency does not guarantee consistency for writes made by other users. For example, if a user updates their Twitter profile, another user may still see the old profile information until replication is complete.

Solution in Causal Clustering

Neo4j ensures Read-your-write consistency using a mechanism called Bookmarks.

Conceptually, a transaction that includes a write request is assigned a Bookmark, which acts as a marker. This marker is then used to determine which node should handle subsequent transactions, ensuring consistent reads.

Because this clustering method guarantees causality, it is referred to as Causal Clustering.

Summary

In this article, we introduced Causal Clustering in Neo4j. It is an essential mechanism for ensuring scalability and reliability in production environments. Be sure to take advantage of it in your Neo4j deployments!