ClickHouse Cluster Setup and Configuration


Being a full-stack ClickHouse Optimization Support and Managed Services provider company we often get queries on ClickHouse installation and configuration for both standalone and clustered infrastructure setup. So we decided to write this blog which will help anyone interested in the setup and configuration of ClickHouse. This document is purely recommended for learning ClickHouse installation and configuration, please don’t ever use this document as a checklist or guidance for installation and configuration of ClicHouse on your production infrastructure. MinervaDB or any of their group companies/subsidiaries are not responsible for any kind of damages caused to your business from following this document for your production setup.  Technically, ClickHouse installation is quite straight forward and you can run ClickHouse on any of the Linux, FreeBSD, or Mac OS X with x86_64, AArch64, or PowerPC64LE CPU architecture.

ClickHouse installation on Debian systems:

From RPM packages

Add the official repository to install from pre-compiled rpm packages for CentOS, RedHat, and all other rpm-based Linux distributions:

ClickHouse installation from repository configured above:

Installation and configuration of ClickHouse from source:

You can install and configure ClickHouse from the source here: https://github.com/ChistaDATA/ClickHouse 

Single server with docker:

Run server

Run client

Step-by-step ClickHouse Cluster setup

  • We will have 1 cluster with 3 shards in this setup
  • Each shard will have 2 replica servers
  • We are using ReplicatedMergeTree and Distributed table for this setup

Cluster setup

We have copied below docker-compose.yml for your reference:

So we have 6 ClickHouse Server containers and one ZooKeeper container. For successful replication, We have to configure ZooKeeper optimally. It’s purely ClickHouse which will take care of Database reliability and consistency in a replicated infrastructure.  Between, We do have customers who replicate data by writing directly into all the available replicas. Though we strongly discourage this model as it will make your application accountable for data consistency across replication. This strategy is not scalable technically and operationally super complicated/complex to operate. we have copied below the config file:

./config/clickhouse_config.xml is the default config file in docker, we have copied the same in this blog too.

## copied below metrika.xml

Each server has its own macros.xml, We have copied below same:

Note: Please confirm your macros setting are in sync. with remote server settings in metrika.xml

Start the server:

Please connect with ClickHouse Server to confirm cluster settings are operational:

When you see this, It means the cluster setting is successful

Replica Table

we have now successfully configured a ClickHouse cluster and replica settings. For clickhouse, we have to create a ReplicatedMergeTree Table as a local table in each server:

Create Distributed Table conn to the local table:

Test the setup with INSERT script

data generation and load:

Count records on the Distributed table:

Count records on the local table:

☛ MinervaDB is trusted by top companies worldwide

Customer Logo