How to partition a large table in PostgreSQL without a long-running lock?

When partitioning a large table in PostgreSQL, the table needs to be locked to prevent any updates, inserts, or deletes while the partitioning operation is in progress. This can result in a long-running lock, which can cause significant downtime for your application.

To avoid long-running locks when partitioning a large table in PostgreSQL, you can use a technique called “table swapping”. This involves creating an empty partitioned table with the same structure as the original table, and then swapping the data from the original table into the new partitioned table.

Here are the general steps to perform table swapping for partitioning a large table in PostgreSQL:

  1. Create an empty partitioned table with the same structure as the original table, using the CREATE TABLE command with the PARTITION BY clause to define the partitioning scheme.
  2. Create the partitions for the new partitioned table, using the CREATE TABLE command with the INHERITS clause to inherit the structure of the partitioned table.
  3. Disable any indexes, triggers, or foreign keys on the original table to improve the performance of the data copy operation.
  4. Copy the data from the original table into the new partitioned table, using the INSERT INTO command with the SELECT clause to select the data from the original table.
  5. Re-enable any indexes, triggers, or foreign keys on the new partitioned table.
  6. Rename the original table to a backup name, and then rename the new partitioned table to the original table name.
  7. Drop the backup table.

By using table swapping, you can avoid long-running locks on the original table while partitioning it. The locking period is limited to the short time it takes to rename the tables, which should be just a few seconds.

Note that this technique requires careful planning and testing, and may not be suitable for all scenarios. It is important to back up your data before attempting any major database operations.

About Shiv Iyer 443 Articles
Open Source Database Systems Engineer with a deep understanding of Optimizer Internals, Performance Engineering, Scalability and Data SRE. Shiv currently is the Founder, Investor, Board Member and CEO of multiple Database Systems Infrastructure Operations companies in the Transaction Processing Computing and ColumnStores ecosystem. He is also a frequent speaker in open source software conferences globally.