The Birth of Replication Slots
Managing WAL files to keep replication stable has always been a hard problem in PostgreSQL. If the Main Server reuses (overwrites) WAL files while the Standby Server is disconnected, replication can no longer be maintained, and you run into the following error.
If you fail to act before this happens, you have to rebuild the Standby Server to continue replication. Of course, to avoid this, operators would have set up a WAL file retention policy, for example by tuning the
max_wal_size parameter (max_keep_segments up to PostgreSQL 9.5).Replication Slots, introduced in PostgreSQL 9.4, keep the WAL files the Standby Server needs on the Main Server even when the connection to the Standby Server is lost. Once WAL files (or records) are delivered to the Standby Server through the Replication Slot, the WAL files are deleted or reused.
Advantages of Replication Slots
- With slot-based replication, the WAL files needed when the Standby Server reconnects are retained even if its connection is lost.
- Retention of the WAL files the Standby Server needs is managed automatically, so the operator does not have to worry about WAL retention settings.
Disadvantages of Replication Slots
- If the Standby Server is disconnected, WAL files are kept instead of being reused, which can fill up the Main Server's disk (pg_wal directory).
- Replication Slots are not dropped automatically, so if replication is no longer maintained, the operator must drop the Replication Slot for the reason above.
Replication Slots come in two types: Physical Replication Slots and Logical Replication Slots.
Physical Replication Slot
Physical Replication Slots are used with Streaming Replication. Changes made on the Main Server are written to WAL files. A Physical Replication Slot reads what is written in the WAL files to keep the Standby Server in sync.
Creating a Physical Replication Slot
Checking a Physical Replication Slot
You can view Replication Slot information in the pg_replication_slots catalog. In the example above, the
restart_lsn column shows that pslot will not remove WAL files before 0/A000060.Logical Replication Slot
Unlike Physical Replication Slots, Logical Replication Slots replicate using messages decoded through a mechanism called logical decoding. Because replication is based on logical changes, data replication is more flexible than with Physical Replication Slots (for example, specific schemas or tables).
Creating a Logical Replication Slot
Unlike Physical Replication Slots, Logical Replication Slots use the
pgoutput module for replication. The pgoutput module ships by default from PostgreSQL 10 and is used to output changes. When a Logical Replication Slot is created, the log position of the current transaction is returned (0/A000650), and logically decoded WAL records are sent to the Standby Server from this position.Checking a Logical Replication Slot
Logical Replication Slot Plugin Types
Type | Description |
pgoutput | The standard output plugin used in Logical Replication. It provides database changes in binary format. |
test_decoding | Provides database changes in a human-readable format; mainly used for testing and development. |
wal2json | Provides database changes in JSON format. (3rd-party plugin) |
Risks of Replication Slots
Replication Slots are designed to keep replication going even when a server fails, but they cannot know the state of the application using the slot. So a slot that was created may end up unused. In that case, WAL files that have not yet been applied must be kept, and they are not cleaned up by VACUUM. If this state continues, in the extreme case PostgreSQL may shut down to prevent XID wraparound, so drop any slot you no longer need.
pg_wal Size With and Without a Replication Slot
Create a test table and insert data until the WAL files add up to 1GB. In this test, inserting about 170,000 rows creates one WAL file (16MB), and we insert enough data to create 64 WAL files in total. (16MB * 64 = 1024MB)
The pg_wal directory now contains 64 WAL files, and their total size is 1009 MB.
If the Standby Server has received all of the WAL generated on the Main Server, WAL files in the Main Server's pg_wal directory are reused within the 1GB set by
max_wal_size.The default value of the
max_wal_size parameter is 1GB.Standby Server Shut Down
Suppose a large volume of transactions runs on the Main Server while the Standby Server is shut down. The size of the pg_wal directory then looks completely different depending on whether a Replication Slot exists.
Without a Replication Slot, the Main Server does not consider whether the Standby Server has received the WAL files, so it reuses (overwrites) existing WAL files to stay within the size set by
max_wal_size.With a Replication Slot, on the other hand, the Main Server checks whether the Standby Server has received the WAL files, and if it cannot receive them, pg_wal can grow beyond
max_wal_size. This is because the files are kept until the Standby Server receives them. Without any action, this can lead to problems such as a full disk on the Main Server or a PostgreSQL shutdown to prevent XID wraparound.Replication Slot Parameter
To use PostgreSQL Replication Slots, check the
max_replication_slots and wal_level parameters.- The
max_replication_slotsparameter sets the number of Replication Slots PostgreSQL can use. Set it to at least the number of Standby Servers.
- If you use pg_basebackup or a similar tool to copy the Main Server, set it to the number of Standby Servers + 1.
- The
wal_levelparameter controls how much information is written to WAL. Its default isreplica, which uses Physical Replication Slots. Set it tologicalto use Logical Replication Slots.
The default value of
max_replication_slots is 10 from PostgreSQL 10. Up to PostgreSQL 9.6, the default is 0.If max_replication_slots < 1, Replication Slots cannot be created.
Related Articles
