Introduction

Every cluster you’ve built so far in this series has quietly been using asynchronous streaming replication — the safe, sensible default. Your primary writes, replicas catch up a few milliseconds later, and if the primary dies right at the wrong instant, you could lose the last few transactions that hadn’t replicated yet.

For most workloads, that’s a perfectly reasonable trade-off. But some workloads — financial transactions, order processing, anything where “we lost the last transaction” is unacceptable — need a stronger guarantee: zero data loss, even during a failover. That’s what synchronous replication buys you, at the cost of some write latency and, in strict mode, potential write pauses.

This post covers replication internals in depth — how CloudNativePG builds on PostgreSQL’s own mature replication technology, the difference between quorum-based and priority-based synchronous replication, the critical dataDurability trade-off, and replication slots. We close with rolling updates — how CloudNativePG upgrades your PostgreSQL version or applies config changes without taking your application down.

What you’ll learn:

Why CloudNativePG uses “application-level” replication instead of storage-level replication
Asynchronous vs. synchronous replication, and when to choose which
Quorum-based (ANY) vs. priority-based (FIRST) synchronous replication
The dataDurability: required vs preferred trade-off — data safety vs. self-healing
How replication slots prevent WAL loss across failovers
How rolling updates work, and the difference between restart and switchover strategies

Prerequisites

Before you start, make sure you have:

[ ] The Kind cluster pg from Post 3 running, with CloudNativePG operator installed
[ ] The cnpg kubectl plugin working
[ ] A 3-instance cluster deployed (we’ll use cluster-example from earlier posts)
[ ] Comfort with the -rw/-ro/-r service model from Post 1

Lab Environment

Component	Version / Details
Kubernetes	Kind v0.27 (from Post 3)
CloudNativePG Operator	v1.28.1
PostgreSQL	16.x / 17.x
kubectl	v1.32+
cnpg plugin	v1.25.0+

Part 1: Why “Application-Level” Replication

Back in Post 1, we touched on this briefly. Now let’s go deeper.

CloudNativePG deliberately builds high availability on top of PostgreSQL’s own native physical replication — not on storage-level replication (like a replicated EBS volume or DRBD). In Kubernetes terminology, this is called application-level replication.

Storage-level replication              Application-level replication (CloudNativePG)
──────────────────────────             ─────────────────────────────────────────────
Block device is mirrored               PostgreSQL streams its own WAL to replicas
Storage layer has no idea               PostgreSQL understands transaction
what a "transaction" is                 boundaries, consistency, and commit semantics
Crash recovery is storage's problem     PostgreSQL's own crash recovery applies
Extra latency from block-level sync     Purpose-built streaming protocol

This isn’t a new or experimental approach — PostgreSQL’s replication technology has been battle-tested and evolving for nearly two decades:

Version	Year	Feature
8.2	2006	Warm Standby with WAL shipping
9.0	2010	Hot Standby and physical streaming replication
9.1	2011	Synchronous replication (priority-based)
9.2	2012	Cascading replication
9.4	2014	Foundations of logical replication
10	2017	Logical publisher/subscriber and quorum-based synchronous replication

CloudNativePG doesn’t reinvent any of this — it just wires PostgreSQL’s own mature replication directly into the Kubernetes API, declaratively.

Part 2: How Streaming Replication Is Set Up Automatically

You’ve already seen this happen invisibly since Post 2. The relationship is simple:

replicas = instances - 1

Set instances: 3 in your Cluster spec, and CloudNativePG automatically creates 1 primary + 2 replicas, all streaming from the primary.

Step 2.1: Inspect the Auto-Created Replication User

Immediately after cluster initialization, the operator creates a dedicated replication user:

kubectl cnpg psql cluster-example

\du streaming_replica

Expected output:

                                   List of roles
    Role name     |            Attributes             | Member of
-------------------+------------------------------------+-----------
 streaming_replica | Replication                        | {}

Notice: no superuser, no createrole, no createdb — just the REPLICATION privilege. This is the principle of least privilege in action, straight from the 4C Security Model we covered in Post 6.

Step 2.2: Verify TLS Client Certificate Authentication

Replication traffic is encrypted and authenticated with TLS client certificates by default — not passwords. Check the generated pg_hba.conf:

kubectl exec -it cluster-example-1 -- \
  cat /var/lib/postgresql/data/pgdata/pg_hba.conf | grep streaming_replica

Expected output:

# Require client certificate authentication for the streaming_replica user
hostssl postgres streaming_replica all cert map=cnp_streaming_replica
hostssl replication streaming_replica all cert map=cnp_streaming_replica

hostssl + cert means: TLS is mandatory, and the client certificate itself proves identity — no password ever crosses the wire for replication traffic.

root@E-5CG1467JZY:/mnt/c/Users/emaideb# kubectl exec -it cluster-example-1 — \
cat /var/lib/postgresql/data/pgdata/pg_hba.conf | grep streaming_replica
Defaulted container “postgres” out of: postgres, bootstrap-controller (init)

Require client certificate authentication for the streaming_replica user

hostssl postgres streaming_replica all cert
hostssl replication streaming_replica all cert
root@E-5CG1467JZY:/mnt/c/Users/emaideb#

Step 2.3: Check Live Replication Status

kubectl cnpg psql cluster-example

SELECT application_name, state, sync_state, replay_lag
FROM pg_stat_replication;

Expected output:

 application_name |   state   | sync_state | replay_lag
-------------------+-----------+------------+-------------
 cluster-example-2 | streaming | async      | 00:00:00
 cluster-example-3 | streaming | async      | 00:00:00

Notice sync_state = async — this confirms we’re currently on asynchronous replication, the default. That’s about to change.

Part 3: Enabling Synchronous Replication — Quorum-Based

Quorum-based synchronous replication is the recommended and most common setup. A transaction commit waits until its WAL record has been replicated to at least N of the available standbys — it doesn’t matter which ones.

Step 3.1: Enable Quorum Synchronous Replication

apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: cluster-example
spec:
  instances: 3
  postgresql:
    synchronous:
      method: any
      number: 1
  storage:
    size: 1Gi

Apply it:

kubectl apply -f cluster-example.yaml

Important: You cannot set synchronous_standby_names directly — CloudNativePG populates it automatically from your synchronous stanza. This is a fixed/reserved parameter, exactly as covered in Post 4.

root@E-5CG1467JZY:/mnt/c/Users/emaideb# cat cluster-example.yaml
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: cluster-example
spec:
instances: 3
storage:
size: 1Gi
postgresql:
parameters:
shared_buffers: “256MB”
max_connections: “200”
work_mem: “8MB”
maintenance_work_mem: “128MB”
effective_cache_size: “1GB”
log_min_duration_statement: “1000” # log queries slower than 1s
synchronous:
method: any
number: 1

Step 3.2: Verify the Generated synchronous_standby_names

kubectl cnpg psql cluster-example

SHOW synchronous_standby_names;

Expected output:

       synchronous_standby_names
------------------------------------------
 ANY 1 (cluster-example-2, cluster-example-3, cluster-example-1)

This reads as: “Wait for acknowledgment from ANY 1 of these instances before considering the transaction committed.” Note that the primary’s own name (cluster-example-1) appears in the list too — this is intentional, allowing the list to remain valid across failovers without needing to be regenerated.

root@E-5CG1467JZY:/mnt/c/Users/emaideb# kubectl cnpg psql cluster-example
psql (17.2 (Debian 17.2-1.pgdg110+1))
Type “help” for help.

postgres=# SHOW synchronous_standby_names;

synchronous_standby_names

ANY 1 (“cluster-example-1″,”cluster-example-3″,”cluster-example-2”)
(1 row)

Step 3.3: Confirm sync_state Changed

SELECT application_name, state, sync_state
FROM pg_stat_replication;

Expected output:

 application_name |   state   | sync_state
-------------------+-----------+------------
 cluster-example-2 | streaming | sync
 cluster-example-3 | streaming | potential

One replica shows sync (actively part of the quorum), the other shows potential (ready to take over as sync replica if the current one drops out). This is PostgreSQL automatically managing failover-readiness within your quorum.

Part 4: Priority-Based Synchronous Replication

Priority-based replication (method: first) is different: standbys are ranked in order, and the highest-priority N standbys are the synchronous ones. If a sync standby disconnects, the next-highest-priority standby is promoted into the sync role automatically.

spec:
  postgresql:
    synchronous:
      method: first
      number: 2

When to use first over any: Priority-based replication is most valuable when extending synchronous replication beyond the current cluster — for example, requiring a specific external standby (in another Kubernetes cluster or on a VM) to always be part of the synchronous set. We’ll see this with standbyNamesPre/standbyNamesPost next.

Part 5: Extending Synchronous Replication Beyond the Cluster

Sometimes you need a specific external replica — perhaps a DR standby in another region — to always participate in your synchronous quorum, regardless of what CloudNativePG does with local pods.

Step 5.1: The Three Control Parameters

Parameter	What it does
`maxStandbyNamesFromCluster`	Caps how many local pod names the operator auto-includes
`standbyNamesPre`	Names prepended before the local pod list
`standbyNamesPost`	Names appended after the local pod list

Step 5.2: Example — Adding an External Standby

spec:
  postgresql:
    synchronous:
      method: any
      number: 1
      maxStandbyNamesFromCluster: 1
      standbyNamesPre:
        - angus

With a 3-instance cluster-example, this produces:

ANY 1 (angus, cluster-example-2)

Only one local pod name got included (maxStandbyNamesFromCluster: 1), and angus (an external standby you manage yourself) was prepended.

⚠️ You own the uptime guarantee here. If angus isn’t actually healthy and replicating, you can jeopardize your cluster’s write availability. CloudNativePG has no visibility into or control over external standby names you add manually.

Part 6: Data Durability — required vs. preferred

This is the single most important trade-off decision in synchronous replication, and it’s easy to get wrong if you don’t understand the implications.

Step 6.1: required — Zero Data Loss, Possible Write Pauses

spec:
  postgresql:
    synchronous:
      method: any
      number: 1
      dataDurability: required   # this is the default

With required, PostgreSQL will not acknowledge a commit until the required number of synchronous standbys confirm. If not enough standbys are healthy, writes pause entirely.

Walkthrough — 3-instance cluster, ANY 1 required:

1. Healthy:        ANY 1 ("foo-2","foo-3","foo-1")     → writes flow normally
2. foo-2 down:      ANY 1 ("foo-3","foo-2","foo-1")     → still fine, foo-3 covers it
3. foo-3 ALSO down: ANY 1 ("foo-2","foo-3","foo-1")     → NO healthy standbys left
                                                            → WRITES PAUSE
4. Standbys return:  back to normal automatically

This is the right choice when data loss is simply unacceptable — financial ledgers, order systems, anything with legal or compliance implications around “we lost a transaction.”

Step 6.2: preferred — Self-Healing, Possible Data Loss

spec:
  postgresql:
    synchronous:
      method: any
      number: 2
      dataDurability: preferred

⚠️ preferred can only be used when standbyNamesPre and standbyNamesPost are unset.

With preferred, the required standby count shrinks automatically as standbys become unavailable, keeping writes flowing:

Walkthrough — 5-instance cluster "bar", ANY 2 preferred:

1. Healthy:              ANY 2 ("bar-2","bar-3","bar-4","bar-5")
2. bar-2, bar-3 down:    ANY 2 ("bar-4","bar-5")             → still requires 2, both present
3. bar-4 ALSO down:      ANY 1 ("bar-5")                     → requirement drops to 1
4. bar-5 ALSO down:      synchronous_standby_names EMPTY     → sync replication OFF
                                                                 writes continue, at RISK
5. Replicas return:       back to normal automatically

⚠️ This mode can lose data if all standbys become unavailable simultaneously — the primary keeps accepting writes with no synchronous guarantee at all at that point. Choose this when availability matters more than the (rare) risk of losing a few transactions.

Step 6.3: Decision Table

Your priority	Use
Never lose a committed transaction, even if it means pausing writes	`dataDurability: required`
Keep accepting writes no matter what, accept rare data loss risk	`dataDurability: preferred`
Extend sync replication to an external, self-managed standby	`standbyNamesPre`/`standbyNamesPost` (requires `required`)

Part 7: Replication Slots — Preventing WAL Loss Across Failovers

Step 7.1: The Problem Replication Slots Solve

Without replication slots, if a replica temporarily disconnects (network blip, pod restart), the primary might delete the WAL segments that replica still needs — forcing a full re-clone. Replication slots (a native PostgreSQL 9.4+ feature) fix this: the primary retains WAL until every attached slot confirms it’s no longer needed.

But there’s a catch: replication slots exist only on the instance that created them — they don’t replicate to standbys. So after a failover, the new primary doesn’t have the old primary’s replication slots, breaking any client that depended on them.

Step 7.2: How CloudNativePG Solves This

CloudNativePG provides a turnkey solution: it synchronizes replication slot state from the primary to every standby continuously, so that whichever instance is promoted after a failover already has the correct slots in place.

kubectl cnpg psql cluster-example

SELECT slot_name, plugin, slot_type, active, wal_status
FROM pg_replication_slots;

Expected output:

       slot_name        | plugin | slot_type | active | wal_status
-------------------------+--------+-----------+--------+------------
 _cnpg_cluster_example_2 |        | physical  | t      | reserved
 _cnpg_cluster_example_3 |        | physical  | t      | reserved

These _cnpg_-prefixed slots are the operator’s HA replication slots — one per replica, automatically kept in sync across the whole cluster.

root@E-5CG1467JZY:/mnt/c/Users/emaideb# kubectl cnpg psql cluster-example
psql (17.2 (Debian 17.2-1.pgdg110+1))
Type “help” for help.

Step 7.3: Configuring Replication Slot Behavior

spec:
  postgresql:
    parameters:
      max_slot_wal_keep_size: "50GB"
  replicationSlots:
    updateInterval: 600
    highAvailability:
      enabled: true   # default
      slotPrefix: "_cnpg_"

Setting	Default	What it does
`highAvailability.enabled`	`true`	Turns HA replication slots on/off
`highAvailability.slotPrefix`	`_cnpg_`	Prefix for auto-created HA slots
`updateInterval`	30s	How often slot state syncs between primary and standbys
`max_slot_wal_keep_size`	unlimited	Caps how much WAL a slot can force the primary to retain

⚠️ Watch max_slot_wal_keep_size. If a replica goes offline for an extended period and slots are unbounded, the primary’s WAL storage can grow without limit trying to retain everything that replica might eventually need. Setting a cap trades some replica resilience for protecting primary disk space.

Part 8: Anti-Affinity for Synchronous Replica Placement

Recall from Post 1: for true multi-AZ resilience, you don’t just want replicas — you want them on different availability zones than the primary. CloudNativePG can enforce this specifically for synchronous replica selection:

spec:
  instances: 3
  postgresql:
    syncReplicaElectionConstraint:
      enabled: true
      nodeLabelsAntiAffinity:
        - topology.kubernetes.io/zone

This ensures that, when choosing which replicas participate in the synchronous quorum, the operator prefers replicas running in a different AZ from wherever the current primary happens to be — directly improving your RPO in the event of a full AZ outage.

⚠️ This option only applies to the deprecated minSyncReplicas/maxSyncReplicas implementation, not the newer synchronous stanza covered in Parts 3–6. If you’re on the current API, AZ-aware placement should instead be handled through general pod anti-affinity rules (covered in a future post on Scheduling).

Part 9: Rolling Updates — Zero-Downtime Upgrades

Now let’s cover what happens when you need to change the PostgreSQL version, adjust resource limits, or apply a configuration change that requires a restart.

Step 9.1: What Triggers a Rolling Update

Trigger	Example
Changing `imageName`	Upgrading PostgreSQL minor version
Changing extension images	`.spec.postgresql.extensions` update
Image catalog update	New image published for your major version
Config change requiring restart	Certain `postgresql.conf` parameters
Changing `.spec.resources`	CPU/memory requests or limits
Operator upgrade	New instance manager version rolled out

Step 9.2: How the Rollout Order Works

Rolling update sequence for a 3-instance cluster:

1. cluster-example-3 (highest serial, replica)  ← upgraded first
2. cluster-example-2 (replica)                  ← upgraded second
3. cluster-example-1 (PRIMARY)                  ← upgraded LAST

Replicas are upgraded one at a time, starting from the highest serial number. The primary is always upgraded last — and critically, the cluster’s identity is preserved throughout. Pods are deleted and recreated with the same PVCs, not re-cloned from scratch.

During the process, service endpoints (-rw, -ro, -r) continuously update to route around whichever pod is currently being replaced — your application experiences this as, at most, a brief connection blip on the primary.

Step 9.3: Unsupervised Updates — restart vs. switchover

By default, primaryUpdateStrategy: unsupervised — fully automated, no human step required.

spec:
  primaryUpdateStrategy: unsupervised
  primaryUpdateMethod: restart   # or: switchover

Method	What happens to the primary	Trade-off
`restart` (default)	Primary pod is restarted in place with the new image	May need to pull the new image after shutdown — slower if image isn’t cached
`switchover`	A replica (already on the new image) is promoted; old primary is shut down	Faster cutover since the new primary is already running the target image, but adds a short leadership change

kubectl apply -f cluster-example.yaml   # after bumping imageName
kubectl cnpg status cluster-example      # watch the rollout happen

⚠️ Important constraint: with switchover, you cannot change the image and PostgreSQL configuration parameters in the same apply — the operator rejects it with a validation error. Update the image first, wait for rollout completion, then update configuration (or vice versa). This exists because config changes tied to a new image version could break PostgreSQL if applied to pods still running the old image mid-switchover.

Step 9.4: Supervised Updates — Manual Control

For clusters where you want a human decision point before the primary is touched:

spec:
  primaryUpdateStrategy: supervised

With this setting, the rollout pauses immediately after all replicas are upgraded — the primary is left untouched until you manually trigger the final step:

# Option A: manual switchover — promotes a replica already on the new image
kubectl cnpg promote cluster-example cluster-example-2

# Option B: manual restart — restarts primary in place (only for non-image changes)
kubectl cnpg restart cluster-example cluster-example-1

When to use supervised: production clusters where you want to control the exact maintenance window for the primary’s brief downtime/failover — for example, coordinating with an application deploy or a low-traffic window.

Common Errors and Fixes

Error 1: Writes hang after enabling required synchronous replication

Symptom: Application writes time out or hang indefinitely after applying dataDurability: required.

Cause: Not enough healthy synchronous standbys are currently available to satisfy the quorum.

Fix:

# Check replica health first
kubectl cnpg status cluster-example

# If replicas are down, fix that first (check pod events/logs)
kubectl describe pod cluster-example-2

# If you need writes to continue during a partial outage, switch to preferred
# (understanding the data-loss trade-off from Part 6)

Error 2: Validation error changing image and config together

Symptom:

error: admission webhook denied the request: cannot change image and
PostgreSQL configuration in the same update when primaryUpdateMethod is switchover

Cause: Exactly the constraint from Step 9.3 — the operator blocks this combination intentionally.

Fix: Split into two sequential applies:

# Step 1: update only the image
kubectl apply -f cluster-example-new-image.yaml
kubectl cnpg status cluster-example   # wait for "Cluster in healthy state"

# Step 2: THEN update configuration
kubectl apply -f cluster-example-new-config.yaml

Error 3: Supervised rollout appears stuck

Symptom: kubectl cnpg status shows replicas upgraded, but the primary never changes — no error, just… nothing.

Cause: This is expected behavior for primaryUpdateStrategy: supervised — it’s waiting for you.

Fix:

# Confirm this is indeed the cause
kubectl get cluster cluster-example -o jsonpath='{.spec.primaryUpdateStrategy}'

# Trigger the final step manually
kubectl cnpg promote cluster-example cluster-example-2

Key Takeaways

✅ CloudNativePG uses PostgreSQL’s own mature, native physical replication (“application-level replication”) rather than storage-level replication — this is a deliberate architectural choice, not a limitation.

✅ Quorum-based (method: any) synchronous replication is the recommended default when you need synchronous guarantees; priority-based (method: first) shines when extending replication to specific external standbys.

✅ dataDurability: required guarantees zero data loss but can pause writes if standbys aren’t available; dataDurability: preferred keeps writes flowing but can lose data if all standbys fail simultaneously. Know which trade-off your workload needs.

✅ Replication slots prevent WAL loss when replicas briefly disconnect — CloudNativePG automatically synchronizes slot state across the cluster so a promoted replica already has the right slots after failover.

✅ Rolling updates preserve cluster identity (same PVCs, no re-cloning) and always upgrade the primary last. Choose unsupervised for fully automated upgrades, supervised when you want a manual checkpoint before the primary is touched.

Test Your Knowledge

Ready to test what you’ve learned? Take the free quiz:

👉 PostgreSQL Replication Quiz → gradeupnow.in/postgres-replication-quiz/

20 questions · Instant feedback · Detailed explanations · Free

What’s Next

This post is part of the PostgreSQL on Kubernetes (CloudNativePG) series:

#	Post	Status
1	CloudNativePG Architecture on AWS EKS	✅ Published
2	Installing CloudNativePG on AWS EKS — Step by Step	✅ Published
3	PostgreSQL on Kind in WSL Ubuntu with Grafana Monitoring	✅ Published
4	PostgreSQL Configuration & Pod Tuning on Kubernetes	✅ Published
5	Bootstrap Methods — initdb, Recovery, pg_basebackup	✅ Published
6	Importing Databases & Security on CloudNativePG	✅ Published
7	Replication & Rolling Updates on Kubernetes	📍 You are here
8	Backup, Recovery & WAL Archiving on AWS S3	⬜ Coming next week

In Post 8, we finally build out the recovery bootstrap method we deferred back in Post 5 — configuring Barman Cloud backups to S3, WAL archiving, scheduled backups, and testing a full point-in-time recovery.

👉 [Next Post: Backup, Recovery & WAL Archiving on AWS S3 → coming next week]

References

Found this helpful? Share it with your DBA team! Questions? Drop them in the comments below.

Screenshot guide: 5 screenshot placeholders in this post. Most valuable for readers: the synchronous_standby_names confirmation (Step 3.2), the replication slots output (Step 7.2), and the supervised rollout paused mid-update (Step 9.4) — that last one clearly shows the human-checkpoint behavior in action.

Introduction

Prerequisites

Lab Environment

Part 1: Why “Application-Level” Replication

Part 2: How Streaming Replication Is Set Up Automatically

Step 2.1: Inspect the Auto-Created Replication User

Step 2.2: Verify TLS Client Certificate Authentication

Require client certificate authentication for the streaming_replica user

Step 2.3: Check Live Replication Status

Part 3: Enabling Synchronous Replication — Quorum-Based

Step 3.1: Enable Quorum Synchronous Replication

Step 3.2: Verify the Generated synchronous_standby_names

synchronous_standby_names

Step 3.3: Confirm sync_state Changed

Part 4: Priority-Based Synchronous Replication

Part 5: Extending Synchronous Replication Beyond the Cluster

Step 5.1: The Three Control Parameters

Step 5.2: Example — Adding an External Standby

Part 6: Data Durability — required vs. preferred

Step 6.1: required — Zero Data Loss, Possible Write Pauses

Step 6.2: preferred — Self-Healing, Possible Data Loss

Step 6.3: Decision Table

Part 7: Replication Slots — Preventing WAL Loss Across Failovers

Step 7.1: The Problem Replication Slots Solve

Step 7.2: How CloudNativePG Solves This

Step 7.3: Configuring Replication Slot Behavior

Part 8: Anti-Affinity for Synchronous Replica Placement

Part 9: Rolling Updates — Zero-Downtime Upgrades

Step 9.1: What Triggers a Rolling Update

Step 9.2: How the Rollout Order Works

Step 9.3: Unsupervised Updates — restart vs. switchover

Step 9.4: Supervised Updates — Manual Control

Common Errors and Fixes

Error 1: Writes hang after enabling required synchronous replication

Error 2: Validation error changing image and config together

Error 3: Supervised rollout appears stuck

Key Takeaways

Test Your Knowledge

What’s Next

References

Leave a Comment Cancel Reply