Introduction
Every cluster in our CloudNativePG series so far has been single-primary, multi-replica — one instance takes writes, the rest stay read-only, and a failover promotes exactly one replica. That covers most workloads.
But some systems need real active-active: multiple nodes accepting writes at the same time, with automatic conflict resolution when two writes hit the same row. That’s what EDB Postgres Distributed (PGD) is for — a genuinely different product from CloudNativePG, built on PostgreSQL’s BDR (Bi-Directional Replication) extension, managed on Kubernetes by its own operator: EDB Postgres® AI for CloudNativePG™ Global Cluster, commonly called PGD4K.
This post is not a theoretical walkthrough. Every command below, every error, and every result came from an actual install on a local Kind cluster running inside WSL Ubuntu on Windows — including the mistakes, because they’re genuinely useful for anyone following along.
What we actually did:
- Installed PGD4K v2.0.1 with PGD 6.4.0 on a Kind cluster in WSL
- Deployed a real
PGDGroup— 2 data nodes + 1 witness node - Hit and fixed a real validation error (missing TLS connectivity block)
- Verified multi-master writes in both directions between two nodes
- Discovered PGD’s global sequence partitioning in our own data
- Ran a real conflict test and confirmed how convergence actually looks
Prerequisites
- [ ] Kind cluster running in WSL Ubuntu on Windows (see our earlier post on setting this up)
- [ ]
kubectlworking from your WSL terminal - [ ] An EDB subscription token (free — via EDB Repos 2.0)
- [ ] At least 8GB RAM available to Docker Desktop
Important: PGD is a separate product from CloudNativePG. CloudNativePG (
postgresql.cnpg.io) manages single-primary HA clusters. PGD4K (pgd.k8s.enterprisedb.io) manages multi-master distributed clusters — and internally, each PGD node is a single-instance CloudNativePGCluster. You do not create aClusteryourself first; thePGDGroupresource creates everything underneath it automatically.
Lab Environment (What We Actually Ran)
| Component | Version We Used |
|---|---|
| Kubernetes | Kind (WSL Ubuntu) |
| EDB CloudNativePG Global Cluster (PGD4K) operator | v2.0.1 |
| PGD extension | 6.4.0 |
| PostgreSQL | 17.10 |
| Operand image | docker.enterprisedb.com/k8s/postgresql-pgd:17.10-pgd640-expanded-ubi9 |
| cert-manager | v1.16.2 |
A note on version choice
At the time of writing, PGD 6.5.0 exists as a standalone product (installed via TPA on VMs), but the Kubernetes operator’s officially published ImageCatalogs and tested support matrix top out at PGD 6.4.0. We confirmed this directly against EDB’s own supported-versions documentation before installing — there was no pgd65 catalog published for Kubernetes yet. So “latest available on Kubernetes” and “latest available overall” are two different answers with PGD, and we deliberately installed the Kubernetes-supported one.
Part 1: Installation
Step 1.1: Install cert-manager
PGD4K requires cert-manager for the mTLS mesh between nodes — non-negotiable.
kubectl apply -f \
https://github.com/cert-manager/cert-manager/releases/download/v1.16.2/cert-manager.yaml
kubectl wait --for=condition=Available --timeout=120s \
deployment --all -n cert-manager
Step 1.2: Namespace and EDB pull secret
kubectl create namespace pgd-operator-system
kubectl create secret -n pgd-operator-system docker-registry edb-pull-secret \
--docker-server=docker.enterprisedb.com \
--docker-username=k8s \
--docker-password="$EDB_SUBSCRIPTION_TOKEN"
Step 1.3: Install the operator
This single manifest installs both the PGD4K operator and its required CloudNativePG (PG4K) operator dependency, in the same namespace:
kubectl apply --server-side --force-conflicts -f \
https://get.enterprisedb.io/pg4k-pgd/pg4k-pgd-2.0.1.yaml
Verify both operators came up:
kubectl get deploy -n pgd-operator-system
NAME READY UP-TO-DATE AVAILABLE
pgd-operator-controller-manager 1/1 1 1
postgresql-operator-controller-manager 1/1 1 1
Step 1.4: Create a self-signed certificate issuer
kubectl apply -f \
https://raw.githubusercontent.com/EnterpriseDB/edb-postgres-for-kubernetes-charts/main/hack/samples/issuer-selfsigned.yaml
kubectl get issuer
NAME READY AGE
client-ca-issuer True 5m
server-ca-issuer True 5m
Step 1.5: Install the PGD 6.4 ImageCatalog
kubectl create -f \
https://get.enterprisedb.io/pgd-k8s-image-catalogs/postgresql-k8s-pgd6.4-expanded-ubi9.yaml
kubectl get imagecatalog
Part 2: Deploying a PGD Group
Step 2.1: The manifest — including a fix we had to make
Our first attempt at a minimal manifest failed validation. Here’s the actual error we hit:
The PGDGroup "region-a" is invalid:
* spec.connectivity.tls.mode: Unsupported value: "": supported values: "verify-ca", "verify-full", "require"
* spec.connectivity.tls.serverCert.certManager.spec: Invalid value: "null": spec.connectivity.tls.serverCert.certManager.spec in body must be of type object: "null"
Cause: connectivity.tls is a required block, not optional — we’d left it out of a “minimal” first draft. Here’s the manifest that actually worked:
apiVersion: pgd.k8s.enterprisedb.io/v1beta1
kind: PGDGroup
metadata:
name: region-a
spec:
instances: 2
witnessInstances: 1
imageCatalogRef:
apiGroup: pgd.k8s.enterprisedb.io
kind: ImageCatalog
name: postgresql-k8s-pgd64-expanded-ubi9
major: 17
pgd:
parentGroup:
name: world
create: true
discovery:
- host: region-a-group
cnp:
storage:
size: 1Gi
witness:
storage:
size: 1Gi
connectivity:
tls:
mode: verify-ca
clientCert:
caCertSecret: client-ca-key-pair
certManager:
spec:
issuerRef:
name: client-ca-issuer
kind: Issuer
group: cert-manager.io
serverCert:
caCertSecret: server-ca-key-pair
certManager:
spec:
issuerRef:
name: server-ca-issuer
kind: Issuer
group: cert-manager.io
kubectl apply -f region-a.yaml
pgdgroup.pgd.k8s.enterprisedb.io/region-a created
Lesson learned: don’t strip the
connectivity.tlsblock to “simplify” a first PGD manifest — it’s required, and theclient-ca-issuer/server-ca-issuernames must match exactly what your cert-managerIssuerresources are actually called.
Step 2.2: What actually got created
kubectl get pods
NAME READY STATUS RESTARTS AGE
region-a-1-1 1/1 Running 0 6m
region-a-2-1 1/1 Running 0 3m
region-a-3-1 1/1 Running 0 5m
Three pods, sequentially numbered — not the -w- suffix naming we initially expected for the witness. The role only shows up in PGD’s own metadata, not the pod name:
kubectl get pgdgroup region-a
NAME DATA INSTANCES WITNESS INSTANCES PHASE AGE
region-a 2 1 PGDGroup - Healthy 8m21s
kubectl describe pgdgroup region-a
Nodes:
node_group_name: region-a
node_kind_name: data
node_name: region-a-1
peer_state_name: ACTIVE
node_group_name: region-a
node_kind_name: data
node_name: region-a-2
peer_state_name: ACTIVE
node_group_name: region-a
node_kind_name: witness
node_name: region-a-3
peer_state_name: ACTIVE
Image:
Pgd: docker.enterprisedb.com/k8s/postgresql-pgd:17.10-pgd640-expanded-ubi9
Phase: PGDGroup - Healthy
Confirmed: region-a-1 and region-a-2 are data nodes, region-a-3 is the witness, all ACTIVE, running PGD 6.4.0 on PostgreSQL 17.10 exactly as targeted.
Important architectural note: we did not create a CloudNativePG
Clusterresource first. ThePGDGroupresource creates everything underneath it — each node ends up as its own independent single-instanceCluster, not a multi-instance HA cluster with its own primary/replica pair. That’s a fundamentally different replication topology from the rest of this series, and it’s why you go straight toPGDGroup, never toCluster, when working with PGD.
Part 3: Real Multi-Master Testing
This is the part that actually matters. A “Healthy” status tells you the plumbing is connected — it doesn’t tell you multi-master writes actually work. We tested it directly.
Test 1: Write on Node 1, Read on Node 2
kubectl exec -ti region-a-1-1 -- psql -d app
CREATE TABLE IF NOT EXISTS test_writes (
id serial PRIMARY KEY,
written_from text,
written_at timestamptz DEFAULT now()
);
INSERT INTO test_writes (written_from) VALUES ('region-a-1');
SELECT * FROM test_writes;
CREATE TABLE
INSERT 0 1
id | written_from | written_at
----+--------------+-------------------------------
2 | region-a-1 | 2026-08-25 17:19:45.062536+00
Read it back from the other data node:
kubectl exec -ti region-a-2-1 -- psql -d app -c "SELECT * FROM test_writes;"
id | written_from | written_at
----+--------------+-------------------------------
2 | region-a-1 | 2026-08-25 17:19:45.062536+00
(1 row)
Confirmed: write on node 1 replicated to node 2.
Test 2: Write on Node 2, Read on Node 1 (Completing the Loop)
One-directional replication isn’t proof of multi-master — CloudNativePG’s own replicas do that too. The real test is writing from the other node:
kubectl exec -ti region-a-2-1 -- psql -d app -c \
"INSERT INTO test_writes (written_from) VALUES ('region-a-2');"
INSERT 0 1
kubectl exec -ti region-a-1-1 -- psql -d app -c \
"SELECT * FROM test_writes ORDER BY id;"
id | written_from | written_at
---------+--------------+-------------------------------
2 | region-a-1 | 2026-08-25 17:19:45.062536+00
2000002 | region-a-2 | 2026-08-25 17:21:17.094308+00
Both nodes wrote. Both writes replicated everywhere. This is genuine bidirectional multi-master.
The id = 2000002 Detail — Not a Bug, the Whole Point
Look closely at those IDs: 2 from node 1, 2000002 from node 2. This isn’t random — it’s global sequence partitioning, one of PGD’s core mechanisms. Each data node owns its own numeric range for serial/sequence columns:
region-a-1's range: 1, 2, 3, 4 ...
region-a-2's range: 2,000,001 2,000,002 2,000,003 ...
This is exactly why two nodes can generate primary keys simultaneously and independently, with zero coordination overhead, and zero collision risk. If you tried this with a plain PostgreSQL serial column replicated via ordinary streaming replication and two writers, you’d get a primary key collision. PGD is architected specifically to avoid that.
Confirming the Write Leader Doesn’t Mean “Only Writer”
kubectl exec -ti region-a-2-1 -- bash -c \
'export PGD_CLI_DSN="dbname=app"; pgd group region-a show'
Write Leader | region-a-1
region-a-1 is the write leader — yet region-a-2 wrote successfully in Test 2 above. This confirms: in PGD, the “write leader” concept is about default connection routing (which node the built-in Connection Manager sends client traffic to by default), not an enforced single-writer restriction like CloudNativePG’s primary/replica model. Both nodes are genuinely active-active.
Test 3: Conflict Resolution — What We Learned by Getting It Wrong First
This is worth documenting honestly, because our first attempt did not actually test a conflict — and realizing why is more instructive than a clean success would have been.
Attempt 1 (flawed): We ran an UPDATE on region-a-1, checked both nodes, then ran an UPDATE on region-a-2 several commands later:
kubectl exec -ti region-a-1-1 -- psql -d app -c \
"UPDATE conflict_test SET value = 'from-region-a-1' WHERE id = 1;"
UPDATE 1
Both nodes agreed on from-region-a-1 almost immediately:
id | value
----+-----------------
1 | from-region-a-1
Then, well after that had replicated, we ran the second update:
kubectl exec -ti region-a-2-1 -- psql -d app -c \
"UPDATE conflict_test SET value = 'from-region-a-2' WHERE id = 1;"
Checking bdr.conflict_history_summary afterward showed 0 rows — because no conflict actually happened. These were two sequential updates minutes apart, not a race. Sequential writes to the same row from different nodes aren’t a “conflict” in PGD’s sense — they’re just normal replication, applied in order. That’s exactly what the empty conflict log correctly reported.
Attempt 2 (correct method): To actually create a race, both writes need to be in flight simultaneously. We used shell backgrounding to fire both at once:
kubectl exec -ti region-a-1-1 -- psql -d app -c "UPDATE conflict_test SET value = 'from-region-a-1-RACE' WHERE id = 1;" &
kubectl exec -ti region-a-2-1 -- psql -d app -c "UPDATE conflict_test SET value = 'from-region-a-2-RACE' WHERE id = 1;" &
wait
Then checked convergence on both nodes immediately:
kubectl exec -ti region-a-1-1 -- psql -d app -c "SELECT * FROM conflict_test;"
kubectl exec -ti region-a-2-1 -- psql -d app -c "SELECT * FROM conflict_test;"
What to look for: the two outputs must show the identical value — it genuinely doesn’t matter which of from-region-a-1-RACE or from-region-a-2-RACE wins. What matters is that every node agrees on the same final state. If nodes disagree after this, that’s data divergence — the exact failure mode conflict resolution exists to prevent.
To confirm PGD actually detected and logged the race (rather than the writes simply landing far enough apart to dodge it again):
kubectl exec -ti region-a-1-1 -- psql -d app -c \
"SELECT * FROM bdr.conflict_history_summary ORDER BY local_time DESC LIMIT 5;"
A populated row here — showing conflict_type and conflict_resolution columns — is the definitive proof that PGD detected genuinely concurrent writes and resolved them deterministically.
The real lesson from this test: “no conflict logged” can mean two very different things — either your writes genuinely didn’t race (our first attempt), or your conflict detection isn’t working (a real problem). Don’t assume a clean conflict log means resolution is working; make sure you’ve actually created a race first, the way we did with
&/waitbackgrounding.
What We Verified vs. What We Didn’t Get To
| Test | Status |
|---|---|
| Installation — operator, cert-manager, ImageCatalog | ✅ Verified, including a real validation error and fix |
PGDGroup deployment — 2 data nodes + 1 witness | ✅ Verified — confirmed PGDGroup - Healthy |
| Multi-master write: node 1 → node 2 | ✅ Verified |
| Multi-master write: node 2 → node 1 | ✅ Verified |
| Global sequence partitioning | ✅ Observed directly in our own data (id 2 vs id 2000002) |
| Write leader ≠ only writer | ✅ Confirmed |
| Conflict resolution (sequential, non-racing writes) | ⚠️ First attempt — correctly showed no conflict, because there wasn’t one |
| Conflict resolution (true concurrent race) | 🔄 In progress at time of writing — see Part 3, Attempt 2 |
| Node failure / write-leader re-election | ⬜ Not yet tested |
| Witness quorum behavior under node loss | ⬜ Not yet tested |
| PGDGroup self-healing after full node + PVC deletion | ⬜ Not yet tested |
We’re publishing this as-is because the installation and bidirectional multi-master proof are solid, fully-verified results on their own — and the conflict-resolution section is genuinely more useful to readers because it shows the wrong way to test it before the right way, rather than only presenting a clean success.
Common Errors We Actually Hit
Error 1: Missing connectivity.tls block
Symptom:
* spec.connectivity.tls.mode: Unsupported value: "": supported values: "verify-ca", "verify-full", "require"
* spec.connectivity.tls.serverCert.certManager.spec: Invalid value: "null"
Cause: connectivity.tls is required in every PGDGroup, referencing your cert-manager issuers by name.
Fix: Include the full connectivity.tls block shown in Part 2, Step 2.1, with clientCert/serverCert pointing at your actual Issuer names (verify with kubectl get issuer first).
Error 2: kubectl: command not found inside a pod shell
Symptom:
[postgres@region-a-1-1 /]$ kubectl exec -ti region-a-2-1 -- psql ...
bash: kubectl: command not found
Cause: kubectl only exists on your WSL host machine — it is not installed inside the PostgreSQL container itself.
Fix: Exit the pod shell (exit) and run kubectl exec commands from your host terminal, not from within an already-open pod session.
Error 3: Assuming sequential writes prove or disprove conflict handling
Symptom: bdr.conflict_history_summary shows 0 rows after two UPDATEs to the same row, leading to the wrong conclusion that conflict resolution “isn’t working.”
Cause: The two writes weren’t actually concurrent — one fully replicated before the second began, so there was nothing to resolve.
Fix: Use shell backgrounding (command & twice, followed by wait) to force genuine concurrency before judging whether conflict resolution is functioning.
Key Takeaways
✅ PGD4K’s officially supported Kubernetes version currently tops out at PGD 6.4.0, even though PGD 6.5.0 exists as a standalone product — check the operator’s own supported-versions page before assuming “latest” means the same thing in both places.
✅ You deploy a PGDGroup directly — never a Cluster first. The operator creates single-instance Cluster resources underneath automatically; that’s an implementation detail, not a step you perform.
✅ connectivity.tls is a required field, not optional — every PGDGroup manifest needs it wired to real cert-manager Issuer names.
✅ We proved genuine bidirectional multi-master writes directly — not from documentation, but from our own two-node test. The unexpected id = 2000002 sequence value was the clearest real-world evidence of PGD’s global sequence partitioning we could have asked for.
✅ Testing conflict resolution correctly requires forcing real concurrency (shell backgrounding, not sequential commands) — otherwise an empty conflict log can misleadingly look like success when no real test occurred.
Test Your Knowledge
Ready to test what you’ve learned? Take the free quiz:
👉 PostgreSQL High Availability Quiz → gradeupnow.in/postgres-replication-quiz/
20 questions · Instant feedback · Detailed explanations · Free
What’s Next
This is Post 1 of a focused mini-series on Postgres Distributed (PGD) on Kubernetes:
| # | Post | Status |
|---|---|---|
| 1 | PGD Real Setup and Multi-Master Testing | 📍 You are here |
| 2 | PGD Node Failure, Witness Quorum & Self-Healing — Tested | ⬜ Coming next |
| 3 | PGD Backup, Recovery & Cross-Region DR | ⬜ Coming soon |
Post 2 picks up exactly where our test checklist left off — killing a data node mid-write, watching write-leader re-election happen live, and confirming the witness node’s role when a data node is down.
References
- EDB Postgres® AI for CloudNativePG™ Global Cluster — Official Documentation
- EDB CloudNativePG Global Cluster — Supported Versions
- EDB Postgres Distributed (PGD) 6.4 Release Announcement
- EDB Postgres Distributed — Conflict Management
- cert-manager Documentation
This post documents a real installation and test session, including the mistakes — because they’re often more instructive than a clean walkthrough. Questions or spotted something we got wrong? Drop it in the comments.