docs: add backup, WAL archive and recovery information (#291)

Reorganised the content, with a different structure,
added license information and instructions for spellchecking.

Closes #290

Signed-off-by: Gabriele Bartolini <gabriele.bartolini@enterprisedb.com>
Signed-off-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
Co-authored-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
This commit is contained in:
Gabriele Bartolini 2025-05-06 11:49:09 +02:00 committed by GitHub
parent 4be694bb1f
commit e26ad81fba
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
15 changed files with 499 additions and 96 deletions

View File

@ -3,7 +3,9 @@ Azurite
BarmanObjectStore
BarmanObjectStoreConfiguration
BarmanObjectStores
CRD
CloudNativePG
DataBackupConfiguration
DigitalOcean
Docusaurus
EKS
@ -24,17 +26,24 @@ ObjectStore
ObjectStoreSpec
ObjectStoreStatus
PITR
PoR
PostgreSQL
Postgres
README
RPO
RTO
RecoveryWindow
RetentionPolicy
SAS
SFO
SPDX
SPDX
ServerRecoveryWindow
TLS
TODO
Uncomment
WAL
WALBackupConfiguration
WALs
api
apiVersion
@ -46,6 +55,7 @@ backend
backends
barmanObjectName
barmancloud
bzip
cd
cloudnative
clusterrole
@ -64,6 +74,7 @@ externalClusters
gcs
gf
github
gzip
hostpath
http
https
@ -89,6 +100,7 @@ rc
recoverability
repos
retentionCheckInterval
retentionPolicy
rolebinding
sc
secretKeyRef

View File

@ -3,6 +3,13 @@
This website is built using [Docusaurus](https://docusaurus.io/), a modern
static website generator.
### Requirements
- Docker
- [Yarn](https://yarnpkg.com/)
- [Dagger](https://dagger.io/)
- [Task](https://taskfile.dev/)
### Installation
```shell
@ -35,6 +42,14 @@ $ yarn serve
By default, this will load your site at http://localhost:3000/.
### Spellchecking
From the top directory:
```shell
task spellcheck
```
### Versioning
Docusaurus allows versioning of the documentation to maintain separate sets of

43
web/docs/compression.md Normal file
View File

@ -0,0 +1,43 @@
---
sidebar_position: 80
---
# Compression
<!-- SPDX-License-Identifier: CC-BY-4.0 -->
By default, backups and WAL files are archived **uncompressed**. However, the
Barman Cloud Plugin supports multiple compression algorithms via
`barman-cloud-backup` and `barman-cloud-wal-archive`, allowing you to optimize
for space, speed, or a balance of both.
### Supported Compression Algorithms
- `bzip2`
- `gzip`
- `lz4` (WAL only)
- `snappy`
- `xz` (WAL only)
- `zstd` (WAL only)
Compression settings for base backups and WAL archives are configured
independently. For implementation details, refer to the corresponding API
definitions:
- [`DataBackupConfiguration`](https://pkg.go.dev/github.com/cloudnative-pg/barman-cloud/pkg/api#DataBackupConfiguration)
- [`WALBackupConfiguration`](https://pkg.go.dev/github.com/cloudnative-pg/barman-cloud/pkg/api#WalBackupConfiguration)
:::important
Compression impacts both performance and storage efficiency. Choose the right
algorithm based on your recovery time objectives (RTO), storage capacity, and
network throughput.
:::
## Compression Benchmark (on MinIO)
| Compression | Backup Time (ms) | Restore Time (ms) | Uncompressed Size (MB) | Compressed Size (MB) | Ratio |
| ----------- | ---------------- | ----------------- | ---------------------- | -------------------- | ----- |
| None | 10,927 | 7,553 | 395 | 395 | 1.0:1 |
| bzip2 | 25,404 | 13,886 | 395 | 67 | 5.9:1 |
| gzip | 116,281 | 3,077 | 395 | 91 | 4.3:1 |
| snappy | 8,134 | 8,341 | 395 | 166 | 2.4:1 |

173
web/docs/concepts.md Normal file
View File

@ -0,0 +1,173 @@
---
sidebar_position: 10
---
# Main Concepts
<!-- SPDX-License-Identifier: CC-BY-4.0 -->
:::important
Before proceeding, make sure to review the following sections of the
CloudNativePG documentation:
- [**Backup**](https://cloudnative-pg.io/documentation/current/backup/)
- [**WAL Archiving**](https://cloudnative-pg.io/documentation/current/wal_archiving/)
- [**Recovery**](https://cloudnative-pg.io/documentation/current/recovery/)
:::
The **Barman Cloud Plugin** enables **hot (online) backups** of PostgreSQL
clusters in CloudNativePG through [`barman-cloud`](https://pgbarman.org),
supporting continuous physical backups and WAL archiving to an **object
store**—without interrupting write operations.
It also supports both **full recovery** and **Point-in-Time Recovery (PITR)**
of a PostgreSQL cluster.
## The Object Store
At the core is the [`ObjectStore` custom resource (CRD)](plugin-barman-cloud.v1.md#objectstorespec),
which acts as the interface between the PostgreSQL cluster and the target
object storage system. It allows you to configure:
- **Authentication and bucket location** via the `.spec.configuration` section
- **WAL archiving** settings—such as compression type, parallelism, and
server-side encryption—under `.spec.configuration.wal`
- **Base backup options**—with similar settings for compression, concurrency,
and encryption—under `.spec.configuration.data`
- **Retention policies** to manage the life-cycle of archived WALs and backups
via `.spec.configuration.retentionPolicy`
WAL files are archived in the `wals` directory, while base backups are stored
as **tarballs** in the `base` directory, following the
[Barman Cloud convention](https://docs.pgbarman.org/cloud/latest/usage/#object-store-layout).
:::tip
For details, refer to the
[API reference for the `ObjectStore` resource](plugin-barman-cloud.v1.md#objectstorespec).
:::
## Integration with a CloudNativePG Cluster
CloudNativePG can delegate continuous backup and recovery responsibilities to
the **Barman Cloud Plugin** by configuring the `.spec.plugins` section of a
`Cluster` resource. This setup requires a corresponding `ObjectStore` resource
to be defined.
:::important
While it is technically possible to reuse the same `ObjectStore` for multiple
`Cluster` resources within the same namespace, it is strongly recommended to
dedicate one object store per PostgreSQL cluster to ensure data isolation and
operational clarity.
:::
The following example demonstrates how to configure a CloudNativePG cluster
named `cluster-example` to use a previously defined `ObjectStore` (also named
`cluster-example`) in the same namespace. Setting `isWALArchiver: true` enables
WAL archiving through the plugin:
```yaml
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: cluster-example
spec:
# Other cluster settings...
plugins:
- name: barman-cloud.cloudnative-pg.io
isWALArchiver: true
parameters:
barmanObjectName: cluster-example
```
## Backup of a Postgres Cluster
Once the object store is defined and the `Cluster` is configured to use the
Barman Cloud Plugin, **WAL archiving is activated immediately** on the
PostgreSQL primary.
Physical base backups are seamlessly managed by CloudNativePG using the
`Backup` and `ScheduledBackup` resources, respectively for
[on-demand](https://cloudnative-pg.io/documentation/current/backup/#on-demand-backups)
and
[scheduled](https://cloudnative-pg.io/documentation/current/backup/#scheduled-backups)
backups.
To use the Barman Cloud Plugin, you must set the `method` to `plugin` and
configure the `pluginConfiguration` section as shown:
```yaml
[...]
spec:
method: plugin
pluginConfiguration:
name: barman-cloud.cloudnative-pg.io
[...]
```
With this configuration, CloudNativePG supports:
- Backups from both **primary** and **standby** instances
- Backups from **designated primaries** in a distributed topology using
[replica clusters](https://cloudnative-pg.io/documentation/current/replica_cluster/)
:::tip
For details on how to back up from a standby, refer to the official documentation:
[Backup from a standby](https://cloudnative-pg.io/documentation/current/backup/#backup-from-a-standby).
:::
:::important
Both backup and WAL archiving operations are executed by sidecar containers
running in the same pod as the PostgreSQL `Cluster` primary instance—except
when backups are taken from a standby, in which case the sidecar runs alongside
the standby pod.
The sidecar containers use a [dedicated container image](images.md) that
includes only the supported version of Barman Cloud.
:::
## Recovery of a Postgres Cluster
In PostgreSQL, *recovery* refers to the process of starting a database instance
from an existing backup. The Barman Cloud Plugin integrates with CloudNativePG
to support both **full recovery** and **Point-in-Time Recovery (PITR)** from an
object store.
Recovery in this context is *not in-place*: it bootstraps a brand-new
PostgreSQL cluster from a backup and replays the necessary WAL files to reach
the desired recovery target.
To perform a recovery, define an *external cluster* that references the
appropriate `ObjectStore`, and use it as the source in the `bootstrap` section
of the target cluster:
```yaml
[...]
spec:
[...]
bootstrap:
recovery:
source: source
externalClusters:
- name: source
plugin:
name: barman-cloud.cloudnative-pg.io
parameters:
barmanObjectName: cluster-example
serverName: cluster-example
[...]
```
The critical element here is the `externalClusters` section of the `Cluster`
resource, where the `plugin` stanza instructs CloudNativePG to use the Barman
Cloud Plugin to access the object store for recovery.
This same mechanism can be used for a variety of scenarios enabled by the
CloudNativePG API, including:
* **Full cluster recovery** from the latest backup
* **Point-in-Time Recovery (PITR)**
* Bootstrapping **replica clusters** in a distributed topology
:::tip
For complete instructions and advanced use cases, refer to the official
[Recovery documentation](https://cloudnative-pg.io/documentation/current/recovery/).
:::

View File

@ -1,35 +0,0 @@
---
sidebar_position: 2
---
# Features
This plugin enables continuous backup to object storage for a PostgreSQL
cluster using the [barman-cloud](https://pgbarman.org/) tool suite.
The features provided by this plugin are:
- Data Directory Backup
- Data Directory Restore
- WAL Archiving
- WAL Restoring
- Point-in-Time Recovery (PITR)
- Replica Clusters
This plugin is compatible with all object storage services supported by
barman-cloud, including:
- Amazon AWS S3
- Google Cloud Storage
- Microsoft Azure Blob Storage
The following storage solutions have been tested and confirmed to work with
this implementation:
- [MinIO](https://min.io/) An S3-compatible object storage solution.
- [Azurite](https://github.com/Azure/Azurite) A simulator for Microsoft Azure Blob Storage.
- [fake-gcs-server](https://github.com/fsouza/fake-gcs-server) A simulator for Google Cloud Storage.
Backups created with in-tree object store support can be restored using this
plugin, ensuring compatibility and reliability across environments.

9
web/docs/images.md Normal file
View File

@ -0,0 +1,9 @@
---
sidebar_position: 40
---
# Container Images
<!-- SPDX-License-Identifier: CC-BY-4.0 -->
TODO

View File

@ -1,23 +1,28 @@
---
sidebar_position: 4
sidebar_position: 20
---
# Installation
**IMPORTANT NOTES:**
<!-- SPDX-License-Identifier: CC-BY-4.0 -->
1. The plugin **must** be installed in the same namespace where the operator is
installed (typically `cnpg-system`).
:::important
1. The plugin **must** be installed in the same namespace as the CloudNativePG
operator (typically `cnpg-system`).
2. Be aware that the operator's **listening namespaces** may differ from its
installation namespace. Ensure you verify this distinction to avoid
configuration issues.
2. Keep in mind that the operator's **listening namespaces** may differ from its
installation namespace. Double-check this to avoid configuration issues.
:::
Heres an enhanced version of your instructions for verifying the prerequisites:
## Verifying the Requirements
## Step 1 - Verify the Prerequisites
Before installing the plugin, make sure the [requirements](intro.md#requirements) are met.
If CloudNativePG is installed in the default `cnpg-system` namespace, verify its version using the following command:
### CloudNativePG Version
Ensure you're running a version of CloudNativePG that is compatible with the
plugin. If installed in the default `cnpg-system` namespace, you can verify the
version with:
```sh
kubectl get deployment -n cnpg-system cnpg-controller-manager -o yaml \
@ -30,10 +35,12 @@ Example output:
image: ghcr.io/cloudnative-pg/cloudnative-pg:1.26.0
```
Ensure that the version displayed is **1.26** or newer.
The version **must be 1.26 or newer**.
Then, use the [cmctl](https://cert-manager.io/docs/reference/cmctl/#installation)
tool to confirm that `cert-manager` is correctly installed:
### cert-manager
Use the [cmctl](https://cert-manager.io/docs/reference/cmctl/#installation)
tool to confirm that `cert-manager` is installed and available:
```sh
cmctl check api
@ -45,17 +52,20 @@ Example output:
The cert-manager API is ready
```
Both checks are necessary to proceed with the installation.
Both checks are required before proceeding with the installation.
## Step 2 - Install the barman-cloud Plugin
## Installing the Barman Cloud Plugin
Use `kubectl` to apply the manifest for the latest commit in the `main` branch:
Install the plugin using `kubectl` by applying the manifest for the latest
release:
<!-- x-release-please-start-version -->
```sh
kubectl apply -f \
https://github.com/cloudnative-pg/plugin-barman-cloud/releases/download/v0.3.0/manifest.yaml
```
<!-- x-release-please-end -->
Example output:
@ -80,8 +90,7 @@ certificate.cert-manager.io/barman-cloud-server created
issuer.cert-manager.io/selfsigned-issuer created
```
After these steps, the plugin will be successfully installed. Make sure it is
ready to use by checking the deployment status as follows:
Finally, check that the deployment is up and running:
```sh
kubectl rollout status deployment \
@ -94,4 +103,4 @@ Example output:
deployment "barman-cloud" successfully rolled out
```
This confirms that the plugin is deployed and operational.
This confirms that the plugin is deployed and ready to use.

60
web/docs/intro.md Normal file
View File

@ -0,0 +1,60 @@
---
sidebar_position: 1
sidebar_label: "Introduction"
---
# Barman Cloud Plugin
<!-- SPDX-License-Identifier: CC-BY-4.0 -->
The **Barman Cloud Plugin** for [CloudNativePG](https://cloudnative-pg.io/)
enables online continuous physical backups of PostgreSQL clusters to object storage
using the `barman-cloud` suite from the [Barman](https://docs.pgbarman.org/release/latest/)
project.
## Requirements
To use the Barman Cloud Plugin, you need:
- [CloudNativePG](https://cloudnative-pg.io) version **1.26** <!-- ADD WHEN 1.27 IS OUT "or later" -->
- [cert-manager](https://cert-manager.io/) to enable TLS communication between
the plugin and the operator
## Key Features
This plugin provides the following capabilities:
- Physical online backup of the data directory
- Physical restore of the data directory
- Write-Ahead Log (WAL) archiving
- WAL restore
- Full cluster recovery
- Point-in-Time Recovery (PITR)
- Seamless integration with replica clusters for bootstrap and WAL restore from archive
:::important
The Barman Cloud Plugin is designed to **replace the in-tree object storage support**
previously provided via the `.spec.backup.barmanObjectStore` section in the
`Cluster` resource.
Backups created using the in-tree approach are fully supported and compatible
with this plugin.
:::
## Supported Object Storage Providers
The plugin works with all storage backends supported by `barman-cloud`, including:
- **Amazon S3**
- **Google Cloud Storage**
- **Microsoft Azure Blob Storage**
In addition, the following S3-compatible and simulator solutions have been
tested and verified:
- [MinIO](https://min.io/) An S3-compatible storage solution
- [Azurite](https://github.com/Azure/Azurite) A simulator for Azure Blob Storage
- [fake-gcs-server](https://github.com/fsouza/fake-gcs-server) A simulator for Google Cloud Storage
:::tip
For more details, refer to [Object Store Providers](object_stores.md).
:::

9
web/docs/migration.md Normal file
View File

@ -0,0 +1,9 @@
---
sidebar_position: 40
---
# Migrating from CloudNativePG
<!-- SPDX-License-Identifier: CC-BY-4.0 -->
TODO

76
web/docs/misc.md Normal file
View File

@ -0,0 +1,76 @@
---
sidebar_position: 90
---
# Miscellaneous
<!-- SPDX-License-Identifier: CC-BY-4.0 -->
## Backup Object Tagging
You can attach key-value metadata tags to backup artifacts—such as base
backups, WAL files, and history files—via the `.spec.configuration` section of
the `ObjectStore` resource.
- `tags`: applied to base backups and WAL files
- `historyTags`: applied to history files only
### Example
```yaml
apiVersion: barmancloud.cnpg.io/v1
kind: ObjectStore
metadata:
name: my-store
spec:
configuration:
[...]
tags:
backupRetentionPolicy: "expire"
historyTags:
backupRetentionPolicy: "keep"
[...]
```
## Extra Options for Backup and WAL Archiving
You can pass additional command-line arguments to `barman-cloud-backup` and
`barman-cloud-wal-archive` using the `additionalCommandArgs` field in the
`ObjectStore` configuration.
- `.spec.configuration.data.additionalCommandArgs`: for `barman-cloud-backup`
- `.spec.configuration.wal.additionalCommandArgs`: for `barman-cloud-wal-archive`
Each field accepts a list of string arguments. If an argument is already
configured elsewhere in the plugin, the duplicate will be ignored.
### Example: Extra Backup Options
```yaml
kind: ObjectStore
metadata:
name: my-store
spec:
configuration:
data:
additionalCommandArgs:
- "--min-chunk-size=5MB"
- "--read-timeout=60"
```
### Example: Extra WAL Archive Options
```yaml
kind: ObjectStore
metadata:
name: my-store
spec:
configuration:
wal:
additionalCommandArgs:
- "--max-concurrency=1"
- "--read-timeout=60"
```
For a complete list of supported options, refer to the
[official Barman Cloud documentation](https://docs.pgbarman.org/release/latest/).

View File

@ -1,5 +1,5 @@
---
sidebar_position: 99
sidebar_position: 50
---
# Object Store Providers
@ -359,7 +359,7 @@ write permissions to the bucket.
## MinIO Gateway
MinIO Gateway can proxy requests to cloud object storage providers like S3 or GCS.
For more information, please refer to [MinIO official documentation](https://docs.min.io/).
For more information, refer to [MinIO official documentation](https://docs.min.io/).
### Setup

View File

@ -1,10 +0,0 @@
---
sidebar_position: 3
---
# Prerequisites
To use this plugin, ensure the following prerequisites are met:
- [**CloudNativePG**](https://cloudnative-pg.io) version **1.25** or newer.
- [**cert-manager**](https://cert-manager.io/) for enabling **TLS communication** between the plugin and the operator.

38
web/docs/retention.md Normal file
View File

@ -0,0 +1,38 @@
---
sidebar_position: 60
---
# Retention Policies
<!-- SPDX-License-Identifier: CC-BY-4.0 -->
The Barman Cloud Plugin supports **automated cleanup of obsolete backups** via
retention policies, configured in the `.spec.retentionPolicy` field of the
`ObjectStore` resource.
:::note
This feature uses the `barman-cloud-backup-delete` command with the
`--retention-policy "RECOVERY WINDOW OF {{ value }} {{ unit }}"` syntax.
:::
#### Example: 30-Day Retention Policy
```yaml
apiVersion: barmancloud.cnpg.io/v1
kind: ObjectStore
metadata:
name: my-store
spec:
[...]
retentionPolicy: "30d"
````
:::note
A **recovery window retention policy** ensures the cluster can be restored to
any point in time between the calculated *Point of Recoverability* (PoR) and
the latest WAL archive. The PoR is defined as `current time - recovery window`.
The **first valid backup** is the most recent backup completed before the PoR.
Backups older than that are marked as *obsolete* and deleted after the next
backup completes.
:::

View File

@ -1,14 +1,28 @@
---
sidebar_position: 5
sidebar_position: 30
---
# Usage
# Using the Barman Cloud Plugin
## Defining the `BarmanObjectStore`
<!-- SPDX-License-Identifier: CC-BY-4.0 -->
A `BarmanObjectStore` object should be created for each object store used in
your PostgreSQL architecture. Below is an example configuration for using
MinIO:
After [installing the plugin](installation.md) in the same namespace as the
CloudNativePG operator, enabling your PostgreSQL cluster to use the Barman
Cloud Plugin involves just a few steps:
- Defining the object store containing your WAL archive and base backups, using
your preferred [provider](object_stores.md)
- Instructing the Postgres cluster to use the Barman Cloud Plugin
From that moment, youll be able to issue on-demand backups or define a backup
schedule, as well as rely on the object store for recovery operations.
The rest of this page details each step, using MinIO as object store provider.
## Defining the `ObjectStore`
An `ObjectStore` resource must be created for each object store used in your
PostgreSQL architecture. Here's an example configuration using MinIO:
```yaml
apiVersion: barmancloud.cnpg.io/v1
@ -30,16 +44,15 @@ spec:
compression: gzip
```
The `.spec.configuration` API follows the same schema as the
The `.spec.configuration` schema follows the same format as the
[in-tree barman-cloud support](https://pkg.go.dev/github.com/cloudnative-pg/barman-cloud/pkg/api#BarmanObjectStoreConfiguration).
Refer to [the CloudNativePG documentation](https://cloudnative-pg.io/documentation/preview/backup_barmanobjectstore/)
for detailed usage.
for additional details.
## Configuring WAL Archiving
Once the `BarmanObjectStore` is defined, you can configure a PostgreSQL cluster
to archive WALs by referencing the store in the `.spec.plugins` section, as
shown below:
Once the `ObjectStore` is defined, you can configure your PostgreSQL cluster
to archive WALs by referencing the store in the `.spec.plugins` section:
```yaml
apiVersion: postgresql.cnpg.io/v1
@ -62,9 +75,8 @@ This configuration enables both WAL archiving and data directory backups.
## Performing a Base Backup
Once WAL archiving is enabled, the cluster is ready for backups. To create a
backup, configure the `backup.spec.pluginConfiguration` section to specify this
plugin:
Once WAL archiving is enabled, the cluster is ready for backups. To issue an
on-demand backup, use the following configuration with the plugin method:
```yaml
apiVersion: postgresql.cnpg.io/v1
@ -72,13 +84,17 @@ kind: Backup
metadata:
name: backup-example
spec:
method: plugin
cluster:
name: cluster-example
method: plugin
pluginConfiguration:
name: barman-cloud.cloudnative-pg.io
```
:::note
You can apply the same concept to the `ScheduledBackup` resource.
:::
## Restoring a Cluster
To restore a cluster from an object store, create a new `Cluster` resource that
@ -106,8 +122,9 @@ spec:
size: 1Gi
```
**NOTE:** The above configuration does **not** enable WAL archiving for the
restored cluster.
:::important
The above configuration does **not** enable WAL archiving for the restored cluster.
:::
To enable WAL archiving for the restored cluster, include the `.spec.plugins`
section alongside the `externalClusters.plugin` section, as shown below:

View File

@ -1,13 +0,0 @@
---
sidebar_position: 1
sidebar_label: "Welcome"
---
# Barman Cloud CNPG-I plugin
**Status:** EXPERIMENTAL
Welcome to the documentation of the [barman-cloud](https://pgbarman.org/) CNPG-I
plugin for [CloudNativePG](https://cloudnative-pg.io/).
Test