Skip to content

Commit 7121f04

Browse files
gsmolkgsmolk
committed
[Issue #102] New options for 'restore' command used for setting 'restore_command' GUC in recovery.conf: '--archive-host', '--archive-post', 'archive-user' and '--restore-command'
1 parent 435d114 commit 7121f04

File tree

7 files changed

+230
-33
lines changed

7 files changed

+230
-33
lines changed

‎Documentation.md‎

Lines changed: 46 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,7 @@ Current version - 2.1.5
6868
* [Compression Options](#compression-options)
6969
* [Archiving Options](#archiving-options)
7070
* [Remote Mode Options](#remote-mode-options)
71+
* [Remote WAL Archive Options](#remote-wal-archive-options)
7172
* [Replica Options](#replica-options)
7273

7374
7. [Authors](#authors)
@@ -546,6 +547,8 @@ Once the restore command is complete, start the database service.
546547

547548
If you are restoring an STREAM backup, the restore is complete at once, with the cluster returned to a self-consistent state at the point when the backup was taken. For ARCHIVE backups, PostgreSQL replays all available archived WAL segments, so the cluster is restored to the latest state possible. You can change this behavior by using the [recovery target options](#recovery-target-options) with the `restore` command. Note that using the [recovery target options](#recovery-target-options) when restoring STREAM backup is possible if the WAL archive is available at least starting from the time the STREAM backup was taken.
548549

550+
To restore cluster on remote host see the section [Using pg_probackup in the Remote Mode](#using-pg-probackup-in-the-remote-mode).
551+
549552
>NOTE: By default, the [restore](#restore) command validates the specified backup before restoring the cluster. If you run regular backup validations and would like to save time when restoring the cluster, you can specify the `--no-validate` flag to skip validation and speed up the recovery.
550553
551554
#### Partial Restore
@@ -624,6 +627,24 @@ For example, to restore latest backup on remote system using remote mode through
624627

625628
pg_probackup restore -B backup_dir --instance instance_name --remote-user=postgres --remote-host=192.168.0.2 --remote-port=2302
626629

630+
Restoring of ARCHIVE backup or performing PITR in remote mode require additional information: destination address, port and username for establishing ssh connection **from** a host with database **to** a host with backup catalog. This information will be used by `restore_command` to copy via ssh WAL segments from archive to PostgreSQL 'pg_wal' directory.
631+
632+
To solve this problem you can use [Remote Wal Archive Options](#remote-wal-archive-options).
633+
634+
For example, to restore latest backup on remote system using remote mode through SSH connection to user `postgres` on host with address `192.168.0.2` via port `2302` and user `backup` on backup catalog host with address `192.168.0.3` via port `2303`, run:
635+
636+
pg_probackup restore -B backup_dir --instance instance_name --remote-user=postgres --remote-host=192.168.0.2 --remote-port=2302 --archive-host=192.168.0.3 --archive-port=2303 --archive-user=backup
637+
638+
Provided arguments will be used to construct 'restore_command' in recovery.conf:
639+
```
640+
# recovery.conf generated by pg_probackup 2.1.5
641+
restore_command = 'pg_probackup archive-get -B backup_dir --instance instance_name --wal-file-path=%p --wal-file-name=%f --remote-host=192.168.0.3 --remote-port=2303 --remote-user=backup'
642+
```
643+
644+
Alternatively you can use `--restore-command` option to provide an entire 'restore_command':
645+
646+
pg_probackup restore -B backup_dir --instance instance_name --remote-user=postgres --remote-host=192.168.0.2 --remote-port=2302 --restore-command='pg_probackup archive-get -B backup_dir --instance instance_name --wal-file-path=%p --wal-file-name=%f --remote-host=192.168.0.3 --remote-port=2303 --remote-user=backup'
647+
627648
>NOTE: The remote backup mode is currently unavailable for Windows systems.
628649
629650
### Running pg_probackup on Parallel Threads
@@ -1054,7 +1075,7 @@ The sample output is as follows:
10541075
Most fields are consistent with plain format, with some exceptions:
10551076

10561077
- size is in bytes.
1057-
- 'closest-backup-id' attribute contain ID of valid backup closest to the timeline, located on some of the previous timelines. This backup is the closest starting point to reach the timeline from other timelines by PITR. If such backup do not exists, then string is empty.
1078+
- 'closest-backup-id' attribute contain ID of valid backup closest to the timeline, located on some of the previous timelines. This backup is the closest starting point to reach the timeline from other timelines by PITR. Closest backup always has a valid status, either OK or DONE. If such backup do not exists, then string is empty.
10581079
- DEGRADED timelines contain 'lost-segments' array with information about intervals of missing segments. In OK timelines 'lost-segments' array is empty.
10591080
- 'N backups' attribute is replaced with 'backups' array containing backups belonging to the timeline. If timeline has no backups, then 'backups' array is empty.
10601081

@@ -1214,7 +1235,8 @@ Deletes all backups and WAL files associated with the specified instance.
12141235
[--compress-algorithm=compression_algorithm] [--compress-level=compression_level]
12151236
[-d dbname] [-h host] [-p port] [-U username]
12161237
[--archive-timeout=timeout] [--external-dirs=external_directory_path]
1217-
[remote_options] [logging_options]
1238+
[--restore-command=cmdline]
1239+
[remote_options] [remote_archive_options] [logging_options]
12181240

12191241
Adds the specified connection, compression, retention, logging and external directory settings into the pg_probackup.conf configuration file, or modifies the previously defined values.
12201242

@@ -1308,8 +1330,9 @@ For details on usage, see the section [Creating a Backup](#creating-a-backup).
13081330
[-j num_threads] [--progress]
13091331
[-T OLDDIR=NEWDIR] [--external-mapping=OLDDIR=NEWDIR] [--skip-external-dirs]
13101332
[-R | --restore-as-replica] [--no-validate] [--skip-block-validation]
1333+
[--restore-command=cmdline]
13111334
[recovery_options] [logging_options] [remote_options]
1312-
[partial_restore_options]
1335+
[partial_restore_options] [remote_archive_options]
13131336

13141337
Restores the PostgreSQL instance from a backup copy located in the *backup_dir* backup catalog. If you specify a [recovery target option](#recovery-target-options), pg_probackup will find the closest backup and restores it to the specified recovery target. Otherwise, the most recent backup is used.
13151338

@@ -1333,7 +1356,10 @@ Disables block-level checksum verification to speed up validation. During automa
13331356
--no-validate
13341357
Skips backup validation. You can use this flag if you validate backups regularly and would like to save time when running restore operations.
13351358

1336-
Additionally [Recovery Target Options](#recovery-target-options), [Remote Mode Options](#remote-mode-options), [Logging Options](#logging-options), [Partial Restore](#partial-restore) and [Common Options](#common-options) can be used.
1359+
--restore-command=cmdline
1360+
Set the [restore_command](https://www.postgresql.org/docs/current/archive-recovery-settings.html#RESTORE-COMMAND) parameter to specified command. Example: `--restore-command='cp /mnt/server/archivedir/%f "%p"'`
1361+
1362+
Additionally [Recovery Target Options](#recovery-target-options), [Remote Mode Options](#remote-mode-options), [Remote WAL Archive Options](#remote-wal-archive-options), [Logging Options](#logging-options), [Partial Restore](#partial-restore) and [Common Options](#common-options) can be used.
13371363

13381364
For details on usage, see the section [Restoring a Cluster](#restoring-a-cluster).
13391365

@@ -1656,9 +1682,24 @@ Specifies pg_probackup installation directory on the remote system.
16561682
--ssh-options=ssh_options
16571683
Specifies a string of SSH command-line options. For example, the following options can used to set keep-alive for ssh connections opened by pg_probackup: `--ssh-options='-o ServerAliveCountMax=5 -o ServerAliveInterval=60'`. Full list of possible options can be found on [ssh_config manual page](https://man.openbsd.org/ssh_config.5).
16581684

1685+
#### Remote WAL Archive Options
1686+
1687+
This section describes the options used to provide the arguments for [Remote Mode Options](#remote-mode-options) in [archive-get](#archive-get) used in [restore_command](https://www.postgresql.org/docs/current/archive-recovery-settings.html#RESTORE-COMMAND) command when restoring ARCHIVE backup or performing PITR.
1688+
1689+
--archive-host=destination
1690+
Provides the argument for `--remote-host` option in `archive-get` command.
1691+
1692+
--archive-port=port
1693+
Default: 22
1694+
Provides the argument for `--remote-port` option in `archive-get` command.
1695+
1696+
--archive-user=username
1697+
Default: PostgreSQL user
1698+
Provides the argument for `--remote-user` option in `archive-get` command. If you omit this option, the the user running PostgreSQL cluster is used.
1699+
16591700
#### Partial Restore Options
16601701

1661-
This section describes the options related to partial restore of cluster from backup. These options can be used with [restore](#restore) command.
1702+
This section describes the options related to partial restore of a cluster from backup. These options can be used with [restore](#restore) command.
16621703

16631704
--db-exclude=dbname
16641705
Specifies database name to exclude from restore. All other databases in the cluster will be restored as usual, including `template0` and `template1`. This option can be specified multiple times for multiple databases.

‎src/configure.c‎

Lines changed: 33 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -122,97 +122,112 @@ ConfigOption instance_options[] =
122122
&instance_config.archive_timeout, SOURCE_CMD, SOURCE_DEFAULT,
123123
OPTION_ARCHIVE_GROUP, OPTION_UNIT_S, option_get_value
124124
},
125+
{
126+
's', 208, "archive-host",
127+
&instance_config.archive.host, SOURCE_CMD, 0,
128+
OPTION_ARCHIVE_GROUP, 0, option_get_value
129+
},
130+
{
131+
's', 209, "archive-port",
132+
&instance_config.archive.port, SOURCE_CMD, 0,
133+
OPTION_ARCHIVE_GROUP, 0, option_get_value
134+
},
135+
{
136+
's', 210, "archive-user",
137+
&instance_config.archive.user, SOURCE_CMD, 0,
138+
OPTION_ARCHIVE_GROUP, 0, option_get_value
139+
},
125140
/* Logging options */
126141
{
127-
'f', 208, "log-level-console",
142+
'f', 211, "log-level-console",
128143
assign_log_level_console, SOURCE_CMD, 0,
129144
OPTION_LOG_GROUP, 0, get_log_level_console
130145
},
131146
{
132-
'f', 209, "log-level-file",
147+
'f', 212, "log-level-file",
133148
assign_log_level_file, SOURCE_CMD, 0,
134149
OPTION_LOG_GROUP, 0, get_log_level_file
135150
},
136151
{
137-
's', 210, "log-filename",
152+
's', 213, "log-filename",
138153
&instance_config.logger.log_filename, SOURCE_CMD, 0,
139154
OPTION_LOG_GROUP, 0, option_get_value
140155
},
141156
{
142-
's', 211, "error-log-filename",
157+
's', 214, "error-log-filename",
143158
&instance_config.logger.error_log_filename, SOURCE_CMD, 0,
144159
OPTION_LOG_GROUP, 0, option_get_value
145160
},
146161
{
147-
's', 212, "log-directory",
162+
's', 215, "log-directory",
148163
&instance_config.logger.log_directory, SOURCE_CMD, 0,
149164
OPTION_LOG_GROUP, 0, option_get_value
150165
},
151166
{
152-
'U', 213, "log-rotation-size",
167+
'U', 216, "log-rotation-size",
153168
&instance_config.logger.log_rotation_size, SOURCE_CMD, SOURCE_DEFAULT,
154169
OPTION_LOG_GROUP, OPTION_UNIT_KB, option_get_value
155170
},
156171
{
157-
'U', 214, "log-rotation-age",
172+
'U', 217, "log-rotation-age",
158173
&instance_config.logger.log_rotation_age, SOURCE_CMD, SOURCE_DEFAULT,
159174
OPTION_LOG_GROUP, OPTION_UNIT_MS, option_get_value
160175
},
161176
/* Retention options */
162177
{
163-
'u', 215, "retention-redundancy",
178+
'u', 218, "retention-redundancy",
164179
&instance_config.retention_redundancy, SOURCE_CMD, 0,
165180
OPTION_RETENTION_GROUP, 0, option_get_value
166181
},
167182
{
168-
'u', 216, "retention-window",
183+
'u', 219, "retention-window",
169184
&instance_config.retention_window, SOURCE_CMD, 0,
170185
OPTION_RETENTION_GROUP, 0, option_get_value
171186
},
172187
/* Compression options */
173188
{
174-
'f', 217, "compress-algorithm",
189+
'f', 220, "compress-algorithm",
175190
assign_compress_alg, SOURCE_CMD, 0,
176191
OPTION_COMPRESS_GROUP, 0, get_compress_alg
177192
},
178193
{
179-
'u', 218, "compress-level",
194+
'u', 221, "compress-level",
180195
&instance_config.compress_level, SOURCE_CMD, 0,
181196
OPTION_COMPRESS_GROUP, 0, option_get_value
182197
},
183198
/* Remote backup options */
184199
{
185-
's', 219, "remote-proto",
200+
's', 222, "remote-proto",
186201
&instance_config.remote.proto, SOURCE_CMD, 0,
187202
OPTION_REMOTE_GROUP, 0, option_get_value
188203
},
189204
{
190-
's', 220, "remote-host",
205+
's', 223, "remote-host",
191206
&instance_config.remote.host, SOURCE_CMD, 0,
192207
OPTION_REMOTE_GROUP, 0, option_get_value
193208
},
194209
{
195-
's', 221, "remote-port",
210+
's', 224, "remote-port",
196211
&instance_config.remote.port, SOURCE_CMD, 0,
197212
OPTION_REMOTE_GROUP, 0, option_get_value
198213
},
199214
{
200-
's', 222, "remote-path",
215+
's', 225, "remote-path",
201216
&instance_config.remote.path, SOURCE_CMD, 0,
202217
OPTION_REMOTE_GROUP, 0, option_get_value
203218
},
204219
{
205-
's', 223, "remote-user",
220+
's', 226, "remote-user",
206221
&instance_config.remote.user, SOURCE_CMD, 0,
207222
OPTION_REMOTE_GROUP, 0, option_get_value
208223
},
209224
{
210-
's', 224, "ssh-options",
225+
's', 227, "ssh-options",
211226
&instance_config.remote.ssh_options, SOURCE_CMD, 0,
212227
OPTION_REMOTE_GROUP, 0, option_get_value
213228
},
214229
{
215-
's', 225, "ssh-config",
230+
's', 228, "ssh-config",
216231
&instance_config.remote.ssh_config, SOURCE_CMD, 0,
217232
OPTION_REMOTE_GROUP, 0, option_get_value
218233
},

0 commit comments

Comments
 (0)