Qnap Postgresql schedule cronjob backup

Recently needs to configure backup for postgresql in qnap nas…. It’s much difficult than imagine…

You can easily google the sample script from postgresql’s wiki.

https://wiki.postgresql.org/wiki/Automated_Backup_on_Linux

However, it cannot be directly used in qnap.

Firstly, psql and pg_dump are not installed in the /bin folder.  You need to use full path for the commands.

Secondly, you will face PQparameterS not found problem…    After some googling, you need to set an env variable (LD_LIBRARY_PATH) in order to run the commands.

Thirdly, the command needs to include -E utf8

Fourthly, the find command in qnap does not support  maxdepth option

Fifthly, the custom backup in qnap does not support compression and that the find command does not support -exec option…

So finally, I added two variables to pg_backup.config file. (you need to create the postgresqlbackup share folder using the GUI first….)

pg_backup.config

##############################
## POSTGRESQL BACKUP CONFIG ##
##############################

##############################
## Added by me ###############
##############################

PSQL_PATH=/share/CACHEDEV1_DATA/.qpkg/PostgreSQL/bin/psql
PG_DUMP_PATH=/share/CACHEDEV1_DATA/.qpkg/PostgreSQL/bin/pg_dump

##### End added by me ########
 
# Optional system user to run backups as.  If the user the script is running as doesn't match this
# the script terminates.  Leave blank to skip check.
BACKUP_USER=
 
# Optional hostname to adhere to pg_hba policies.  Will default to "localhost" if none specified.
HOSTNAME=
 
# Optional username to connect to database as.  Will default to "postgres" if none specified.
USERNAME=
 
# This dir will be created if it doesn't exist.  This must be writable by the user the script is
# running as.
BACKUP_DIR=/share/postgresqlbackup/
 
# List of strings to match against in database name, separated by space or comma, for which we only
# wish to keep a backup of the schema, not the data. Any database names which contain any of these
# values will be considered candidates. (e.g. "system_log" will match "dev_system_log_2010-01")
SCHEMA_ONLY_LIST=""
 
# Will produce a custom-format backup if set to "yes"
ENABLE_CUSTOM_BACKUPS=yes
 
# Will produce a gzipped plain-format backup if set to "yes"
ENABLE_PLAIN_BACKUPS=yes
 
 
#### SETTINGS FOR ROTATED BACKUPS ####
 
# Which day to take the weekly backup from (1-7 = Monday-Sunday)
DAY_OF_WEEK_TO_KEEP=5
 
# Number of days to keep daily backups
DAYS_TO_KEEP=7
 
# How many weeks to keep weekly backups
WEEKS_TO_KEEP=5
 
######################################

Then in the pg_backup_rotated.sh will look like this:

#!/bin/bash

#########################################
##### QNAP 4.1.3/ POSTGRESQL 9.3.4.1 ####
#########################################

export set LD_LIBRARY_PATH=/share/CACHEDEV1_DATA/.qpkg/PostgreSQL/lib/

###########################
####### LOAD CONFIG #######
###########################

while [ $# -gt 0 ]; do
case $1 in
-c)
CONFIG_FILE_PATH=”$2″
shift 2
;;
*)
${ECHO} “Unknown Option \”$1\”” 1>&2
exit 2
;;
esac
done

if [ -z $CONFIG_FILE_PATH ] ; then
SCRIPTPATH=$(cd ${0%/*} && pwd -P)
CONFIG_FILE_PATH=”${SCRIPTPATH}/pg_backup.config”
fi

if [ ! -r ${CONFIG_FILE_PATH} ] ; then
echo “Could not load config file from ${CONFIG_FILE_PATH}” 1>&2
exit 1
fi

source “${CONFIG_FILE_PATH}”

###########################
#### PRE-BACKUP CHECKS ####
###########################

# Make sure we’re running as the required backup user
if [ “$BACKUP_USER” != “” -a “$(id -un)” != “$BACKUP_USER” ] ; then
echo “This script must be run as $BACKUP_USER. Exiting.” 1>&2
exit 1
fi

###########################
### INITIALISE DEFAULTS ###
###########################

if [ ! $HOSTNAME ]; then
HOSTNAME=”localhost”
fi;

if [ ! $USERNAME ]; then
USERNAME=”postgres”
fi;

###########################
#### START THE BACKUPS ####
###########################

function perform_backups()
{
SUFFIX=$1
FINAL_BACKUP_DIR=$BACKUP_DIR”`date +\%Y-\%m-\%d`$SUFFIX/”

echo “Making backup directory in $FINAL_BACKUP_DIR”

if ! mkdir -p $FINAL_BACKUP_DIR; then
echo “Cannot create backup directory in $FINAL_BACKUP_DIR. Go and fix it!” 1>&2
exit 1;
fi;

###########################
### SCHEMA-ONLY BACKUPS ###
###########################

for SCHEMA_ONLY_DB in ${SCHEMA_ONLY_LIST//,/ }
do
SCHEMA_ONLY_CLAUSE=”$SCHEMA_ONLY_CLAUSE or datname ~ ‘$SCHEMA_ONLY_DB'”
done

SCHEMA_ONLY_QUERY=”select datname from pg_database where false $SCHEMA_ONLY_CLAUSE order by datname;”

echo -e “\n\nPerforming schema-only backups”
echo -e “——————————————–\n”

SCHEMA_ONLY_DB_LIST=`”$PSQL_PATH” -h “$HOSTNAME” -U “$USERNAME” -At -c “$SCHEMA_ONLY_QUERY” postgres`

echo -e “The following databases were matched for schema-only backup:\n${SCHEMA_ONLY_DB_LIST}\n”

for DATABASE in $SCHEMA_ONLY_DB_LIST
do
echo “Schema-only backup of $DATABASE”

if ! “$PG_DUMP_PATH” -E utf8 -Fp -s -h “$HOSTNAME” -U “$USERNAME” “$DATABASE” | gzip > $FINAL_BACKUP_DIR”$DATABASE”_SCHEMA.sql.gz.in_progress; then
echo “[!!ERROR!!] Failed to backup database schema of $DATABASE” 1>&2
else
mv $FINAL_BACKUP_DIR”$DATABASE”_SCHEMA.sql.gz.in_progress $FINAL_BACKUP_DIR”$DATABASE”_SCHEMA.sql.gz
fi
done

###########################
###### FULL BACKUPS #######
###########################

for SCHEMA_ONLY_DB in ${SCHEMA_ONLY_LIST//,/ }
do
EXCLUDE_SCHEMA_ONLY_CLAUSE=”$EXCLUDE_SCHEMA_ONLY_CLAUSE and datname !~ ‘$SCHEMA_ONLY_DB'”
done

FULL_BACKUP_QUERY=”select datname from pg_database where not datistemplate and datallowconn $EXCLUDE_SCHEMA_ONLY_CLAUSE order by datname;”

echo -e “\n\nPerforming full backups”
echo -e “——————————————–\n”

for DATABASE in `”$PSQL_PATH” -h “$HOSTNAME” -U “$USERNAME” -At -c “$FULL_BACKUP_QUERY” postgres`
do
if [ $ENABLE_PLAIN_BACKUPS = “yes” ]
then
echo “Plain backup of $DATABASE”

if ! “$PG_DUMP_PATH” -E utf8 -Fp -h “$HOSTNAME” -U “$USERNAME” “$DATABASE” | gzip > $FINAL_BACKUP_DIR”$DATABASE”.sql.gz.in_progress; then
echo “[!!ERROR!!] Failed to produce plain backup database $DATABASE” 1>&2
else
mv $FINAL_BACKUP_DIR”$DATABASE”.sql.gz.in_progress $FINAL_BACKUP_DIR”$DATABASE”.sql.gz
fi
fi

if [ $ENABLE_CUSTOM_BACKUPS = “yes” ]
then
echo “Custom backup of $DATABASE”

if ! “$PG_DUMP_PATH” -E utf8 -Z 0 -Fc -h “$HOSTNAME” -U “$USERNAME” “$DATABASE” | gzip > $FINAL_BACKUP_DIR”$DATABASE”.custom.gz.in_progress; then
echo “[!!ERROR!!] Failed to produce custom backup database $DATABASE”
else
mv $FINAL_BACKUP_DIR”$DATABASE”.custom.gz.in_progress $FINAL_BACKUP_DIR”$DATABASE”.custom.gz
fi
fi

done

echo -e “\nAll database backups complete!”
}

# MONTHLY BACKUPS

DAY_OF_MONTH=`date +%d`

if [ $DAY_OF_MONTH -eq 1 ];
then
# Delete all expired monthly directories
find $BACKUP_DIR -maxdepth 1 -name “*-monthly” | xargs /bin/rm -rf

perform_backups “-monthly”

exit 0;
fi

# WEEKLY BACKUPS

DAY_OF_WEEK=`date +%u` #1-7 (Monday-Sunday)
EXPIRED_DAYS=`expr $((($WEEKS_TO_KEEP * 7) + 1))`

if [ $DAY_OF_WEEK = $DAY_OF_WEEK_TO_KEEP ];
then
# Delete all expired weekly directories
find $BACKUP_DIR -maxdepth 1 -mtime +$EXPIRED_DAYS -name “*-weekly” | xargs /bin/rm -rf

perform_backups “-weekly”

exit 0;
fi

# DAILY BACKUPS

# Delete daily backups 7 days old or more
find $BACKUP_DIR -mtime +$DAYS_TO_KEEP -name “*-daily” | xargs /bin/rm -rf

perform_backups “-daily”

The pg_backup.sh file has no use.

Then you need to put the two files in somewhere. e.g. /share/postgresqlbackup/scripts/

Then you can run the script to see whether it is working…

After that, you need to use crontab in qnap to schedule it to run…  however, crontab in Qnap is not very easy to setup…. you need to see the instruction in the bottom of this page.

http://wiki.qnap.com/wiki/Add_items_to_crontab

1. Edit /etc/config/crontab and add your custom entry.
2. Run ‘crontab /etc/config/crontab’ to load the changes.
3. Restart cron, i.e. ‘/etc/init.d/crond.sh restart’

Remember to chmod 755 /share/postgresqlbackup/scripts/pg_backup_rotated.sh

This works for me in Qnap firmware version 4.1.3 and postgresql 9.3.4.1.  Hope this will work for you. But I do not guarantee this will work for you. Use it at your own risk.

Leave a Reply