Add facility to borg-backup role to run a command and save the output
of it to a separate archive file during the backup process.
This is mostly useful for database backups. Compressed on-disk logs
are terrible for differential backups because revisions have
essentially no common data. By saving the uncompressed stream
directly from mysqldump, we allow borg the chance to de-duplicate,
saving considerable space on the backup servers.
This is implemented for our ansible-managed servers currently doing
dumps. We also add it to the testinfra.
This also separates the archive names for the filesystem and stream
backup with unique prefixes so they can be pruned separately.
Otherwise we end up keeping only one of the stream or filesystem
backups which isn't the intention. However, due to issues with
--append-only mode we are not issuing prune commands at this time.
Note the updated dump commands are updated slightly, particularly with
"--skip-extended-insert" which was suggested by mordred and
significantly improves incremental diff-ability by being slightly more
verbose but keeping much more of the output stable across dumps.
Change-Id: I500062c1c52c74a567621df9aaa716de804ffae7
The database dumps are ~3gb each, so are starting to fill up the disk.
Reduce on-disk rotations to 2 and rely on remote backups more.
Change-Id: Ie8fc8fd7c4c36ca37e6813104f4c711ea55186ab
We're using logrotate to keep a small number of db backups locally. We
write these backups to disk compressed. We don't want logrotate to
recompress them. This is unnecessary extra work.
Change-Id: Iafe1628ff421f47cf3e5cbee14998eeceb60be4c
Release notes for 1.8.3 and 1.8.4 mention a number of useful fixes
(there was no 1.8.1 nor 1.8.2). In particular, a workaround was
included for a known source of client-side hangs (some of our broken
"loading..." pads appear to exhibit similar log entries suggesting
this could address that long-standing problem). Unfortunately, in
the breaking changes, it's noted that plugin compatibility may be
impacted so we should test the one plugin we're using to make sure
it's still okay.
We force the skinName setting to no-skin as the default has changed
to colibris in 1.8.3. We don't want colibris, at least not yet, as it
breaks our existing headings plugin.
We also make a small change to the css in the editor so that each line
has its text fully rendered. See https://github.com/ether/etherpad-lite/issues/4106
for more details.
Change-Id: Ieae29a1d0fe28f879abd35b191291357bb5d3f2c
In places like crontab entries we use full paths to executables because
PATH is different under cron. Unfortunately, this meant we broke
docker-compose commands using /usr/bin/docker-compose when we installed
it under /usr/local/bin/docker-compose. In particular this impacted
database backups on gitea nodes and etherpad.
Update these paths so that everything is happy again.
Change-Id: Ib001baab419325ef1a43ac8e3364e755a6655617
We want to use stop_grace_period to manage gerrit service stops. This
feature was added in docker-compose 1.10 but the distro provides 1.5.
Work around this by installing docker-compose from pypi.
This seems like a useful feature and we want to manage docker-compose
the same way globally so move docker-compose installation into the
install-docker role.
New docker-compose has slightly different output that we must check for
in the gitea start/stop machinery. We also need to check for different
container name formatting in our test cases. We should pause here and
consider if this has any upgrade implications for our existing services.
Change-Id: Ia8249a2b84a2ef167ee4ffd66d7a7e7cff8e21fb
This was copy pasted from the gerrit config and we didn't
s/gerrit/etherpad/ on the log file path for port 80 vhost.
Change-Id: Ie5494c239360a6de3042f34cc3dc050867b92b39
We need to install the ep_headings plugin. We've got a hack in place
to do it in prod, but it seems like every time we npm install the
plugin in ansible, we need to restart the etherpad container
for $reasons.
It's cleaner to just build an image based on upstream with ep_headings
installed.
Change-Id: Id3b31bbabbbc5f6dcfe56486e48916384292f693
We need to install the ep_headings plugin, which we were doing
before. We could also do this by building a custom image, but for
now just bind-mount it in.
Change-Id: I95f7c0dff4f1de8709a1e5e32eb818ef2852a654
Without this we run in development mode. The logs indicate this is
less good and production mode is faster/better.
Change-Id: Idc19bdcb84375f29cb973f5271e8c090465c7f5b
We need to set this in the settings.json.
While we're there, set the defaults in my.cnf. I don't think that
actually matters, but it also shouldn't hurt.
Change-Id: I52e65136383f6c06d1693dd3d396f527c7e0b377
Upstream likes building the settings file into the image, but that's
less exciting, let's bind-mount ours in.
Depends-On: https://review.opendev.org/717491/
Change-Id: Ia1894d884ef2a84e1282345b77fe07bf8898f367
