Dirvish Backup – Multiple seperate backup schedules

I’ve been used Dirvish now for just over a year. It replaced a number of rsync replication scripts that I had running that were doing rolling backups. While moving to Dirvish has required a few extra scripts to be written, it has been a worthwhile experience. My own scripts weren’t able to handle holding backups for longer periods, at least not gracefully. The biggest issue we had was trying to get Dirvish to do different backups on different schedules. The Dirvish config, while it may look like it allows this at first glance, it really isn’t setup for it. Backups once per day is its bread and butter.

Hopefully this will help clear up a few minor issues with Dirvish and get you running with multiple independent schedules.

Credit where credit is due, some of this is a result of a different sources on Google. We have modified this a number of times over the last year to fit our needs so I’m not totally sure how much of the original remains.

Note: This is from a Debian based system. Paths reflect same.

Initial Dirvish Configuration

For this guide, out setup consists of 1 host which we backup once per day, and the same host which has a directory which gets backed up once per hour. Backups are being stored under /storage/Backups/dirvish.

Our master.conf file – notice that no hosts are actually defined here.

bank:
     /storage/Backups/dirvish
exclude:
     lost+found/
     *~
     .nfs*
expire-default: +15 days
expire-rule:
#       MIN HR    DOM MON       DOW  STRFTIME_FMT
    *   *     *   *         1    +3 months
    *   *     1-7 *         1    +6 months
    *   *     1-7 1,4,7,10  1    +6 month
    *   10-20 *   *         *    +4 days
#   *   *     *   *         2-7  +15 days

runall-daily.conf

Runall:
     host     02:00

runall-hourly.conf

Runall:
      host/hourly/folder

/etc/cron.d/dirvish – This is what calls the jobs

00 01 * * *     root    /usr/sbin/dirvish-expire --quiet   # Expire old images
00 02 * * *     root    /usr/sbin/dirvish-runall --quiet --config runall-daily.conf
00 * * * *      root    /usr/sbin/dirvish-runall --quiet --config runall-hourly.conf

With those in place, our host backs up at 2am every day. The hourly script kicks in every hour. We setup the vaults as normal in the folders defined about. So the hourly is /storage/Backups/dirvish/host/hourly.

Only thing that needs changing is the image-default option in the configs.

Daily vault: image-default: %Y%m%d
Hourly vault: image-default: %Y%m%d%H

Living with it

This setup has run great for us. We get what we need backed up when we need it backed up. There has been a few notable excepts however.

First, one of our hosts started locking up during backup windows. Dirvish then went nuts and started marking incomplete backups correct somehow. We noticed when our bandwidth shot up as it was recopying full machine images across.

Second, our expire rules tend to leave too much data. Yes we could probably fix this, and we probably will when space becomes an issue. I guess the first thing is for our hourlys to reduce down to one a day on the older sets instead of keeping the whole day. But since Dirvish is so good with space, a few months of hourlys isn’t taking too much space.

Overall however, I can’t say we’ll be moving from Dirvish anytime soon for backups, at least for our linux machines.

Race Report – Mondello Open Race [2010-09-02]

Quite possibly the last race I’d get to this year, it was a must. More so after missing two races a few weeks ago due to not having recovered from a hospital visit.

The course was to follow a number of laptops around the Mondello circuit which meant closed roads for the race. But since Mondello is a race course, it is pretty flat. So flat that SportTracks thinks I lost and gained 0m over the course, however I think it just can’t handle the drift in the elevation from the Garmin. Either way, a flat fast course would probably not suit me and I knew it. The goal became to sit in the group, stay with the front group if and when it broke, to do as little as possible on the front, and then not go at the end either at all or until at least 2-3 other people had started their sprint. May sound like a strange goal but they are things I need to learn to get better at racing.

The Warmup

Due to a crash on the N7, I like many others arrived a bit later than I would have wanted. After the sign on and a quick change, I managed a whole 10 minute warmup. But since the evening was warm it would be enough. Or at least it would have to be. Within 5 minutes from stopping, the usual talk about the race was done and we were off.

The Race

While signing on I asked how many laps it would be. The answer was that racing would go for about 50 minutes and then they’d call 3 laps remaining. I should have taken this as a warning sign. Thinking back, I remember looking at the GPS after 10 minutes and being bored. Such a flat course and with a huge number of riders behind us, well there wasn’t going to be a break away.

By 20 minutes I was doing what I could to convince myself not to abandon. To take the race as a training spin and suck it up. The problem however was that it wasn’t overly stressful. Power was about 230-240watts with most of that being the little sprints out of corners when people ahead braked.

And a few people did brake in corners, but not in the way to stop them going into people ahead. At least one person is sticking in my head as everytime I saw him he was accelerating by to a corner and then braking hard. No racing line, no speed through the corner. The first few laps this meant very hard accelerating out of the corner to jump by and grab the wheel ahead. By mid way through the race, I had learnt what he was doing (as had others it seemed), and he started to get left on his line each time as we all took a wider line around and past.

The turn out for the event was high too. 85 I think was the number. Which made for large groups in the corners as people didn’t want to be left behind. Made worse by the fact that the pace was so leisurely. With only short straights and long corners, no pace line really formed. The main straight was usually the slowest part of the course as people slowed right down to try get others to go through.

On the second to last lap I think we were caught. But it was on the last lap going onto the back straight that the pace started picking up. If only we had had this pace from the off, I’d have enjoyed it more.

Blue was speed

In the end, I didn’t finish. I punctured during the 4th last corner. Luckily I held it up and moved to the side and everyone got past. If it had been the next corner, well things could have been different. It was taken faster and usually with more than one or two riders across. But it meant limping home at a slow pace and ironically down the pit lane.

The Summary

Everyone I spoke to after enjoyed it. Things like the big rocks on the corners were all but forgotten at this stage. The closed roads are a plus too. But for me the course was too short and boring. Probably a side affect of all the long 100km+ training rides earlier in the year. Can’t say I’d be jumping to do it again.

The Stats

Distance: 44.47km
Time: 1:05:43
Avg Speed: 40.6km/h
Calories: 956
Avg Power: 243watts
Normalised Power: 267watts