Getting rsync to delete old non-empty directories containing excluded files

For some older projects we’re still deploying code with rsync, its not perfect but it works. Temporary files are excluded using --exclude-from=exclude.txt, this is great until the parent folder of an excluded file needs to be deleted.

For example, say your file structure looks like this:
Rsync delete file structure

And you remove the data/cache directory completely:

$ rm -rf data/cache
$ rsync --exclude="*.tmp" --exclude="data/logs/*.log"

You’re going to get the error “cannot delete non-empty directory: data/cache” because data/cache/config.tmp has not been removed by rsync.

Unfortunately --delete-excluded won’t work because you don’t want to delete the log files and other .tmp files created on the destination.

Fortunately rsync has filters (which include/exclude are shorthand for anyway), with a mode called “perishable”. Perishable excludes behave exactly as required, files are excluded from sync, and aren’t deleted on the destination unless they are in a directory that no longer exists.

The syntax for excluding files becomes:
$ rsync --filter="-,p *.tmp" --filter="-,p data/logs/"

The “-” signifies the filter is to exclude matches. The “p” makes the exclude perishable.

If, like us, you were using --exclude-from for the patterns, you can with this syntax:
$ rsync --filter="merge,p- /home/ideal/scripts/push/excludes/lenta.txt"

This information is all available in the rsync docs under “FILTER RULES”, it just took me a while to figure out the right syntax. Hopefully these snippets help someone out.

oEmbed fun

I’m way behind the times, I know, but I’ve finally had the opportunity to help build an oEmbed provider. Check it out in action:

To get the chart in to this post I used the ChartBlocks wordpress plugin.

The plugin adds ChartBlocks as an oEmbed provider for WordPress. This means that when I insert the URL:

in to the post on its own line, the blog visits the oEmbed page at:

to find out how to embed the chart.

This saves me having to grab the embed code for the chart, or update if it it ever changes. The internet is awesome sometimes!

High speed train time comparison

I must disclaim, I have no real interest in trains, but this is my second nerdy train post on the blog in a year!?

With all this talk of a new HS3 line I thought I’d compare reported operating speeds against average speeds across some of the better known lines. The results are in:

I tried to keep the distances similar when calculating average speeds from stations, but didn’t do too well. The average speed is the length of track from Station A to B divided by timetabled route length.

From To Miles Hours
Tokyo Sendai 202 1.05
Paris London 306 2.06
London Birmingham 116.66 0.83
Paris Marseilles 466 3

So it would appear the proposed HS2 will operate at speeds of up to 225mph (25mph faster than the Shinkansen) but still be 76mph slower on average.

ChartBlocks goes in to beta

A project I’ve been working on for some time now has hit public beta. It is called “ChartBlocks” and allows you to design and publish charts online in a number of formats, with analytics and sharing built right in.

This will be the first public embed of a chart from the platform:

If you’re looking to create vector charts, which you can download and publish, or share with the internet (through Facebook, twitter, an embed on your website) sign up for the beta here.