Syncing remote Craft sites to your development machine

Keeping multiple Craft sites in sync is one of the few painful aspects of developing with Craft. When I'm working on my blog, I write all of my posts on the remote server, but when I want to make changes to the code I want to do it locally. So, on a regular basis, my production blog gets ahead of my local development copy, and in the past I had to manually sync down my data and uploaded assets.

I asked a question about this on the Craft StackExchange, and I got some great answers. I came away thinking there's no perfect solution, so I built two new solutions, and want to present them together with a few great suggestions I got in that thread. Please note that I don't think any of these solutions are perfect for every setting, but instead that they're all tools in the belt of a Craft developer.

In order of ease-of-use, the solutions I've found for syncing assets from a remote Craft site to a local one are the following:

1. My DownloadAssets plugin
2. My SyncCraft shell script
3. Rsync via Gulp (easier if you're already using Gulp)
4. Rsync via Grunt (easier if you're already using Grunt)
5. Capistrano or other automation tools

Note that, just because rsync via Gulp/Grunt may be harder to implement if you're not running Gulp or Grunt already, if you are already running Gulp or Grunt, you'll absolutely want to consider a solution that fits in with a tool you're already using. I personally use Gulp on all of my sites, and will strongly consider using Rsync via Gulp if I can get it to be as flexible as syncCraft. However, I'll be using syncCraft for the moment, since it also syncs my database.

1. DownloadAssets plugin

DownloadAssets is a Craft plugin that adds a dashboard widget that makes it simple to download a Zip archive of all of your assets, either by source or for the entire site. Note that it only downloads the assets from Local sources (not S3, etc.).

2. syncCraft

SyncCraft is a simple shell script that allows you to download and import your Craft database and sync down only new assets into your local asset directory. The initial configuration can be a little bit of work, but once it's set up, syncing down your remote data and files is a snap.

3. Rsync via Gulp

Dave Coggins wrote this fantastic answer on StackOverflow on how to use Rsync via Gulp:

Another alternative to grunt is to use http://gulpjs.com/. This is what I use for minifying css js and assets etc. I've been meaning to setup a way of syncing folders so I've put together gulp task to do it. I have roughly tested it but you might want to look over the code before you use it on a production site :) Be aware that it is setup to sync the folder so it will remove any local files that are not present on your staging/production server.

To use gulp you need to have node.js installed with npm. First install gulp globally:

$ npm install -g gulp

You might need to run that as sudo.

Next, in the root of your craft project create a gulpfile.js that looks something like this:

// Gulp
var gulp = require('gulp');

// Plugins
var rsync = require("rsyncwrapper").rsync;

// Pull down assets and sync local folder
gulp.task('synclocal', function(){
    rsync({
        src: "username@hostname.com:/path/to/assets",
        dest: "assets",
        ssh: true,
        recursive: true,
        syncDest: true,
        compareMode: "checksum"
    },function (error,stdout,stderr,cmd) {
        if ( error ) {
            // failed
            console.log(error.message);
        } else {
            // success
            console.log("folder synced!")
        }
    });
});

Finally we need to make sure rsyncwrapper is install. You can do this by running:

$ npm install rsyncwrapper

You should now be able to run the task by typing:

$ gulp synclocal

4. Rsync via Grunt

Marion Newlevant shared a similar tip, but for Rsync with Grunt instead of Gulp.

5. Capistrano etc.

Capistrano is one of several tools built for automating common tasks on multiple servers. It's extremely powerful and built for managing this type of workflow, but it's also a lot of work to learn. However, if you're looking for something with a lot of power and flexibility, one of these tools will certainly be your best option.

Gabagabo

I hope this writeup will help you get in a better place for keeping your assets in sync between your Craft installs. Do you have suggestions, corrections, or requests for my scripts? Let me know on Twitter at @stauffermatt.