Stage File Proxy - no more big file dumps

Who doesn't know the following scenario: one is on vacation, maybe on a farm. Letting it all go easy, the sun is shining on your head, cowbells are ringing all around, everything feels just wonderful.
And then ... there comes the inevitable call from a colleague. "Johannes, could you have a look at this thing on site xy and just fix it. We need it till tomorrow." I won't go into if that is a healthy idea on a vacation, but normally I'll dive headlong into the problem. Of course I took my laptop with me just in case something would come up ...

Creating a local development copy "in a minute"

With Git code is set up quickly. In many cases the problem is better testable with actual content of the site, so I also want that. For that task a database dump with Backup-Migrate is handy (configuration is of course bundled into features). Pulling the database is a quick job, as it normally makes up for a file size of a few megabytes. Now we basically got all the content, as far as it consists of text.

But our site also has a lot of images and preview images in views. To get that, the traditional way would be copying the entire sites/default/files-directory from the server (e.g. using FTP). With some projects this can amount to several gigabytes - so this can take long. But we only wanted to perform a quick fix on the site.

Stage File Proxy - in a minute, for real!

To do the sitebuilding work, we only need a few images that are in the subset of content we are working on locally. So it's a mess do really download all the files. This is where Stage File Proxy comes in.

The module saves the work of pulling all the images manually. Instead, with each request the local dev copy checks if it already has the desired file in the local file system. If not, it looks it up on the remote server and downloads it. For that, the module only needs to be installed on your local machine, not the remote one.

Setup is easy:

1. Download the module and put it into the modules folder (drush dl stage_file_proxy)
2. Activate the module (drush en stage_file_proxy)
3. in settings.php enter the remote url: $conf['stage_file_proxy_origin'] = 'http://my-great-domain.com';

Of course the process is also documented in the INSTALL.txt of the module.

Not much to do after that - given the write permissions in the local installation for the files-directory are set correctly.

Stage File Proxy is also taking care of the Imagecache styles. Instead of downloading the derivates created by imagecache, it downloads the respective source iamge. So the derivates can be created locally. If you do not want that, switch off this behaviour by writing into settings.php $conf["stage_file_proxy_use_imagecache_root"] = FALSE;

By setting $conf['stage_file_proxy_hotlink'] = TRUE; you even can avoid downloading files completely. Instead the file requested will be served as a redirect 301 to the fle on the (remote) server.

If you placed your origin installation behind an access authentication (via .htaccess/.htpasswd), you simply have to add your authentication data as domain prefix like that: $conf['stage_file_proxy_origin'] = 'http://username:password@ my-great-domain.com';.

Wrap-up

Okay, the scenario above won't hit you that often on vacation, but almost every day at work if you are a developer. Since the Drupal 7 version of Stage File Proxy is out, I use it on almost every local site. Especially for working in a team it is great. You do not have to pass around big file dumps or tell everyone to get the files via FTP - the module does it by itself, nice and quickly. On a staging site before the go live, it also saves you a lot of time of tedious file copying.

So far I have only had one case where it did not work. We had a site with a lot of videos that were just too big to be downloaded by PHP. But maybe one of you readers already has a solution for that in mind.

You find the module - of course - on drupal.org http://drupal.org/project/stage_file_proxy, there is a dev version for Drupal 6 and Drupal 7. A big thanks to netaustin and robw for their great work!