Long story short - things went well: https://code.google.com/p/bpipe/source/browse/ReleaseNotes.txt
I'd intended to document the whole process as I went along, but as often is the case, necessity dictates that getting the job done means sometimes neglecting to take the time to record how you did it.
Not that I am too regretful. Diving headfirst has been a great refresher in version control and a chance to try Git, as well as try some rudimentary environments using Vagrant and Puppet
I don't think that the saying 'Show, don't tell' really applies to blog posts, but in order to get another post out the door, I'll risk it.
My steps in betting a Bpipe SLURM implementation were going to involve (the show):
- Vagrant (and Chef/Puppet)
- Bpipe & SLURM
The tell is:
- Vagrant, Puppet and some nasty shell scripts to have a reproducible dev environment for Bpipe: https://github.com/lonsbio/bpipe-vagrant/tree/master
- See 1 and 3.
- Using 1, clone and develop a version of Bpipe that (sort of) works under SLURM! https://code.google.com/r/andrewlonsdale-bpipe-dev/
There is still work to do - it is preliminary support only - including adding MPI and SMP support for Bpipe SLURM. This will likely require more work on this (https://github.com/lonsbio/slurm-cluster-vagrant).
I won't promise to document these next steps, and then forget to as I go head first into doing them
However with any luck there will be some more 'tell' soon.