Tuesday, 22 October 2013

Well that went quickly...

What seems like only last week but was almost two months ago was a post on what I intended to do given I needed Bpipe working under SLURM. 

Long story short - things went well: https://code.google.com/p/bpipe/source/browse/ReleaseNotes.txt

I'd intended to document the whole process as I went along, but as often is the case, necessity dictates that getting the job done means sometimes neglecting to take the time to record how you did it.

Not that I am too regretful. Diving headfirst has been a great refresher in version control and a chance to try Git, as well as try some rudimentary environments using Vagrant and Puppet

I don't think that the saying 'Show, don't tell' really applies to blog posts, but in order to get another post out the door, I'll risk it.

My steps in betting a Bpipe SLURM implementation were going to involve (the show):
  1. Vagrant (and Chef/Puppet)
  2. Git
  3. Bpipe & SLURM
The tell is:
  1. Vagrant, Puppet and some nasty shell scripts to have a reproducible dev environment for Bpipe:  https://github.com/lonsbio/bpipe-vagrant/tree/master
  2. See 1 and 3.
  3. Using 1, clone and develop a version of Bpipe that (sort of) works under SLURM! https://code.google.com/r/andrewlonsdale-bpipe-dev/
There is still work to do - it is preliminary support only - including adding MPI and SMP support for Bpipe SLURM. This will likely require more work on this (https://github.com/lonsbio/slurm-cluster-vagrant).

I won't promise to document these next steps, and then forget to as I go head first into doing them

However with any luck there will be some more 'tell' soon.