Module request: I-TASSER


#1

I-TASSER is a suite for predicting protein structure. We have a user on our local cluster who puts a lot of CPU hours in on this package, and we’d like to help them expand to a cluster on Alces Flight.

Could it be added to Gridware to simplify this? I’m told by our sysadmins it’s relatively easy to install (we have v5.0), but the license isn’t completely open (free for non-commercial use, but not for commercial use it seems).

https://zhanglab.ccmb.med.umich.edu/I-TASSER/download/

Thanks!


#2

Hi @dwjperry,

Unfortunately, as the I-TASSER code isn’t freely available for download, it’s not possible for us to add this package to the Gridware library. However, one option may be to use the customization feature of Flight Compute to automate the installation of I-TASSER when the cluster is started.

The Flight Compute docs outline how cluster customization works. Here is an outline of the steps you may need to follow which might be helpful:

  • Determine what the name of the customization bucket for your AWS account is, e.g.:
[alces@login1(cluster1) ~]$ alces about customizer
 Customizer bucket prefix: s3://alces-flight-a1i0ytdmvzv3ztv3/customizer
  • Download the code manually from the Zhang Lab site, and upload to your S3 customization bucket at, say s3://alces-flight-<your ID>/apps/I-TASSER5.0.tar.bz2
  • Create a customization script on the cluster, install-itasser.sh that retrieves and installs the software. We’d recommend extracting the archive into the /opt/apps area, which is exported via NFS to all compute nodes. Perhaps something like this (though I imagine there are extra steps that you may also need to include):
#!/bin/bash
PATH=/opt/clusterware/opt/s3cmd:$PATH
s3cmd get s3://alces-flight-<your id>/apps/I-TASSER5.0.tar.bz2 /tmp/I-TASSER5.0.tar.bz2
cd /opt/apps
tar xjf /tmp/I-TASSER5.0.tar.bz2
  • Push the customization script to your customization bucket (NB. this step requires Flight Compute 2017.1r1 or higher; if you’re using an older release, please follow the manual instructions as outlined in the docs):
[alces@login1(cluster1) ~]$ alces customize push install-itasser.sh
Pushing install-itasser to repository account...
Push complete.
Updating repository index...
Repository index updated.
  • Verify your script operates as expected:
[alces@login1(markt-cluster1) ~]$ alces customize apply account/install-itasser
Running event hooks for install-itasser
Running configure hook: /opt/clusterware/var/lib/customizer/account-install-itasser/configure.d/install-itasser.sh
[...]
  • When launching future clusters, specify install-itasser in the “Customization profiles to enable” parameter to instruct Flight Compute to automatically execute the script when the cluster is launched.

Let us know how you get on!


#3

Thank you for the detailed reply, I’ll give that a go!


#4

Took me a while to get around to trying this… seems to be working ( needs more testing), but I’m confused about the customizer bucket behavior. I manually created a bucket (since I’d like it to be in local region given large files) and passed in name at time of cluster creation.

[alces@login1(AWSMPPersonalHPCcomputecluster) ~]$ alces about customizer
Customizer bucket prefix: s3://alces-flight-i-tasser
[alces@login1(AWSMPPersonalHPCcomputecluster) ~]$ nano install-itasser.sh
[alces@login1(AWSMPPersonalHPCcomputecluster) ~]$ ls
install-itasser.sh
[alces@login1(AWSMPPersonalHPCcomputecluster) ~]$ chmod +x install-itasser.sh
[alces@login1(AWSMPPersonalHPCcomputecluster) ~]$ alces customize push install-itasser.sh
Pushing install-itasser to repository account…
Push complete.
Updating repository index…
Repository index updated.

That seems to have been picked up, but the customization bucket actually ends up in another bucket (e.g. alces-flight-ngzk…) in another region. What am I missing?


#5

Hi @dwjperry,

You’re not missing anything, the issue with pushing to your bucket is a known issue in 2017.1 which I’m happy to say has been sorted in the development build for the next Flight release, 2017.2, which is due for release sometime next month.

In the meantime, the issue can be fixed rather easily on your existing Flight instance, it’s just a matter of replacing a value that isn’t overwritten when specifying a custom bucket in /opt/clusterware/etc/cluster-customizer/config.yml. If you open this file with sudo privileges and edit it so it looks like:

profiles: []
repositories:
  feature: "s3://alces-flight-profiles-eu-west-1/2017.1/features"
  account: "s3://alces-flight-i-tasser/customizer"

Then the push command will go to the correct destination.

Let us know if that works for you!