Stability HPC Cluster User Guide: https://stabilityai.notion.site/Stability-HPC-Cluster-User-Guide-226c46436df94d24b682239472e36843
SLURM Guide: https://slurm.schedmd.com/quickstart.html
First step is to make sure you have configured your Stability AI user account (see Stability HPC User Guide, specifically “Configure your Stability.AI user account” section.)
When you get to the setting up of your public key in https://hpc.stability.ai:8888/ note that you should paste your public key like so: ssh-ed25519 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA+AAAAC3N [email protected]
SSHing into int1 (change yourname to the username you setup when you first created your stabilityai hpc account, and change .ssh/stability to the path to your private ssh key):
ssh -i ~/.ssh/stability yourname[@int1.hpc.stability.ai](<mailto:[email protected]>)
Update: ext1 is now int1 and unfortunately we cannot add anyone new into the cluster until Stability undergoes changes to the cluster to support external research
Assuming you are able to ssh into ext1, you should now make a python environment for yourself in your home directory (/admin/home-yourname).
cd ~
python3.11 -m venv mindeye
source mindeye/bin/activate
nano .bashrc # this navigates you into editing your .bashrc file
# scroll to the bottom of the file and add the following line:
source ~/mindeye/bin/activate
# save your edits: (ctrl-x --> y)
Make a folder for yourself at /weka/proj-fmri/yourname
, the proj-fmri folder is a shared workspace where we do all our work.
Your home folder should take up less than 200GB and your /weka files/folders should take up less than 500GB! If you are past this quota limit you will need to remove files or stash stuff on s3 storage (see below s3 section). You can also make use of /scratch for temporary storage, which has its own independent shared quota amongst all users of several terabytes (only accessible from within compute node). If you use /scratch, remember to delete the files when you are done.