Stability HPC Cluster User Guide: https://stabilityai.notion.site/Stability-HPC-Cluster-User-Guide-226c46436df94d24b682239472e36843

SLURM Guide: https://slurm.schedmd.com/quickstart.html

First step is to make sure you have configured your Stability AI user account (see Stability HPC User Guide, specifically “Configure your Stability.AI user account” section.)

When you get to the setting up of your public key in https://hpc.stability.ai:8888/ note that you should paste your public key like so: ssh-ed25519 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA+AAAAC3N [email protected]

SSHing into int1 (change yourname to the username you setup when you first created your stabilityai hpc account, and change .ssh/stability to the path to your private ssh key): ssh -i ~/.ssh/stability yourname[@int1.hpc.stability.ai](<mailto:[email protected]>)

Update: ext1 is now int1 and unfortunately we cannot add anyone new into the cluster until Stability undergoes changes to the cluster to support external research

Setting up your folders & environment

Assuming you are able to ssh into ext1, you should now make a python environment for yourself in your home directory (/admin/home-yourname).

Create venv environment

  1. Create a virtual environment in your home directory
cd ~

python3.11 -m venv mindeye

source mindeye/bin/activate
  1. Edit .bashrc file to ensure you activate the venv every time you ssh into the cluster
nano .bashrc # this navigates you into editing your .bashrc file

# scroll to the bottom of the file and add the following line:

source ~/mindeye/bin/activate

# save your edits: (ctrl-x --> y)
  1. Pip install whatever packages you need

Create your own proj-fmri folder

Make a folder for yourself at /weka/proj-fmri/yourname, the proj-fmri folder is a shared workspace where we do all our work.

Quota limits

Your home folder should take up less than 200GB and your /weka files/folders should take up less than 500GB! If you are past this quota limit you will need to remove files or stash stuff on s3 storage (see below s3 section). You can also make use of /scratch for temporary storage, which has its own independent shared quota amongst all users of several terabytes (only accessible from within compute node). If you use /scratch, remember to delete the files when you are done.