• Test bed
  • In service
  • Openly-available system for Prototyping and testing
  • Funded by STFC, Hypertec, Dell, AMD
  • Partitions
    • 1 nodes, with 2 Intel Xeon 6756E accelerators per node
      • Manufactured by Dell
      • Scheduler: Direct SSH
    • 1 nodes, with 2 AMD EPYC 9755 accelerators per node
      • Manufactured by Dell
      • Scheduler: Direct SSH
    • 1 nodes, with AMD EPYC 7773X accelerator per node
      • Manufactured by Dell
      • Scheduler: Direct SSH
    • 2 nodes, with 2 AMD EPYC 9654 accelerators per node
      • Manufactured by Dell
      • Scheduler: Slurm
    • 1 nodes, with 2 Intel Xeon Platinum 8260L accelerators per node
      • Manufactured by Intel
      • Scheduler: Slurm
  • Interconnects:

COSMA CPU Compute Lab

COSMA (The Compute Optimised System for Modelling and Analysis) is a High Performance Computing facility hosted at Durham University, operated by the Institute for Computational Cosmology on behalf of DiRAC.

The following CPU testbed nodes are available within COSMA:

Node/Partition RAM CPU Access Account Limitations
mad11 1TB 256 cores (Intel Sierra Forest) Direct SSH/Slurm do025 —
mad12 1.5TB 256 cores (AMD EPYC Turin) Direct SSH/Slurm do025 —
mad06 1TB 128 cores (AMD EPYC Milan-X) Direct SSH/Slurm do025 —
cosma5 1.5TB 256 cores (AMD EPYC Bergamo) Slurm hpcicc ICC collaborators only
cosma8-shm3 0.75TB 192 cores (AMD EPYC Milan) Slurm do009 —
cosma8-ska 4TB 56 cores (Intel Ice Lake) Slurm do011 SKA users only
cosma8-milan 1TB 128 cores (AMD EPYC Milan) Slurm Any DiRAC RAC users only
cosma8-rome 1TB 128 cores (AMD EPYC Rome) Slurm Any DiRAC RAC users only
cosma7-shm2 5.8TB 48 cores (Intel Cascade Lake) Slurm do025 —

Nodes marked “Direct SSH/Slurm” have a hybrid setup where they can be used interactively during the day (8am–7pm), but accept Slurm workloads overnight.

Documentation

Gaining access

Access requires a COSMA account, obtained via the DiRAC SAFE portal.

  1. Create a SAFE account with an institutional email.
  2. Upload an SSH public key on SAFE. If you do not have one, generate with ssh-keygen -t ed25519.
  3. Request a login account. This requires selecting a project, either:
    • A specific project listed in the table above
    • The project code do025 for nodes not tied to specific project
    • A DiRAC project code for a given allocation (provided by a supervisor), if you are already associated with a DiRAC project
  4. Wait for the account to be approved by the project manager. Keep an eye on your email!
  5. Connect to COSMA via SSH: ssh username@login8.cosma.dur.ac.uk (Note: On first login you will be asked to change the password provided in your email)

Visit https://cosma.readthedocs.io/en/latest/account.html for more details. Contact cosma-support@durham.ac.uk for any questions.

Usage

For nodes marked “Direct SSH” above, connect directly via SSH from a login node; for example, ssh mad11. Otherwise, jobs are submitted via Slurm.

Restrictions

On nodes where there is no scheduler, you are requested to monitor node usage (for example, using the w command) to avoid trampling over other people’s benchmarking work.