User Guide

Technical Summary

Voyager is an Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS) innovative AI system designed specifically for science and engineering research at scale. Voyager is focused on supporting research in science and engineering that is increasingly dependent upon artificial intelligence and deep learning as a critical element in the experimental and/or computational work. Featuring the Habana Gaudi training and first-generation Habana inference processors, along with a high-performance, low latency 400 gigabit-per-second interconnect from Arista. Voyager will give researchers the ability to work with extremely large data sets using standard AI tools, like TensorFlow and PyTorch, or develop their own deep learning models using developer tools and libraries from Habana Labs.

Voyager is an NSF-funded system, developed in collaboration with Supermicro, and Intel’s Habana Lab and operated by the San Diego Supercomputer Center at UC San Diego, and began a 3-year testbed phase in early 2022.

Resource Allocation Policies

• Current Status: Testbed Phase
• 3-year testbed phase will be available to select focused projects, as well as workshops and industry interactions.
• The testbed phase will be followed up with a 2 year allocation phase to the broader NSF community and User workshops.
• To get access to Voyager, please send a request to HPC Consulting.

Job Scheduling Policies

• Currently no policies set
•  Kubernetes for job scheduling
  
  

Technical Details

Systems Software Environment

System Access

Logging in to Voyager

Voyager uses ssh key pairs for access. Approved users will need to send their ssh public key to consult@sdsc.edu to gain access to the system.

To log in to Voyager from the command line, use the hostname:

login.voyager.sdsc.edu

The following are examples of Secure Shell (ssh) commands that may be used to log in to Expanse:

ssh <your_username>@login.voyager.sdsc.edu
ssh -l <your_username> login.voyager.sdsc.edu

Notes and hints

• Voyager will not maintain local passwords,  your public key will need to be appended to your ~/.ssh/authorized_keys file to enable access from authorized hosts. We accept RSA, ECDSA and ed25519 keys. Make sure you have a strong passphrase on the private key on your local machine.
  • You can use ssh-agent or keychain to avoid repeatedly typing the private key password.
  • Hosts which connect to SSH more frequently than ten times per minute may get blocked for a short period of time
• Do not use the login node for computationally intensive processes, as hosts for running workflow management tools, as primary data transfer nodes for large or numerous data transfers or as servers providing other services accessible to the Internet. The login nodes are meant for file editing, simple data analysis, and other tasks that use minimal compute resources. All computationally demanding jobs should be run using kubernetes.

Voyager cnvrg.io

Voyager will feature cnvrg.io.  cnvrg.io is a machine learning(ML) platform to help manage, build and automate ML workflows.  cnvrg.io will provide a quick and easy way for Voyager users to collaborate, integrate, manage files and submit and monitor jobs.

Adding Users to a Project

Approved Voyager project PIs and co-PIs can add/remove users(accounts) to/from a Voyager. Please submit a support ticket to consult@sdsc.edu to add/remove users.

Modules

Environment Modules provide for dynamic modification of your shell environment. Module commands set, change, or delete environment variables, typically in support of a particular application. They also let the user choose between different versions of the same software or different combinations of related codes.

Voyager uses Lmod, a Lua-based module system. Users will ONLY need  the kubernetes module, which is loaded by default. 

Running Jobs on Voyager

Voyager runs Kubernetes. Kubernetes is an open-source platform for managing containerized workloads and services. A Kubernetes cluster consists of a set of worker machines, called nodes, that run containerized applications. The application workloads are executed by placing containers into Pods to run on nodes. The resources required by the Pods are specified in YAML files. Some basic Kubernetes commands and examples of running jobs are provided below

The Kubernetes command-line tool, kubectl, allows you to run commands against Kubernetes clusters. You can use kubectl to deploy applications, inspect and manage cluster resources, and view logs. For configuration, kubectl looks for a file named config in the $HOME/.kube directory. For more information including a complete list of kubectl operations, see the kubectl reference documentation.

Set up Kubectl environment

On login.voyager.sdsc.edu set up the kubectl environment by loading the kubernetes module: 

$vgr-1-20:~$ module load kubernetes/voyager

Review your currently loaded modules:

$vgr-1-20:~$ module list
Currently Loaded Modules:

1. shared

2. dot

3. default-environment

4. DefaultModules

5. kubernetes/voyager/1.18.15

Usage Guidelines

There are currently no limits set on Voyager resources. The limits for each partition noted in the table below are the maximum available. Resource limits will be set based on Early User Period evaluation.