Using a GPU on Skipjack

This guide explains how to request a GPU for class exercises on JHU’s Skipjack cluster. You will use Slurm, the cluster’s job scheduler, to reserve resources before running your code.

1Log in

You need an active JHU ARCH account with access to Skipjack. Open a terminal on your computer and run:

ssh YOUR_JHED_ID@login.arch.jhu.edu

Replace YOUR_JHED_ID with your own username. The terminal displays a login URL and a code. Open that URL in your browser, sign in with your JHED credentials, and complete MFA. Return to the terminal and press Enter when prompted.

Keep the terminal open while working. You are initially on a login node. Use this node to edit files and submit jobs; request a compute allocation before running GPU code.

2Understand the class GPU resources

The class partition is named interactive_gpu. At verification on October 20, 2026, its configuration was:

MIG (Multi-Instance GPU) divides a physical GPU into smaller instances so multiple people can share the hardware. Your request below gives you one instance, rather than an entire A100. Code must fit within that instance’s memory and compute capacity.

Slurm starts jobs as resources become available. Submission does not guarantee an immediate start, and 48 advertised instances does not mean 48 are currently free. Your instructor must coordinate with the cluster manager if the whole class needs guaranteed capacity at a particular time.

3Start an interactive GPU session

After logging in, run:

srun --partition=interactive_gpu \
  --gres=gpu:2g.20gb:1 \
  --ntasks=1 \
  --cpus-per-task=1 \
  --mem=2G \
  --time=01:00:00 \
  --pty bash

This requests one MIG GPU instance, one CPU task, and 2 GB of system RAM for one hour. System RAM is separate from the instance’s approximately 20 GB of GPU memory. Use your instructor’s resource settings if an exercise needs more CPU or system memory.

You may see:

srun: job ... queued and waiting for resources

Leave the command running while you wait. Once the job starts, a shell opens on a compute node. Check your allocation:

hostname
echo "$SLURM_JOB_ID"
nvidia-smi -L

The GPU listing should include a MIG 2g.20gb instance. nvidia-smi may also list physical GPUs; that listing does not mean your job has permission to use all of them.

Run your exercise from this allocated shell. Do not open a new SSH connection and assume it has the same GPU allocation.

4Check Python and run your code

Activate the Python environment specified by your instructor inside the allocated shell. If the course uses environment modules, you can inspect available modules with:

module avail

Follow the course’s installation or environment instructions. Requesting a GPU does not automatically install PyTorch or other Python packages.

If your course environment includes PyTorch, verify GPU access:

python - <<'PY'
import torch

print("CUDA available:", torch.cuda.is_available())
assert torch.cuda.is_available(), "PyTorch cannot access CUDA"
print("Visible devices:", torch.cuda.device_count())
print("Device:", torch.cuda.get_device_name(0))

x = torch.randn(1024, 1024, device="cuda")
y = x @ x
torch.cuda.synchronize()
print("GPU matrix multiplication succeeded:", tuple(y.shape))
PY

Then run your program:

python your_script.py

Your program must explicitly use the GPU. For example, in PyTorch:

import torch

device = torch.device("cuda")
model = model.to(device)
inputs = inputs.to(device)
outputs = model(inputs)

Let Slurm manage GPU visibility; do not manually select another user’s GPU or override CUDA_VISIBLE_DEVICES.

5Monitor and finish your session

Show your own running and pending jobs:

squeue -u "$USER"

Common states:

For more detail about a job:

scontrol show job JOB_ID

Replace JOB_ID with its numeric ID. When finished, leave the allocated shell:

exit

This releases the GPU and returns you to the login shell. To cancel one of your jobs, including a pending job:

scancel JOB_ID

Release resources promptly when you finish. Jobs are limited to their requested duration, so save results and checkpoints before time expires. Keep an interactive session’s SSH terminal connected; use a batch job for work that should run independently of your terminal.

6Run a batch job

A batch job runs a script without requiring an interactive shell. Save this as gpu_job.sbatch:

#!/bin/bash
#SBATCH --job-name=class-gpu
#SBATCH --partition=interactive_gpu
#SBATCH --gres=gpu:2g.20gb:1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH --mem=2G
#SBATCH --time=01:00:00
#SBATCH --output=class-gpu-%j.out

set -euo pipefail

# Add the environment activation commands provided by your instructor here.

hostname
nvidia-smi -L
python your_script.py

From the directory containing your Python script, submit it:

sbatch gpu_job.sbatch

Slurm prints a job ID. Check progress and read the output file:

squeue -u "$USER"
cat class-gpu-JOB_ID.out

Replace JOB_ID with the numeric ID returned by sbatch. The output file appears when the job starts. Batch jobs also need the course environment activation commands; add them before submitting.

After a job ends, inspect its final status:

sacct -j JOB_ID --format=JobID,State,ExitCode,Elapsed

COMPLETED with exit code 0:0 indicates a successful process exit. Check the program’s output to confirm the expected result.

7Troubleshooting

To inspect current partition configuration and node status:

scontrol show partition interactive_gpu
sinfo -p interactive_gpu

8GPU memory and choosing a model

Plan for approximately 20 GB of GPU memory per allocated MIG slice. The physical A100 has 80 GB, but your 2g.20gb allocation exposes only a portion of its memory and compute capacity. The --mem=2G setting in the examples requests system RAM, not GPU memory.

Requesting multiple slices does not automatically combine their memory into a single larger GPU. A model must fit in each slice unless your application explicitly supports splitting it across devices.

For a dense language model on one 20 GB slice, use the following as initial planning estimates. These are not measured limits on Skipjack, and they assume the model is resident on the GPU without CPU or disk offloading.

For class exercises, start with 100–500M parameters for full training, 7–8B for QLoRA, or 7–8B quantized for inference. Validate the exact model and exercise before assigning it to students.

Why the limits vary

GPU memory holds more than model weights:

• Inference: weights, the attention KV cache, temporary buffers, and framework overhead. Longer contexts and larger batches increase memory use.
• Full training: weights, gradients, optimizer state, activations, and temporary buffers. Sequence length and batch size can substantially change the total.
• QLoRA: a frozen, quantized base model plus trainable adapters and training activations. This is adapter fine-tuning, not full training of all base-model parameters.

Useful approximate calculations, using decimal GB:

The exact training-state cost depends on optimizer and precision settings. Quantization metadata and some unquantized layers increase memory above the simple weight estimates. A model whose weights barely fit may still fail during generation or training.

Fitting in memory does not mean a job will finish within four hours. MIG provides a fraction of the physical GPU’s compute resources. Use small experiments, measure runtime, and save checkpoints for longer work.

Sources: Hugging Face model memory guide and bitsandbytes quantization guide. The latter documents a constrained 13B adapter fine-tuning example on a 16 GB GPU; it is not a guarantee for every model or setup.

9Distributed training and inference

Separate independent jobs are the simplest way to use several MIG slices. Each job can train a different model, test a hyperparameter setting, or process a separate portion of an inference dataset, provided its model fits in one slice. Access to multiple simultaneous allocations remains subject to account limits and scheduler availability.

Using several slices for one coordinated model requires additional validation:

Do not assume ordinary multi-GPU tutorials work unchanged on MIG. NVIDIA documents experimental MIG support starting with NCCL 2.31, requiring a driver supporting CUDA 13.0 or later. Each rank must use a distinct MIG instance. MIG does not support NVLink peer-to-peer communication; NVIDIA also specifies disabling Multi-Node NVLink with NCCL_MNNVL_ENABLE=0 for this configuration.

The single-slice tests described above did not verify the installed NCCL/PyTorch versions, multi-slice communication, distributed performance, or permission to allocate several slices. Requesting more slices alone does not establish distributed compatibility.

For an assignment teaching DDP, FSDP, or tensor parallelism, coordinate with the instructor and cluster manager to validate a working setup in advance. A temporary allocation of full GPUs may be more appropriate for these exercises.

Source: NVIDIA NCCL documentation: Using MIG instances.

Verification and limits

On October 20, 2026, two short tests using the instructor’s dkhasha1 account successfully requested a 2g.20gb instance. A CUDA driver test detected one visible MIG device and successfully allocated, wrote, read back, and verified GPU memory. Both jobs completed with exit code 0:0, and their allocations were released.

The PyTorch example is a student check to run in the course environment; that environment was not tested during the infrastructure check. Each student must verify their own account access. Availability and partition configuration can change.

ARCH portal: https://portal.arch.jhu.edu

Support: helpdesk@arch.jhu.edu