Run PDE-OBS from Git

Copy-ready paths for a single Linux server and the SeaWulf Slurm cluster.

Linux CPU/GPU SeaWulf Slurm Verified smoke first Exact Git revision

Choose the machine

Need a custom factor slice? Use the Benchmark Builder first to generate matching YAML, quality gates, and local or SeaWulf commands.

Planning the paper comparison? Read the matched-mask observation-training protocol before submitting GPU jobs; the random 3% checkpoint belongs to a separate transfer/OOD table.

Single Linux server

Use a virtual environment and tmux. Keep datasets and runs outside Git, then begin with the two-sample smoke workflow.

Open the complete Linux server guide

SeaWulf cluster

Pin one Git commit, build inside an allocation, and chain generation, validation, training, and evaluation with Slurm dependencies.

Open the complete SeaWulf guide

Linux server: verified smoke run

Run these commands after connecting over SSH. Keep the session alive with tmux before starting longer work.

git clone https://github.com/ru1ch3n/PartialObs--PDEBench.git
cd PartialObs--PDEBench
git checkout YOUR_RELEASE_TAG_OR_COMMIT
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip setuptools wheel
python -m pip install ".[train,test]"

export PDEOBS_DATA="$PWD/datasets"
export PDEOBS_RUNS="$PWD/runs"
mkdir -p "$PDEOBS_DATA" "$PDEOBS_RUNS"
pdeobs doctor
pdeobs protocol --check

tmux new -s pdeobs
pdeobs generate --config configs/dataset/smoke.yaml \
  --output "$PDEOBS_DATA/smoke"
pdeobs aggregate --input "$PDEOBS_DATA/smoke" \
  --output "$PDEOBS_DATA/smoke/summary.json" --validate-shards
pdeobs train --config configs/experiment/recovery_unet_smoke.yaml \
  --output "$PDEOBS_RUNS/smoke-train"

Use pdeobs doctor --gpu when CUDA is expected. The full guide also covers the strict 34-sample signal workflow, evaluation, resume, and safe Git updates.

SeaWulf: dependency-chained smoke example

On SeaWulf, do not build or run long work on the login node. This example pins the environment to the checked-out commit and stops downstream work automatically if validation fails.

ssh YOUR_NETID@milan.seawulf.stonybrook.edu
module load slurm
git clone https://github.com/ru1ch3n/PartialObs--PDEBench.git
cd PartialObs--PDEBench
git checkout YOUR_RELEASE_TAG_OR_COMMIT

export PDEOBS_GROUP=YOUR_GROUP
export PDEOBS_COMMIT="$(git rev-parse --short=12 HEAD)"
export PDEOBS_ENV="/gpfs/projects/$PDEOBS_GROUP/envs/pdeobs-$PDEOBS_COMMIT"
export PDEOBS_DATA="/gpfs/scratch/$USER/pdeobs/data"
export PDEOBS_RUNS="/gpfs/scratch/$USER/pdeobs/runs"
mkdir -p logs "$PDEOBS_DATA/plans" "$PDEOBS_RUNS"

# Build only after entering a compute allocation.
srun --partition=short-40core-shared --nodes=1 --ntasks=1 \
  --cpus-per-task=4 --mem=16G --time=02:00:00 --pty bash -l
bash hpc/seawulf/bootstrap.sh
exit

"$PDEOBS_ENV/bin/python" -m pdeobs plan \
  --config configs/dataset/smoke.yaml --tier tiny \
  --output "$PDEOBS_DATA/plans/smoke.jsonl"

generation_job="$(sbatch --parsable --array=0-0 \
  hpc/seawulf/generate_array.sbatch configs/dataset/smoke.yaml \
  "$PDEOBS_DATA/smoke" "$PDEOBS_DATA/plans/smoke.jsonl")"
generation_job="${generation_job%%;*}"

validation_job="$(sbatch --parsable --dependency="afterok:$generation_job" \
  hpc/seawulf/aggregate_cpu.sbatch "$PDEOBS_DATA/smoke" \
  "$PDEOBS_DATA/smoke/summary.json" "$PDEOBS_DATA/plans/smoke.jsonl")"
validation_job="${validation_job%%;*}"

training_job="$(sbatch --parsable --dependency="afterok:$validation_job" \
  hpc/seawulf/train_gpu.sbatch configs/experiment/recovery_unet_smoke.yaml \
  --output "$PDEOBS_RUNS/smoke-train")"
training_job="${training_job%%;*}"
squeue -j "$generation_job,$validation_job,$training_job"
Storage: SeaWulf scratch is temporary and not backed up. Copy valuable validated outputs to an independent archive.

Before scaling