NumIO
ensembles

Configuration

Ensembles uses an mpi batch script for configuration. This script can be run using sbatch ${SCRIPT_NAME}. If your script is in the same folder you're in and is called "mpi.slurm" you would run: ./sbatch mpi.slurm.

Generating a configuration file

mpiexec Settings

The mpiexec binary is at

Reserve minutes of time.
Use tasks per node.
Use the partition.
Write the log to .
Write errors to .

Numio Settings

If NumIO is disabled, idle for seconds instead.

The ensembles binary is at

Ensembles will log data every seconds.

The graph data will be stored in the directory.

The graph will be stored as a file.

The json data will be stored in the directory.

The matrix for ensembles will be x 500

Numio will perform iterations.

If they are enabled, numio will read and write to the file at every iterations.

Collective comm size will be with a frequency of .

Numio will use a chunk multiplier of with and a compress level of .

Daemon Settings

CPU

This daemon stressed the cpu as much as possible by doing calculations on all available CPU cores.

Idle

This daemon just idles.

RAM

This daemon writes as much as possible into ram.

find

This daemon runs the find utility. This is commonly used to find files on linux systems.

This is where the find binary can be found. Find is usually already in your path. So leaving this as a default should be fine.

This is the directory that will be searched by the find daemon It's ideal if you have read permissions on all of it.

read

This daemon creates a file and the repeatedly reads from it.

This is where the file should be generated

write

This is where the file should be generated.

iperf

IPerf is a program to measure network performance by sending large amounts of data. This daemon required a usable iperf3 server. There is a list of public servers, but you can also set up one on your local network. (List of examples.). The most basic way to do this is to run iperf3 -s on another machine on the same network. This will not work if it's firewalled.

where the iperf3 binary is. it should be installed together with the conda environment and be in your path already.

This is where the iperf server is. Either an ip or a url.

port of the iperf server. by default this will be 5201

      
#!/bin/bash
  
#SBATCH --time=10
#SBATCH --ntasks=1
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --partition=west
#SBATCH --output=job.out
#SBATCH --error=job.err

export ENSEMBLES_IDLE_ONLY=False
export ENSEMBLES_IDLE_ONLY_TIME=12

export ENSEMBLES_MPIEXEC_PATH=mpiexec
export ENSEMBLES_MPIEXEC_NODES=1

export ENSEMBLES_NUMIO_PATH=numio-posix

export ENSEMBLES_LOG_EVERY_X_SECONDS=3
export ENSEMBLES_GRAPH_OUT_PATH=plots
export ENSEMBLES_GRAPH_FILETYPE=pdf
export ENSEMBLES_DATA_OUT_PATH=data

export ENSEMBLES_LINES=500
export ENSEMBLES_ITERATIONS=9000
export ENSEMBLES_PERT=True

export ENSEMBLES_NUMIO_RW_PATH=matrix.out
export ENSEMBLES_NUMIO_RW_FREQUENCY=64
export ENSEMBLES_NUMIO_READ=True
export ENSEMBLES_NUMIO_WRITE=True
export ENSEMBLES_NUMIO_R_IMMEDIATE=True
export ENSEMBLES_NUMIO_W_NOFILESYNC=False
export ENSEMBLES_NUMIO_W_IMMEDIATE=True

export ENSEMBLES_NUMIO_COLLECTIVE_COMMS=True
export ENSEMBLES_NUMIO_COLLECTIVE_COM_SIZE=200
export ENSEMBLES_NUMIO_COLLECTIVE_COM_FREQ=64
export ENSEMBLES_NUMIO_FPISIN=True
export ENSEMBLES_NUMIO_FILE_PER_PROCESS=True
export ENSEMBLES_NUMIO_ASYNC_WRITE=False
export ENSEMBLES_NUMIO_GZIP=True
export ENSEMBLES_NUMIO_LZ4=False
export ENSEMBLES_NUIMIO_CHUNK_MULTIPLIER=1
export ENSEMBLES_NUMIO_COMPRESS_LEVEL=1

export ENSEMBLES_NUMIO_THREAD=1

export ENSEMBLES_BACKGROUND_PROCESS_LIST='[]' 

export ENSEMBLES_IPERF_PATH=iperf3
export ENSEMBLES_IPERF_SERVER_IP=136.172.61.247
export ENSEMBLES_IPERF_PORT=5201

export ENSEMBLES_WRITE_PATH=$HOME/ensembles_write

export ENSEMBLES_READ_PATH=$HOME/ensembles_read

export ENSEMBLES_FIND_PATH=find
export ENSEMBLES_FIND_SEARCH_PATH=$HOME

mpiexec python src/main.py