simulation

Contains all helper functions to automate parallelization with MPI, handle caching and stepping of simulations.

Module summary

Classes

Simulation(scheduler, root[, logger, profiler])

Convenience simulation handler, creates a step-by-step simulation sequence and creates file system structure for saving of data to disk.

Functions

MPI_action(action[, iterable, root])	Simulation step MPI post step action decorator.
MPI_single_process(process_id)	Simulation step single process method restriction decorator.
cached_step(caches)	Simulation step caching decorator
iterable_cache(steps, caches[, MPI, log, reduce])	Simulation step cache iteration decorator
iterable_step(iterable[, MPI, log, reduce])	Simulation step iteration decorator.
log_exceptions(func)	If instance has a logger, log exceptions raised by this method.
mpi_copy(*args, **kwargs)
mpi_mkdir(*args, **kwargs)
mpi_rmdir(*args, **kwargs)
mpi_wrap_master_thread(func)	Wrap function to only execute on thread rank=0
store_step(store[, iterable])	Simulation step storing decorator

Contents

Simulation

class sorts.simulation.Simulation(scheduler, root, logger=True, profiler=True, **kwargs)

Bases: object

Convenience simulation handler, creates a step-by-step simulation sequence and creates file system structure for saving of data to disk.

Parameters

• scheduler (Scheduler) – A scheduler instance to run. This input is used to assure that the same logger and profiler is used for the Simulation and the Scheduler.

• root (str/pathlib.Path) – The path to the root folder where all files will be stored.

• logger (bool) – If False, do not instantiate a logger.

• profiler (bool) – If False, do not instantiate a profiler.

branch(name, empty=False, linkfiles=None)

Create branch by creating a copy of the current branch state and checkout that branch. If the branch exists, just checkout that branch.

Parameters

• name (str) – Name of the new branch

• empty (bool) – If True do not copy the state of the current branch.

• linkfiles (list/bool) – If a list of paths that should be soft-linked rather then copied. If True, soft-link all files.

Returns

None

checkout(branch)

Change to branch.

delete(branch)

Delete branch.

get_path(name=None)

Given a relative file path, get the absolute path including root and branch.

property log_path

Path to the current log-output folder

make_paths()

Make all the folder for the current branch according to self.paths.

property paths

List of the name of all folders

Functions

sorts.simulation.MPI_action(action, iterable=False, root=0)

Simulation step MPI post step action decorator.

Parameters

• action (str) – Mode of operations on node-data communication, available options are “gather”, “gather-clear”, “bcast” and “barrier”.

• iterable (bool) – Indicates if the “gather”, “gather-clear” or “bcast” should consider an iterable (that has been parallelized with MPI).

• root (int) – The target MPI process for the “gather”, “gather-clear” and for the source process for “bcast” if iterable=False.

sorts.simulation.MPI_single_process(process_id)

Simulation step single process method restriction decorator.

Parameters

process_id (int) – The process id the wrapped function should only execute on. All other processes return None.

sorts.simulation.cached_step(caches)

Simulation step caching decorator

Parameters

caches (str) – Is a list of strings for the caches to be used, available by default is “h5” and “pickle”. Custom caches are implemented by implementing methods with the string name but prefixed with load_ and save_.

sorts.simulation.iterable_cache(steps, caches, MPI=False, log=False, reduce=None)

Simulation step cache iteration decorator

Parameters

• steps (str/list) – The name/list of names of the cached steps to be iterated over. It uses the step name to find the files in the corresponding folder.

• caches (str/list) – The name/list of cache-methods to be used to load the caches of the steps.

• MPI (bool) – Determines if the iteration should be MPI-parallelized.

• log (bool) – Use the self.logger instance, if it exists, to log the execution of the iteration.

• reduce (function) – A pointer to the binary-function used to reduce the results.

sorts.simulation.iterable_step(iterable, MPI=False, log=False, reduce=None)

Simulation step iteration decorator.

Parameters

• iterable (str/list) – The name/list of names of the instance properties (fetched using getattr) to iterate over. Can be multiple levels, e.g. object.subobject.a_list.

• MPI (bool) – Determines if the iteration should be MPI-parallelized

• log (bool) – Use the self.logger instance, if it exists, to log the execution of the iteration.

• reduce (function) – A pointer to the binary-function used to reduce the results.

sorts.simulation.log_exceptions(func)

If instance has a logger, log exceptions raised by this method.

sorts.simulation.mpi_copy(*args, **kwargs)

sorts.simulation.mpi_mkdir(*args, **kwargs)

sorts.simulation.mpi_rmdir(*args, **kwargs)

sorts.simulation.mpi_wrap_master_thread(func)

Wrap function to only execute on thread rank=0

sorts.simulation.store_step(store, iterable=False)

Simulation step storing decorator

Parameters

• store (str/list) – The name/list names of the properties to save the method return as (set using setattr). Can be multiple levels, e.g. object.subobject.a_property. The order of the names correspond to the order of method returned variables.

• iterable (bool) – Determines if the return of the method is an iteration or not. If its an iteration, it splits the return values into different lists based on the number of variables.