simulation¶
Contains all helper functions to automate parallelization with MPI, handle caching and stepping of simulations.
Module summary¶
Classes
|
Convenience simulation handler, creates a step-by-step simulation sequence and creates file system structure for saving of data to disk. |
Functions
|
Simulation step MPI post step action decorator. |
|
Simulation step single process method restriction decorator. |
|
Simulation step caching decorator |
|
Simulation step cache iteration decorator |
|
Simulation step iteration decorator. |
|
If instance has a logger, log exceptions raised by this method. |
|
|
|
|
|
|
|
Wrap function to only execute on thread rank=0 |
|
Simulation step storing decorator |
Contents¶
Simulation
-
class
sorts.simulation.
Simulation
(scheduler, root, logger=True, profiler=True, **kwargs)[source] Bases:
object
Convenience simulation handler, creates a step-by-step simulation sequence and creates file system structure for saving of data to disk.
- Parameters
scheduler (Scheduler) – A scheduler instance to run. This input is used to assure that the same logger and profiler is used for the Simulation and the Scheduler.
root (str/pathlib.Path) – The path to the root folder where all files will be stored.
logger (bool) – If
False
, do not instantiate a logger.profiler (bool) – If
False
, do not instantiate a profiler.
-
branch
(name, empty=False, linkfiles=None)[source] Create branch by creating a copy of the current branch state and checkout that branch. If the branch exists, just checkout that branch.
- Parameters
name (str) – Name of the new branch
empty (bool) – If
True
do not copy the state of the current branch.linkfiles (list/bool) – If a list of paths that should be soft-linked rather then copied. If
True
, soft-link all files.
- Returns
None
-
checkout
(branch)[source] Change to branch.
-
delete
(branch)[source] Delete branch.
-
get_path
(name=None)[source] Given a relative file path, get the absolute path including root and branch.
-
property
log_path
Path to the current log-output folder
-
make_paths
()[source] Make all the folder for the current branch according to
self.paths
.
-
property
paths
List of the name of all folders
Functions
-
sorts.simulation.
MPI_action
(action, iterable=False, root=0)[source] Simulation step MPI post step action decorator.
- Parameters
action (str) – Mode of operations on node-data communication, available options are “gather”, “gather-clear”, “bcast” and “barrier”.
iterable (bool) – Indicates if the “gather”, “gather-clear” or “bcast” should consider an iterable (that has been parallelized with MPI).
root (int) – The target MPI process for the “gather”, “gather-clear” and for the source process for “bcast” if
iterable=False
.
-
sorts.simulation.
MPI_single_process
(process_id)[source] Simulation step single process method restriction decorator.
- Parameters
process_id (int) – The process id the wrapped function should only execute on. All other processes return
None
.
-
sorts.simulation.
cached_step
(caches)[source] Simulation step caching decorator
- Parameters
caches (str) – Is a list of strings for the caches to be used, available by default is “h5” and “pickle”. Custom caches are implemented by implementing methods with the string name but prefixed with
load_
andsave_
.
-
sorts.simulation.
iterable_cache
(steps, caches, MPI=False, log=False, reduce=None)[source] Simulation step cache iteration decorator
- Parameters
steps (str/list) – The name/list of names of the cached steps to be iterated over. It uses the step name to find the files in the corresponding folder.
caches (str/list) – The name/list of cache-methods to be used to load the caches of the steps.
MPI (bool) – Determines if the iteration should be MPI-parallelized.
log (bool) – Use the
self.logger
instance, if it exists, to log the execution of the iteration.reduce (function) – A pointer to the binary-function used to reduce the results.
-
sorts.simulation.
iterable_step
(iterable, MPI=False, log=False, reduce=None)[source] Simulation step iteration decorator.
- Parameters
iterable (str/list) – The name/list of names of the instance properties (fetched using
getattr
) to iterate over. Can be multiple levels, e.g.object.subobject.a_list
.MPI (bool) – Determines if the iteration should be MPI-parallelized
log (bool) – Use the
self.logger
instance, if it exists, to log the execution of the iteration.reduce (function) – A pointer to the binary-function used to reduce the results.
-
sorts.simulation.
log_exceptions
(func)[source] If instance has a logger, log exceptions raised by this method.
-
sorts.simulation.
mpi_copy
(*args, **kwargs)[source]
-
sorts.simulation.
mpi_mkdir
(*args, **kwargs)[source]
-
sorts.simulation.
mpi_rmdir
(*args, **kwargs)[source]
-
sorts.simulation.
mpi_wrap_master_thread
(func)[source] Wrap function to only execute on thread rank=0
-
sorts.simulation.
store_step
(store, iterable=False)[source] Simulation step storing decorator
- Parameters
store (str/list) – The name/list names of the properties to save the method return as (set using
setattr
). Can be multiple levels, e.g.object.subobject.a_property
. The order of the names correspond to the order of method returned variables.iterable (bool) – Determines if the return of the method is an iteration or not. If its an iteration, it splits the return values into different lists based on the number of variables.