Configuration

The pygcam configuration system allows the user to control a range of parameters defining file locations, parameters related to the parallel execution of Monte Carlo trials, and other project-related parameters.

See also

The pygcam.mcs configuration parameters extend those defined for pygcam. See pygcam’s Configuration System page for more info.

Default values for all configuration parameters defined in pygcam.mcs are provided in pygcam/mcs/etc/mcs.cfg. Values can be overridden in the user’s configuration file (~/.pygcam.cfg) or a site-wide configuration file (at the location indicated by environment variable PYGCAM_SITE_CONFIG). New values in either of these files can establish a different default value that applies to all projects, or define a value on a per-project basis.

The system defaults file

#
# Configuration defaults for pygcam's MCS subsystem
#
[DEFAULT]
MCS.SandboxRoot = %(Home)s/mcs

MCS.EchoSQL = False
MCS.DbFile  = pygcammcs.sqlite

MCS.SandboxDir     = %(MCS.SandboxRoot)s/%(GCAM.ProjectName)s/%(GCAM.ProjectSubdir)s
MCS.SandboxDbDir   = %(MCS.SandboxDir)s/db
MCS.SandboxDbPath  = %(MCS.SandboxDbDir)s/%(MCS.DbFile)s
MCS.SandboxSimsDir = %(MCS.SandboxDir)s/sims
MCS.SandboxWorkspace = %(MCS.SandboxDir)s/Workspace
MCS.SandboxWorkspaceInputDir = %(MCS.SandboxWorkspace)s/input

Sqlite.URL  = sqlite:///%(MCS.SandboxDbPath)s
MCS.SandboxDbURL = %(Sqlite.URL)s

# args to pass to queued program
MCS.ProgramArgs =

# Maximum dirs per subdirectory before creating a new level
MCS.MaxSimDirs = 1000

### Plotting support ###
MCS.SandboxPlotDir    = %(MCS.SandboxDir)s/plot
MCS.PlotType          = png
MCS.PlotShowHistogram = False
MCS.PlotShowKDE       = True
MCS.PlotShowShading   = True

#
# Enable optional features to debug high abort rate
#
# Whether to quit R after running the data system
MCS.Debug.QuitR = False

# Whether to decache all cached files involved in applying trial
# data to GCAM input XML files, after writing them out.
MCS.Debug.Decache = False

#
# ipyparallel stuff
#
SLURM.StopJobsCommand   = scancel -u %(User)s
PBS.StopJobsCommand     = qselect -u %(User)s | xargs qdel
LSF.StopJobsCommand     = bkill -u %(User)s

IPP.Scheduler     = slurm
IPP.Queue         = slurm
IPP.Account       =
IPP.Profile       = pygcam
IPP.ClusterId     = mcs
IPP.TasksPerNode  = 4
IPP.MaxEngines    = 300
IPP.MinutesPerRun = 20
IPP.MinTimeToRun  = 12
IPP.WorkDir       = %(Home)s/.ipython/profile_%(IPP.Profile)s
IPP.OtherEngineArgs  =
IPP.OtherClusterArgs =
IPP.StartupWaitTries = 5
IPP.StartupWaitSecs  = 5
IPP.StopJobsCommand  = %(SLURM.StopJobsCommand)s
IPP.ResultLoopWaitSecs = 30

# These values are no-ops on SLURM
IPP.PrologScript = none
IPP.EpilogScript = none

# Whether to create a copy of the reference workspace to isolate the
# sandboxes from potential change to that workspace or to create a
# copy on a better-performing disk on an HPC system.
MCS.CopyWorkspace = True

# Useful for standard directory setup
MCS.ProjectFilesDir   = %(GCAM.ProjectDir)s/mcs

# User's parameters.xml and results.xml files are copied to
# {simDir}/app-xml for reference since these might vary by simulation.
MCS.ProjectParametersFile = %(MCS.ProjectFilesDir)s/parameters.xml
MCS.ProjectResultsFile    = %(MCS.ProjectFilesDir)s/results.xml

# Where to look for functions specified in <TrialFunc> elements
MCS.TrialFuncDir = %(MCS.ProjectFilesDir)s

# Where to look for functions specified in <WriteFunc> elements
MCS.WriteFuncDir = %(MCS.ProjectFilesDir)s

# Deprecated
# Any directories between the scenario local-xml dir and the scenario name,
# e.g., for scenario files in {simDir}/local-xml/project1/scenario1/config.xml
# you would set this to "project1"
# MCS.ScenarioSubdir = %(GCAM.ScenarioSubdir)s

# The steps to run before running the "gcam" step
MCS.SetupSteps = setup,prequery

# Allows skipping generic "setup" step when using setup component steps
MCS.SetupSkipSteps  =

#
# Querying and post-processing results
#
# PostProcessor and DiffScript will be removed once integrated with pygcam
MCS.PostProcessorSteps = diff

# Which years to evaluate
MCS.Years = 2010-2100:5

# Files to link from the reference workspace to run-time MCS workspace.
MCS.WorkspaceFilesToLink = %(GCAM.InputFiles)s

# Set to a dir in which to create GCAM "output" dirs for each trial that
# are symlinked to files on a temporary disk, e.g., that are deleted when
# the job completes. (An alternative to the in-memory database option.)
MCS.TempOutputDir =

# The maximum number of seconds to sleep before running trials in the
# Worker task, meant to avoid overloading a file system with access
# requests for the same file. Zero means don't sleep.
MCS.MaxRandomSleep = 0