Configuration

The pygcam configuration system allows the user to control a range of parameters defining file locations, parameters related to the parallel execution of Monte Carlo trials, and other project-related parameters.

See also

The pygcam.mcs configuration parameters extend those defined for pygcam. See pygcam’s Configuration System page for more info.

Default values for all configuration parameters defined in pygcam.mcs are provided in pygcam/mcs/etc/mcs.cfg. Values can be overridden in the user’s configuration file (~/.pygcam.cfg) or a site-wide configuration file (at the location indicated by environment variable PYGCAM_SITE_CONFIG). New values in either of these files can establish a different default value that applies to all projects, or define a value on a per-project basis.

The system defaults file

#
# Configuration defaults for pygcam-mcs
#

[DEFAULT]
MCS.Root        = %(Home)s/mcs
MCS.RunDir      = %(MCS.Root)s/%(GCAM.ProjectName)s

MCS.RunDbDir    = %(MCS.RunDir)s/db
MCS.RunSimsDir  = %(MCS.RunDir)s/sims

MCS.EchoSQL         = False
MCS.DbFile          = pygcammcs.sqlite
MCS.DbPath          = %(MCS.RunDbDir)s/%(MCS.DbFile)s

Sqlite.URL          = sqlite:///%(MCS.DbPath)s

### Postgres support ###
# Connection URL format:
# postgresql+psycopg2://user:password@host:port/dbname[?key=value&key=value...]
# ~/.pgpass can contain hostname:port:database:username:password,
# but the file must not be group/world readable (must chmod 600).
Postgres.User    = mcsuser
Postgres.Pass    =
Postgres.Host    = localhost
Postgres.Port    = 5432
Postgres.DbName  = %(GCAM.ProjectName)s
# Can be full path if not in user's PATH
Postgres.CreateDbExe  = createdb
Postgres.CreateDbArgs = -p %(Postgres.Port)s -h %(Postgres.Host)s %(Postgres.DbName)s

Postgres.URL     = postgresql+psycopg2://%(Postgres.User)s:%(Postgres.Pass)s@%(Postgres.Host)s/%(Postgres.DbName)s

MCS.DbURL       = %(Sqlite.URL)s

# args to pass to queued program
MCS.ProgramArgs    =

# Maximum dirs per subdirectory before creating a new level
MCS.MaxSimDirs     = 1000

### Plotting support ###
MCS.PlotDir           = %(MCS.RunDir)s/plot
MCS.PlotType          = png
MCS.PlotShowHistogram = False
MCS.PlotShowKDE       = True
MCS.PlotShowShading   = True

#
# ipyparallel stuff
#
SLURM.StopJobsCommand   = scancel -u %(User)s
PBS.StopJobsCommand     = qselect -u %(User)s | xargs qdel
LSF.StopJobsCommand     = bkill -u %(User)s

IPP.Scheduler     = slurm
IPP.Queue         = slurm
IPP.Account       =
IPP.Profile       = pygcam
IPP.ClusterId     = mcs
IPP.TasksPerNode  = 4
IPP.MaxEngines    = 300
IPP.MinutesPerRun = 20
IPP.MinTimeToRun  = 12
IPP.WorkDir       = %(Home)s/.ipython/profile_%(IPP.Profile)s
IPP.OtherEngineArgs  =
IPP.OtherClusterArgs =
IPP.StartupWaitTries = 5
IPP.StartupWaitSecs  = 5
IPP.StopJobsCommand  = %(SLURM.StopJobsCommand)s
IPP.ResultLoopWaitSecs = 30

# Experimental; these values are no-ops on SLURM
IPP.PrologScript = none
IPP.EpilogScript = none

#
# Config file for gcammcs
#
MCS.RunWorkspace    = %(MCS.RunDir)s/Workspace
MCS.RunInputDir     = %(MCS.RunWorkspace)s/input

# Useful for standard directory setup
MCS.UserFilesDir   = %(GCAM.ProjectDir)s/mcs

# User's parameters.xml and results.xml files are copied to
# {simDir}/app-xml for reference since these might vary by simulation.
MCS.ParametersFile = %(MCS.UserFilesDir)s/parameters.xml
MCS.ResultsFile    = %(MCS.UserFilesDir)s/results.xml

# Where to look for functions specified in <TrialFunc> elements
MCS.TrialFuncDir    = %(MCS.UserFilesDir)s

# Where to look for functions specified in <WriteFunc> elements
MCS.WriteFuncDir    = %(MCS.UserFilesDir)s

# Any directories between the scenario local-xml dir and the scenario name,
# e.g., for scenario files in {simDir}/local-xml/project1/scenario1/config.xml
# you would set this to "project1"
MCS.ScenarioSubdir     = %(GCAM.ScenarioGroup)s

#
# Querying and post-processing results
#
# PostProcessor and DiffScript will be removed once integrated with pygcam
MCS.PostProcessorSteps = diff

# Which years to evaluate
MCS.Years = 2010-2100:5

# Files to link from the reference workspace to run-time MCS workspace.
MCS.WorkspaceFilesToLink = %(GCAM.InputFiles)s

# Set to a dir in which to create GCAM "output" dirs for each trial that
# are symlinked to files on a temporary disk, e.g., that are deleted when
# the job completes. (An alternative to the in-memory database option.)
MCS.TempOutputDir =