pygcam.mcs.XMLParameterFile

API

class pygcam.mcs.XMLParameterFile.CSVQuery(element)

Wraps a <CsvQuery> element, which holds CSV query information.

runQuery(tree)

Run a query on a CSV file to find the element in the given column that meets the given constraints.

stringMatch(df)

Handle any string matching constraints since these can’t be handled in a df.query()

class pygcam.mcs.XMLParameterFile.XMLCorrelation(element, param, target)

Store information from a <Correlation> element

classmethod corrMatrix()

Generate a correlation matrix representing the correlation definitions, or None if no Correlation definitions were found.

classmethod createCorrelations(element, param)

A <Correlation> element can have 1 or more “With” elements specifying what to correlate with, e.g., <With name=”otherParam”>0.6</With>

class pygcam.mcs.XMLParameterFile.XMLDataFile(element, param)

Holds DataFrames representing files already loaded, keyed by abs pathname. A single data file can hold vectors of values for multiple Parameters; this way the file is loaded only once. This can be used to load pre-generated trial data, e.g. exported from other software.

Two different file formats are recognized. If the filename ends in ‘.tsv’, the file is assumed to be tab-delimited and the first column is treated as the index, though the numbering is reset. If the filename doesn’t end in ‘.tsv’, it’s assumed to be a CSV file with no index present, just data columns.

The reason for this difference is to support the legacy approach (.tsv) as well as a new approach.

classmethod getData(pathname)

Return the DataFrame created by loading the file at pathname. Read the file if it’s not already in the cache, otherwise, just return the stored DF.

ppf(*args)

Pseudo-ppf that just returns a column of data from the cached dataframe.

class pygcam.mcs.XMLParameterFile.XMLDistribution(element, param)

A wrapper around a <Distribution> element to provide some convenience functions.

ppf(*args)

ppf() takes a first arg that can be a scalar value (percentile) or a list of percentiles, which is how we use it in this application.

class pygcam.mcs.XMLParameterFile.XMLInputFile(element)

<InputFile> abstraction that represents the XML input file(s) that are associated with a given <ScenarioComponent> name. This is necessary because there may be different versions of the file for this name in different scenarios. Each instance of XMLInputFile holds a set of XMLRelFile instances which each represent an actual GCAM input file. Each of these may be referenced from multiple scenarios.

Also supports generating trial data to be used with CSV files in the GCAM data system. In this case use <InputFile name=”whatever” type=”csv”>

callFileFunctions(xmlFile, trialDir)

Call any defined per-input-file functions to allow arbitrary manipulation of XML after applying all distributions.

Parameters:
  • xmlFile – (XMLFile or subclass) container for file being operated on
  • trialDir – (str) this trial’s trial-dir
generateRandomVars()

Called during ‘gensim’ to generate XMLRandomVars, save variable names to the DB, and save the CSV file with all trial data. No need to actually run the queries at this point.

getComponentName()

Return the name attribute of the element for this <InputFile>, which is a unique tag identifying a “logical” file, which may appear in different versions in various scenarios, depending on what those scenarios needed to modify.

loadFiles(context, scenNames, writeConfigFiles=True)

Find the distinct pathnames associated with our component name. Each scenario that refers to this path is stored in a set in self.inputFiles, keyed by pathname. The point is to read and update the target file only once, even if referred to multiple times.

mergeParameters(element)

Save the parameters associated with the given <InputFile> element within self since it holds additional definitions for the same file. This allows a single input file to be referenced by multiple groups of parameters.

runQueries()

Run the queries for all the parameters in this <InputFile> on each of the physical XMLRelFiles associated with this XMLInputFile.

class pygcam.mcs.XMLParameterFile.XMLParameter(element, tree=None)

Stores information for a single parameter definition, whether a distribution, exogenous data, or a Python function. For <Distribution> and <DataFile>, there must be an accompanying <Query> node with an xpath expression.

classmethod applyTrial(simId, trialNum, df)

Copy random values for the given trial from the DataFrame df into this parameter’s elements.

classmethod getInstance(name)

Find an XMLParameter instance by name

classmethod getInstances()

Return a list of the stored XMLParameter objects

runQuery(tree)

Run the XPath query defined for a parameter, save the set of elements found. If tree is None, the XMLRandomVars are created, but the query isn’t run. This is used by “gensim” to create the CSV file holding the trial data.

updateElements(simId, trialNum, df)

Update an element’s text (assuming it’s a number) by multiplying it by a factor, adding a delta, or substituting a given value.

class pygcam.mcs.XMLParameterFile.XMLParameterFile(filename)

Represents the overall parameters.xml file.

getFilename()

Return the filename for this XMLFile

loadInputFiles(context, scenNames, writeConfigFiles=True)

Load the input files, for each scenario in scenNames. Scenarios are found in {simDir}/{scenName}.

writeLocalXmlFiles(trialDir)

Write copies of all modified XML files

class pygcam.mcs.XMLParameterFile.XMLPythonFunc(element, param)

Implements user-defined Python function to be called to generate a series of values. The func can access parameters by calling XMLParameter.getInstances(), or XMLParameter.getInstance(name) to get one by name.

The XMLSchema ensures the format is pkg.mod.submod.func or a legal variant.

ppf(*args)

Call the function specified in the XML file, passing in the arguments to the ppf(), which should be a list of percentile values for which to return values from whatever underlies this Python function.

Parameters:args – the list of percentile values for which to return values. These can be ignored, e.g., if the function computes some values based on other trial data.
Returns:array-like
class pygcam.mcs.XMLParameterFile.XMLQuery(element)

Wraps a <Query> element, which holds an xpath expression.

runQuery(tree)

Run an XPath query on the given tree and return a list of the elements found.

class pygcam.mcs.XMLParameterFile.XMLRandomVar(element, param)

Stores pointer to an XMLDistribution (unless this is a shared RV). Provides access to values by trial number via valueFunc.

ppf(*args)

Pass-thru to the Distribution’s ppf() method. Called by LHS.

class pygcam.mcs.XMLParameterFile.XMLRelFile(inputFile, relPath, simId)

A minor extension to XMLFile to store the original relative pathname that was indicated in the config file identifying this file.

class pygcam.mcs.XMLParameterFile.XMLTrialData(element, param)

An abstract class that provides the protocol required of classes that can produce trial data, i.e., XMLDistribution, XMLDataFile, or XMLPythonFunc

getApply()

Default distro application is ‘direct’, i.e., directly set the element value.

getMode()

Get the containing parameter mode, one of {‘direct’, ‘shared’, ‘independent’}

class pygcam.mcs.XMLParameterFile.XMLVariable(element, param, varNum=None)

Simple variable that wraps an Element and provides methods to get, set, cache, and restore Element values. This is subclassed by XMLRandomVar to provide a ppf() function for generating samples.

setValue(value)

Set the value of the element stored in an XMLVariable.

storeFloatValue()

Store the current value as a float

pygcam.mcs.XMLParameterFile.decache()

Clear all instance caches so a new run can begin cleanly

pygcam.mcs.XMLParameterFile.trialRelativePath(relPath, prefix)

Convert a pathname that was relative to “exe” to be relative to “exe/../../trial-xml” instead. For example, “../input/gcamdata/foo.xml” becomes “../../trial-xml/input/gcamdata/foo.xml”.