pygcam.utils
¶
This module contains constants, functions, and classes that are used by various other modules of the pygcam toolkit.
API¶
-
class
pygcam.utils.
ParseCommaList
(option_strings, dest, nargs=None, **kwargs)¶
-
pygcam.utils.
coercible
(value, type, raiseError=True)¶ Attempt to coerce a value to type and raise an error on failure.
Parameters: - value – any value coercible to type
- type – any Python type
Returns: (type) the coerced value, if it’s coercible, otherwise None if raiseError is False
Raises: PygcamException – if not coercible and raiseError is True
-
pygcam.utils.
copyFileOrTree
(src, dst)¶ Copy src to dst, where the two can both be files or directories. If src and dst are directories, dst must not exist yet.
Parameters: - src – (str) path to a source file or directory
- dst – (str) path to a destination file or directory.
Returns: none
-
pygcam.utils.
copyResource
(relpath, dest, overwrite=True)¶ Copy a resource from the ‘pygcam’ package to the given destination.
Parameters: - relpath – (str) a path relative to the pygcam package
- dest – (str) the pathname of the file to create by copying the resource.
- overwrite – (bool) if False, raise an error if the destination file already exists.
Returns: none
-
pygcam.utils.
deleteFile
(filename)¶ Delete the given filename, but ignore errors, like “rm -f”
Parameters: filename – (str) the file to remove Returns: none
-
pygcam.utils.
digitColumns
(df, asInt=False)¶ Get a list of columns with integer names (as strings, e.g., “2007”) in df. If asInt is True return as a list of integers, otherwise as strings.
-
pygcam.utils.
ensureCSV
(file)¶ Ensure that the file has a ‘.csv’ extension by replacing or adding the extension, as required.
Parameters: file – (str) a filename Returns: (str) the filename with a ‘.csv’ extension.
-
pygcam.utils.
ensureExtension
(filename, ext)¶ Force a filename to have the given extension, ext, adding it to any other extension, if present. That is, if filename is
foo.bar
, and ext isbaz
, the result will befoo.bar.baz
. If ext doesn’t start with a “.”, one is added.Parameters: - filename – filename
- ext – the desired filename extension
Returns: filename with extension ext
-
pygcam.utils.
flatten
(listOfLists)¶ Flatten one level of nesting given a list of lists. That is, convert [[1, 2, 3], [4, 5, 6]] to [1, 2, 3, 4, 5, 6].
Parameters: listOfLists – a list of lists, obviously Returns: the flattened list
-
pygcam.utils.
getBatchDir
(scenario, resultsDir)¶ Get the name of the directory holding batch query results..
Parameters: - scenario – (str) the name of a scenario
- resultsDir – (str) the directory in which the batch results directory should be created
Returns: (str) the pathname to the batch results directory
-
pygcam.utils.
getBooleanXML
(value)¶ Get a value from an XML file and convert it into a boolean True or False.
Parameters: value – any value (it’s first converted to a string) Returns: True if the value is in [‘true’, ‘yes’, ‘1’], False if the value is in [‘false’, ‘no’, ‘0’]. An exception is raised if any other value is passed. Raises: PygcamException
-
pygcam.utils.
getRegionList
(workspace=None, states='withGlobal')¶ Set the list of the defined region names from the data system, if possible, otherwise use the built-in list of 32 regions.
Parameters: - workspace – (str) the path to a
Main_User_Workspace
directory that has the fileinput/gcamdata/inst/extdata/common/GCAM_region_names.csv
, orNone
, in which case the value of config variableGCAM.SourceWorkspace
(if defined) is used. If workspace is empty orNone
, and the config variableGCAM.SourceWorkspace
is empty (the default value), the built-in default 32-region list is returned. - states – (str) One of {‘together’, ‘only’, ‘none’}. Default is ‘together’.
Returns: a list of strings with the names of the defined regions
- workspace – (str) the path to a
-
pygcam.utils.
getResource
(relpath)¶ Extract a resource (e.g., file) from the given relative path in the pygcam package.
Parameters: relpath – (str) a path relative to the pygcam package Returns: the file contents
-
pygcam.utils.
getYearCols
(years, timestep=5)¶ Generate a list of names of year columns in GCAM result files from a string indicating a year range.
Parameters: - years – (str) a string of the form “2020-2050”
- timestep – (int) the number of years between timesteps
Returns: (list of strings) the names of the corresponding columns
-
pygcam.utils.
get_path
(pathname, defaultDir)¶ Return pathname if it’s an absolute pathname, otherwise return the path composed of pathname relative to the given defaultDir.
-
pygcam.utils.
importFrom
(modname, objname, asTuple=False)¶ Import modname and return reference to objname within the module.
Parameters: - modname – (str) the name of a Python module
- objname – (str) the name of an object in module modname
- asTuple – (bool) if True a tuple is returned, otherwise just the object
Returns: (object or (module, object)) depending on asTuple
-
pygcam.utils.
importFromDotSpec
(spec)¶ Import an object from an arbitrary dotted sequence of packages, e.g., “a.b.c.x” by splitting this into “a.b.c” and “x” and calling importFrom().
Parameters: spec – (str) a specification of the form package.module.object Returns: none Raises: PygcamException – if the import fails
-
pygcam.utils.
is_abspath
(pathname)¶ Return True if pathname is an absolute pathname, else False.
-
pygcam.utils.
loadModuleFromPath
(modulePath, raiseOnError=True)¶ Load a module from a ‘.py’ or ‘.pyc’ file from a path that ends in the module name, i.e., from “foo/bar/Baz.py”, the module name is ‘Baz’.
Parameters: - modulePath – (str) the pathname of a python module (.py or .pyc)
- raiseOnError – (bool) if True, raise an error if the module cannot be loaded
Returns: (module) a reference to the loaded module, if loaded, else None.
Raises: PygcamException
-
pygcam.utils.
mkdirs
(newdir, mode=504)¶ Try to create the full path newdir and ignore the error if it already exists.
Parameters: newdir – the directory to create (along with any needed parent directories) Returns: nothing
-
pygcam.utils.
model_years
()¶ Return the defined model years by parsing xml/modeltime.xml. This is used by the setup plugin to convert years to model period for setting the stop year.
Returns: (list of int) the defined model years.
-
pygcam.utils.
printSeries
(series, label, header='', asStr=False)¶ Print a series of values, with a give label.
Parameters: - series – (convertible to pandas Series) the values
- label – (str) a label to print for the data
Returns: none
-
pygcam.utils.
pushd
(directory)¶ Context manager that changes to the given directory and then returns to the original directory. Usage is
with pushd('/foo/bar'): ...
Parameters: directory – (str) a directory to chdir to temporarily Returns: none
-
pygcam.utils.
queueForStream
(stream)¶ Create a thread to read from a non-socket file descriptor and its contents to a socket so non-blocking read via select() works on Windows. (Since Windows doesn’t support select on pipes.)
Parameters: stream – (file object) the input to read from, presumably a pipe from a subprocess Returns: (int) a file descriptor for the socket to read from.
-
pygcam.utils.
readScenarioName
(configFile)¶ Read the file configFile and extract the scenario name.
Parameters: configFile – (str) the path to a GCAM configuration file Returns: (str) the name of the scenario defined in configFile
-
pygcam.utils.
removeFileOrTree
(path, raiseError=True)¶ Remove a file or an entire directory tree. Handles removal of symlinks on Windows, as these are treated differently in that system.
Parameters: - path – (str) the pathname of a file or directory.
- raiseError – (bool) if True, re-raise any error that occurs during the file operations, else errors are ignored.
Returns: none
-
pygcam.utils.
resourceStream
(relpath)¶ Return a stream on the resource found on the given path relative to the pygcam package.
Parameters: relpath – (str) a path relative to the pygcam package Returns: (file-like stream) a file-like buffer opened on the desired resource.
-
pygcam.utils.
saveToFile
(txt, dirname='', filename='')¶ Save the given text to a file in the given directory.
Parameters: - txt – (str) the text to save
- dirname – (str) path to a directory
- filename – (str) the name of the file to create
Returns: none
-
pygcam.utils.
shellCommand
(command, shell=True, raiseError=True)¶ Run a shell command and optionally raise PygcamException error.
Parameters: - command – the command to run, with arguments. This can be expressed either as a string or as a list of strings.
- shell – if True, run command in a shell, otherwise run it directly.
- raiseError – if True, raise ToolError on command failure.
Returns: exit status of executed command
Raises: ToolError
-
pygcam.utils.
simpleFormat
(s, varDict)¶ Simple version of str.format that does not treat ‘.’ as an attribute reference.
Parameters: - s – (str) string with args in curly braces
- varDict – (dict) dictionary of var names and values
Returns: (str) formatted string
-
pygcam.utils.
symlinkOrCopyFile
(src, dst)¶ Symlink a file unless GCAM.CopyAllFiles is True, in which case, copy the file.
Parameters: - src – (str) filename of original file
- dst – (dst) filename of copy
Returns: none
-
pygcam.utils.
systemOpenFile
(path)¶ Ask the operating system to open a file at the given pathname.
Parameters: path – (str) the pathname of a file to open Returns: none
-
pygcam.utils.
writeXmldbDriverProperties
(outputDir='.', inMemory=True, filterFile='', batchFile='', batchLog='')¶ Write a XMLDBDriver.properties file using the values passed in the arguments.
Parameters: - outputDir – (str) where to write the file
- inMemory – (bool) if True, the
in-memory
attribute is set to True - filterFile – (str) a file that filters GCAM query output to limit what’s written to the database
- batchFile – (str) the path to an XML batch query file
- batchLog – (str) the path to a log file into which to direct batch query messages (queries can be pretty verbose…)
Returns: none