.. _python-api:
.. meta::
:description: CFG configuration format Python API
:keywords: CFG, configuration, Python
The CFG API for Python
----------------------
If you haven't used CFG from Python before, you can skip straight to
:ref:`python-getting-started`.
Backwards-Incompatible Changes
++++++++++++++++++++++++++++++
The original implementation was in 2008. The latest version of that
implementation is `0.4.2 `_, released
in May 2019. A new implementation was started in 2018 (with some changes to
the format) and this differs from the earlier implementation in a number of
ways:
Format
~~~~~~
* The format now uses ``true``, ``false`` and ``null`` rather than ``True``,
``False`` and ``None``. This is for JSON compatibility.
* The format now uses ``${A.B.C}`` for references rather than ``$A.B.C``. This
is to allow better expressivity in paths, and to allow interpolation functionality
to be implemented.
* Multiple strings following one another are concatenated into a single string.
Code
~~~~
* There is no support for writing configurations through the API, only for
reading them.
* :mod:`config` is now a package rather than a module.
* The classes ``ConfigInputStream``, ``ConfigOutputStream``, ``ConfigList``,
``ConfigMerger``, ``ConfigReader``, ``Container``, ``Expression``,
``Mapping``, ``Namespace``, ``Reference``, ``SeqIter`` and ``Sequence`` are
not in the new implementation.
* The ``ConfigResolutionError`` exception is not in the new implementation.
* The ``Config`` class in the new implementation is completely different.
* The functions ``defaultMergeResolve``, ``defaultStreamOpener``, ``isWord``,
``makePath`` and ``overwriteMergeResolve`` are not in the new implementation.
If your code relies on specific features of the old implementation, be sure to
specify ``config<0.5`` in your dependencies.
Modules
+++++++
The Python implementation is divided into three modules:
* ``config`` contains the high-level API which you will normally
interact with.
* ``config.tokens`` contains code pertaining to tokens and lexical
scanning of CFG.
* ``config.parser`` contains code pertaining to parsing CFG and returning
Abstract Syntax Trees (ASTs).
Installation
++++++++++++
You can install this updated version of the library using
.. code-block:: shell
$ pip install config >= 0.5.0
You should install into a virtual environment.
There's a minimal example of a program that uses CFG `here
`_.
.. _python-getting-started:
Getting Started with CFG in Python
++++++++++++++++++++++++++++++++++
.. include:: api-snip-01.inc
.. include:: test0.cfg.txt
Loading a configuration
~~~~~~~~~~~~~~~~~~~~~~~
The configuration above can be loaded like this:
.. code-block:: pycon
>>> import io, os, sys, config
>>> cfg = config.Config('test0.cfg')
This will assume the file is encoded using UTF-8. If that's not the case, you
can pass an `encoding=` keyword parameter with the desired encoding.
Access elements with keys
~~~~~~~~~~~~~~~~~~~~~~~~~
Accessing elements of the configuration with a simple key is just like using a
dictionary:
.. code-block:: pycon
>>> cfg['a']
'Hello, '
>>> cfg['b']
'world!'
Access elements with paths
~~~~~~~~~~~~~~~~~~~~~~~~~~
As well as simple keys, elements can also be accessed using :term:`path`
strings:
.. code-block:: pycon
>>> cfg['c.d']
'e'
.. include:: api-snip-02.inc
.. code-block:: pycon
>>> cfg['f.g']
'h'
.. include:: api-snip-03.inc
Access to ``datetime`` objects
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can also get native Python ``datetime`` objects from a configuration, by
using an ISO date/time pattern in a :term:`backtick-string`:
.. code-block:: pycon
>>> cfg['christmas_morning'] # using `2019-12-25 08:39:49`
datetime.datetime(2019, 12, 25, 8, 39, 49)
Access to other Python objects
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Access to other Python objects is also possible using the
:term:`backtick-string` syntax, provided that they are either environment
values or objects contained within importable modules:
.. code-block:: pycon
>>> cfg['error'] # using `sys.stderr`
<_io.TextIOWrapper name='' mode='w' encoding='UTF-8'>
>>> cfg['error'] is sys.stderr # Is it the exact same object?
True
>>> cfg['output'] # using `sys:stdout`
<_io.TextIOWrapper name='' mode='w' encoding='UTF-8'>
>>> cfg['output'] is sys.stdout # Is it the exact same object?
True
.. note::
The ```sys.stderr``` form is only for backward compatibility. You should
use the ```sys:stderr``` form, which will be faster -- the all-dots form
will try repeated imports to see where the split is between the module and
some object within it.
Note that modules themselves can be accessed, just like any other object:
.. code-block:: pycon
>>> cfg['module'] # using `logging.handlers:`
Access to environment variables
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To access an environment variable, use a :term:`backtick-string` of the form
```$VARNAME```:
.. code-block:: pycon
>>> cfg['home'] == os.path.expanduser('~') # using `$HOME`
True
.. include:: api-snip-07.inc
Access to computed values
~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: api-snip-04.inc
.. code-block:: cfg-body
:caption: test0a.cfg
total_period : 100
header_time: 0.3 * ${total_period}
steady_time: 0.5 * ${total_period}
trailer_time: 0.2 * ${total_period}
base_prefix: '/my/app/'
log_file: ${base_prefix} + 'test.log'
When this file is read in, the computed values can be accessed directly:
.. code-block:: pycon
>>> cfg = config.Config('test0a.cfg')
...
>>> cfg['header_time']
30.0
>>> cfg['steady_time']
50.0
>>> cfg['trailer_time']
20.0
>>> cfg['log_file']
'/my/app/test.log'
Including one configuration inside another
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
There are times when it's useful to include one configuration inside another.
For example, consider the following configuration files:
.. code-block:: cfg-body
:caption: logging.cfg
version: 1,
disable_existing_loggers: false,
formatters: {
brief: {
class: 'logging.Formatter',
format: '%(name)20.20s %(message)s'
}
},
handlers: {
file: {
level: 'INFO',
class: `logging:FileHandler`,
formatter: 'brief',
filename: 'run/server.log',
mode: 'w',
encoding: 'utf-8'
},
error: {
level: 'ERROR',
class: `logging:FileHandler`,
formatter: 'brief',
filename: 'run/server-errors.log',
mode: 'w',
encoding: 'utf-8'
},
debug: {
level: 'DEBUG',
class: `logging:FileHandler`,
formatter: 'brief',
filename: 'run/server-debug.log',
mode: 'w',
encoding: 'utf-8'
}
},
loggers: {
mylib: {
level: 'INFO'
}
'mylib.detail': {
level: 'DEBUG'
}
},
root: {
handlers: ['file', 'error', 'debug'],
level: 'WARNING'
}
.. code-block:: cfg-body
:caption: redirects.cfg
cookies: {
url: 'http://www.allaboutcookies.org/',
permanent: false
},
freeotp: {
url: 'https://freeotp.github.io/',
permanent: false
},
'google-auth': {
url: 'https://play.google.com/store/apps/details?id=com.google.android.apps.authenticator2',
permanent: false
}
.. code-block:: cfg-body
:caption: main.cfg
secret: 'some random application secret',
port: 8000,
server: 'cheroot',
sitename: 'My Test Site',
default_access: 'public',
ignore_trailing_slashes: true,
site_options: {
want_ipinfo: false,
show_form: true,
cookie_bar: true
},
connection: 'postgres+pool://db_user:db_pwd@localhost:5432/db_name',
debug: true,
captcha_length: 4,
captcha_timeout: 5,
session_timeout: 7 * 24 * 60 * 60, # 7 days in seconds
redirects: @'redirects.cfg',
email: {
sender: 'no-reply@my-domain.com',
host: 'smtp.my-domain.com:587',
user: 'smtp-user',
password: 'smtp-pwd'
}
logging: @'logging.cfg'
.. include:: api-snip-05.inc
.. code-block:: pycon
>>> import config
>>> cfg = config.Config('main.cfg')
...
>>> cfg['redirects.freeotp.url']
'https://freeotp.github.io/'
>>> cfg['redirects.freeotp.permanent']
False
>>> cfg['logging.root.level']
'WARNING'
>>> cfg['logging.handlers.file.class']
Avoiding unnecessary repetition
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: api-snip-08.inc
.. code-block:: cfg-body
:caption: logging.cfg (partial)
handlers: {
file: {
level: 'INFO',
class: `logging:FileHandler`,
formatter: 'brief',
filename: 'run/server.log',
mode: 'w',
encoding: 'utf-8'
},
error: {
level: 'ERROR',
class: `logging:FileHandler`,
formatter: 'brief',
filename: 'run/server-errors.log',
mode: 'w',
encoding: 'utf-8'
},
debug: {
level: 'DEBUG',
class: `logging:FileHandler`,
formatter: 'brief',
filename: 'run/server-debug.log',
mode: 'w',
encoding: 'utf-8'
}
},
This portion could be rewritten as:
.. code-block:: cfg-body
:caption: logging.cfg (partial)
defs: {
base_file_handler: {
class: `logging:FileHandler`,
formatter: 'brief',
mode: 'w',
encoding: 'utf-8'
}
},
handlers: {
file: ${defs.base_file_handler} + {
level: 'INFO',
filename: 'run/server.log',
mode: 'a',
},
error: ${defs.base_file_handler} + {
level: 'ERROR',
filename: 'run/server-errors.log',
},
debug: ${defs.base_file_handler} + {
level: 'DEBUG',
filename: 'run/server-debug.log',
}
},
.. include:: api-snip-06.inc
.. code-block:: pycon
>>> import config
>>> cfg = config.Config('main.cfg')
...
>>> cfg['logging.handlers.file.class']
>>> cfg['logging.handlers.file.level']
'INFO'
>>> cfg['logging.handlers.file.formatter']
'brief'
>>> cfg['logging.handlers.file.encoding']
'utf-8'
>>> cfg['logging.handlers.file.mode']
'a'
>>> cfg['logging.handlers.file.filename']
'run/server.log'
>>> cfg['logging.handlers.error.mode']
'w'
>>> cfg['logging.handlers.error.filename']
'run/server-errors.log'
The definition of ``logging.handlers.file`` as ``${defs.base_file_handler} + {
level: 'INFO', filename: 'run/server.log', mode: 'a' }`` has resulted in an
evaluation which first fetches the ``defs.base_file_handler`` value, which is
a mapping, and "adds" to that the literal mapping which defines the ``level``,
``filename`` and ``mode`` keys. The ``+`` operation for mappings is
implemented as a copy of the left-hand side merged with the right-hand side.
Note that the ``mode`` value for ``logging.handlers.file`` is overridden by
the right-hand side to ``'a'``, whereas that for e.g.
``logging.handlers.error`` is unchanged as ``'w'``.
We could do some further refinement by factoring out the common location for
the log files:
.. code-block:: cfg-body
:caption: logging.cfg (partial)
defs: {
base_file_handler: {
class: `logging:FileHandler`,
formatter: 'brief',
mode: 'w',
encoding: 'utf-8'
}
log_prefix: 'run/',
},
handlers: {
file: ${defs.base_file_handler} + {
level: 'INFO',
filename: ${defs.log_prefix} + 'server.log',
mode: 'a',
},
error: ${defs.base_file_handler} + {
level: 'ERROR',
filename: ${defs.log_prefix} + 'server-errors.log',
},
debug: ${defs.base_file_handler} + {
level: 'DEBUG',
filename: ${defs.log_prefix} + 'server-debug.log',
}
}
with the same result as before. It *is* slightly more verbose than before, but
the location of all files can be changed in just one place now, as opposed to
three, as it was before.
A larger example - Django configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Let's look at a slightly larger example:
.. include:: log1.cfg.txt
This is the analogue of the configuration dictionary for the "fairly complex
logging setup" documented in `Django 2.2
`__.
Let's assume we have both Django and the following on Python's ``sys.path``:
.. code-block:: python3
:caption: project/logging.py
import logging
class SpecialFilter(logging.Filter):
def __init__(self, foo):
self.foo = foo
def filter(self, record):
return True
We can load the configuration easily enough:
.. code-block:: pycon
>>> import config, logging.config
>>> cfg = config.Config('log1.cfg')
...
>>> logging.config.dictConfig(cfg.as_dict())
>>> logging.getLogger('django').handlers
[ (INFO)>]
>>> logging.getLogger('django.request').handlers
[]
>>> logging.getLogger('myproject.custom').handlers
[ (INFO)>, ]
Classes
+++++++
The ``Location`` class
~~~~~~~~~~~~~~~~~~~~~~
This represents a source location and has two integer attributes, ``line`` and
``column``. The line must be positive and the column non-negative (newlines have an
ending column of zero, as column 1 would have the first character of the next line).
The ``ConfigError`` class
~~~~~~~~~~~~~~~~~~~~~~~~~
This is an ``Exception`` subclass which has an error message in the ``message``
attribute and a ``Location`` instance in the ``location`` attribute indicating the
position in the source of the error, where appropriate. (Some errors don't have a
location, and in such cases the ``location`` attribute of the instance will be set to
``None``.)
The ``Config`` class
~~~~~~~~~~~~~~~~~~~~
This class implements access to a CFG configuration. You'll generally
interface to CFG files using this class. When you pass in a stream or file
path to the constructor the CFG source in the stream or file is parsed and
converted into an internal form which can then be accessed in a manner
analogous to a Python dictionary.
.. index::
single: Config; Python class
.. class:: Config
.. cssclass:: class-members-heading
Attributes
.. attribute:: path
The path to the file from which the configuration has been read. You won't
usually need to set this, unless you want to load a configuration from text that
references included files. In that case, set the path to a value whose parent
directory contains the included files, or specify relevant directories using
:attr:`include_path`. When a configuration is loaded from a file, this attribute
is automatically set.
.. attribute:: include_path
A list of directories which is searched for included sub-configurations if the
parent directory of :attr:`path` (or the current directory, if `path` isn't set)
doesn't contain them.
.. attribute:: context
A mapping of strings to objects which is used whenever an identifier
is encountered when evaluating an expression in a configuration file.
.. attribute:: no_duplicates
Whether keys are allowed to be duplicated in mappings. This defaults to
``True`` if not provided -- a :class:`ConfigError` is raised if a
duplicate key is seen. If ``False``, then if a duplicate key is seen,
its value silently replaces the value associated with the earlier
appearance of the same key.
.. attribute:: strict_conversions
If ``True``, :class:`ConfigError` is raised if a :term:`backtick-string` can't
be converted. This defaults to ``True`` if not provided. It's intended to help
catch typos in backtick-strings.
.. cssclass:: class-members-heading
Methods
.. index::
single: __init__; Config class
.. method:: __init__(stream_or_path, **kwargs)
:noindex:
Returns a new instance.
:param stream_or_path: Either a stream containing the CFG source to be
read (which could be an instance of
``io.StringIO`` or a disk file opened in text
mode with an appropriate encoding), or the path
to a file containing CFG source. It can also be
``None``, in which case a stream needs to be
passed to the :meth:`load` method or a file path
to the :meth:`load_file` method in order to load
a configuration into the instance.
:param kwargs: Keyword parameters, see below.
:kwarg cache: If set to ``True``, a cache dictionary is initialised and
used to cache retrieved values. If not provided, it
defaults to ``False``.
:kwarg context: A dictionary which is used to look up variables. If not
provided, an empty dictionary is used.
:kwarg include_path: A list of directories to be searched for included
configurations when an include expression is seen. If not
provided, an empty list is used. Note that the parent
directory of the ``path`` attribute (or the current
directory if that isn't set) is always searched for
included configurations, before the ``include_path`` is
consulted.
:kwarg no_duplicates: Whether keys are allowed to be duplicated in
mappings. This defaults to ``True`` if not
provided -- a :class:`ConfigError` is raised if a
duplicate key is seen. If ``False``, then if a
duplicate key is seen, its value silently replaces
the value associated with the earlier appearance of
the same key.
:kwarg strict_conversions: If ``True``, back-tick string conversion raises a
:class:`ConfigError` if a string can't be
converted. This defaults to ``True`` if not
provided. It's intended to help catch typos in
backtick-strings.
:kwarg path: If provided, this will be used to set the :attr:`path` attribute of
the instance. You don't need to provide this if `stream_or_path` is
a path.
.. method:: load(stream)
Loads CFG source into this instance.
:param stream: A stream containing the CFG source to be read. This could be
an instance of ``io.StringIO`` or a disk file opened in
text mode with an appropriate encoding.
.. cssclass:: mt10
The source must be for a :term:`mapping body` or a
:term:`mapping`. All top-level configurations must be of
this type, though included sub-configurations could also
take the form of a list.
.. method:: close()
Closes this instance, after which no other methods should be called on it.
If :attr:`can_close` is ``True``, the :attr:`stream` will be closed.
.. method:: as_dict()
Return's this instance's data as a Python dictionary.
.. method:: get(key, default_value)
:param str key: A key or path into this configuration.
:param default_value: A value to return if ``key`` cannot be found.
Get an element from this instance using `key`, and returning the corresponding
value or `default_value` if the key isn't present. The `key` can either be an
actual key or a path.
Note that normal indexed access ``cfg[key]`` is also possible, and in this case,
if the key or path are not found in the configuration, a ``ConfigError`` is
raised.
A ``ConfigError`` will also be raised if a path is invalid in some way.