.. _python-api: .. meta:: :description: CFG configuration format Python API :keywords: CFG, configuration, Python The CFG API for Python ---------------------- If you haven't used CFG from Python before, you can skip straight to :ref:`python-getting-started`. Backwards-Incompatible Changes ++++++++++++++++++++++++++++++ The original implementation was in 2008. The latest version of that implementation is `0.4.2 `_, released in May 2019. A new implementation was started in 2018 (with some changes to the format) and this differs from the earlier implementation in a number of ways: Format ~~~~~~ * The format now uses ``true``, ``false`` and ``null`` rather than ``True``, ``False`` and ``None``. This is for JSON compatibility. * The format now uses ``${A.B.C}`` for references rather than ``$A.B.C``. This is to allow better expressivity in paths, and to allow interpolation functionality to be implemented. * Multiple strings following one another are concatenated into a single string. Code ~~~~ * There is no support for writing configurations through the API, only for reading them. * :mod:`config` is now a package rather than a module. * The classes ``ConfigInputStream``, ``ConfigOutputStream``, ``ConfigList``, ``ConfigMerger``, ``ConfigReader``, ``Container``, ``Expression``, ``Mapping``, ``Namespace``, ``Reference``, ``SeqIter`` and ``Sequence`` are not in the new implementation. * The ``ConfigResolutionError`` exception is not in the new implementation. * The ``Config`` class in the new implementation is completely different. * The functions ``defaultMergeResolve``, ``defaultStreamOpener``, ``isWord``, ``makePath`` and ``overwriteMergeResolve`` are not in the new implementation. If your code relies on specific features of the old implementation, be sure to specify ``config<0.5`` in your dependencies. Modules +++++++ The Python implementation is divided into three modules: * ``config`` contains the high-level API which you will normally interact with. * ``config.tokens`` contains code pertaining to tokens and lexical scanning of CFG. * ``config.parser`` contains code pertaining to parsing CFG and returning Abstract Syntax Trees (ASTs). Installation ++++++++++++ You can install this updated version of the library using .. code-block:: shell $ pip install config >= 0.5.0 You should install into a virtual environment. There's a minimal example of a program that uses CFG `here `_. .. _python-getting-started: Getting Started with CFG in Python ++++++++++++++++++++++++++++++++++ .. include:: api-snip-01.inc .. include:: test0.cfg.txt Loading a configuration ~~~~~~~~~~~~~~~~~~~~~~~ The configuration above can be loaded like this: .. code-block:: pycon >>> import io, os, sys, config >>> cfg = config.Config('test0.cfg') This will assume the file is encoded using UTF-8. If that's not the case, you can pass an `encoding=` keyword parameter with the desired encoding. Access elements with keys ~~~~~~~~~~~~~~~~~~~~~~~~~ Accessing elements of the configuration with a simple key is just like using a dictionary: .. code-block:: pycon >>> cfg['a'] 'Hello, ' >>> cfg['b'] 'world!' Access elements with paths ~~~~~~~~~~~~~~~~~~~~~~~~~~ As well as simple keys, elements can also be accessed using :term:`path` strings: .. code-block:: pycon >>> cfg['c.d'] 'e' .. include:: api-snip-02.inc .. code-block:: pycon >>> cfg['f.g'] 'h' .. include:: api-snip-03.inc Access to ``datetime`` objects ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You can also get native Python ``datetime`` objects from a configuration, by using an ISO date/time pattern in a :term:`backtick-string`: .. code-block:: pycon >>> cfg['christmas_morning'] # using `2019-12-25 08:39:49` datetime.datetime(2019, 12, 25, 8, 39, 49) Access to other Python objects ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Access to other Python objects is also possible using the :term:`backtick-string` syntax, provided that they are either environment values or objects contained within importable modules: .. code-block:: pycon >>> cfg['error'] # using `sys.stderr` <_io.TextIOWrapper name='' mode='w' encoding='UTF-8'> >>> cfg['error'] is sys.stderr # Is it the exact same object? True >>> cfg['output'] # using `sys:stdout` <_io.TextIOWrapper name='' mode='w' encoding='UTF-8'> >>> cfg['output'] is sys.stdout # Is it the exact same object? True .. note:: The ```sys.stderr``` form is only for backward compatibility. You should use the ```sys:stderr``` form, which will be faster -- the all-dots form will try repeated imports to see where the split is between the module and some object within it. Note that modules themselves can be accessed, just like any other object: .. code-block:: pycon >>> cfg['module'] # using `logging.handlers:` Access to environment variables ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To access an environment variable, use a :term:`backtick-string` of the form ```$VARNAME```: .. code-block:: pycon >>> cfg['home'] == os.path.expanduser('~') # using `$HOME` True .. include:: api-snip-07.inc Access to computed values ~~~~~~~~~~~~~~~~~~~~~~~~~ .. include:: api-snip-04.inc .. code-block:: cfg-body :caption: test0a.cfg total_period : 100 header_time: 0.3 * ${total_period} steady_time: 0.5 * ${total_period} trailer_time: 0.2 * ${total_period} base_prefix: '/my/app/' log_file: ${base_prefix} + 'test.log' When this file is read in, the computed values can be accessed directly: .. code-block:: pycon >>> cfg = config.Config('test0a.cfg') ... >>> cfg['header_time'] 30.0 >>> cfg['steady_time'] 50.0 >>> cfg['trailer_time'] 20.0 >>> cfg['log_file'] '/my/app/test.log' Including one configuration inside another ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ There are times when it's useful to include one configuration inside another. For example, consider the following configuration files: .. code-block:: cfg-body :caption: logging.cfg version: 1, disable_existing_loggers: false, formatters: { brief: { class: 'logging.Formatter', format: '%(name)20.20s %(message)s' } }, handlers: { file: { level: 'INFO', class: `logging:FileHandler`, formatter: 'brief', filename: 'run/server.log', mode: 'w', encoding: 'utf-8' }, error: { level: 'ERROR', class: `logging:FileHandler`, formatter: 'brief', filename: 'run/server-errors.log', mode: 'w', encoding: 'utf-8' }, debug: { level: 'DEBUG', class: `logging:FileHandler`, formatter: 'brief', filename: 'run/server-debug.log', mode: 'w', encoding: 'utf-8' } }, loggers: { mylib: { level: 'INFO' } 'mylib.detail': { level: 'DEBUG' } }, root: { handlers: ['file', 'error', 'debug'], level: 'WARNING' } .. code-block:: cfg-body :caption: redirects.cfg cookies: { url: 'http://www.allaboutcookies.org/', permanent: false }, freeotp: { url: 'https://freeotp.github.io/', permanent: false }, 'google-auth': { url: 'https://play.google.com/store/apps/details?id=com.google.android.apps.authenticator2', permanent: false } .. code-block:: cfg-body :caption: main.cfg secret: 'some random application secret', port: 8000, server: 'cheroot', sitename: 'My Test Site', default_access: 'public', ignore_trailing_slashes: true, site_options: { want_ipinfo: false, show_form: true, cookie_bar: true }, connection: 'postgres+pool://db_user:db_pwd@localhost:5432/db_name', debug: true, captcha_length: 4, captcha_timeout: 5, session_timeout: 7 * 24 * 60 * 60, # 7 days in seconds redirects: @'redirects.cfg', email: { sender: 'no-reply@my-domain.com', host: 'smtp.my-domain.com:587', user: 'smtp-user', password: 'smtp-pwd' } logging: @'logging.cfg' .. include:: api-snip-05.inc .. code-block:: pycon >>> import config >>> cfg = config.Config('main.cfg') ... >>> cfg['redirects.freeotp.url'] 'https://freeotp.github.io/' >>> cfg['redirects.freeotp.permanent'] False >>> cfg['logging.root.level'] 'WARNING' >>> cfg['logging.handlers.file.class'] Avoiding unnecessary repetition ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. include:: api-snip-08.inc .. code-block:: cfg-body :caption: logging.cfg (partial) handlers: { file: { level: 'INFO', class: `logging:FileHandler`, formatter: 'brief', filename: 'run/server.log', mode: 'w', encoding: 'utf-8' }, error: { level: 'ERROR', class: `logging:FileHandler`, formatter: 'brief', filename: 'run/server-errors.log', mode: 'w', encoding: 'utf-8' }, debug: { level: 'DEBUG', class: `logging:FileHandler`, formatter: 'brief', filename: 'run/server-debug.log', mode: 'w', encoding: 'utf-8' } }, This portion could be rewritten as: .. code-block:: cfg-body :caption: logging.cfg (partial) defs: { base_file_handler: { class: `logging:FileHandler`, formatter: 'brief', mode: 'w', encoding: 'utf-8' } }, handlers: { file: ${defs.base_file_handler} + { level: 'INFO', filename: 'run/server.log', mode: 'a', }, error: ${defs.base_file_handler} + { level: 'ERROR', filename: 'run/server-errors.log', }, debug: ${defs.base_file_handler} + { level: 'DEBUG', filename: 'run/server-debug.log', } }, .. include:: api-snip-06.inc .. code-block:: pycon >>> import config >>> cfg = config.Config('main.cfg') ... >>> cfg['logging.handlers.file.class'] >>> cfg['logging.handlers.file.level'] 'INFO' >>> cfg['logging.handlers.file.formatter'] 'brief' >>> cfg['logging.handlers.file.encoding'] 'utf-8' >>> cfg['logging.handlers.file.mode'] 'a' >>> cfg['logging.handlers.file.filename'] 'run/server.log' >>> cfg['logging.handlers.error.mode'] 'w' >>> cfg['logging.handlers.error.filename'] 'run/server-errors.log' The definition of ``logging.handlers.file`` as ``${defs.base_file_handler} + { level: 'INFO', filename: 'run/server.log', mode: 'a' }`` has resulted in an evaluation which first fetches the ``defs.base_file_handler`` value, which is a mapping, and "adds" to that the literal mapping which defines the ``level``, ``filename`` and ``mode`` keys. The ``+`` operation for mappings is implemented as a copy of the left-hand side merged with the right-hand side. Note that the ``mode`` value for ``logging.handlers.file`` is overridden by the right-hand side to ``'a'``, whereas that for e.g. ``logging.handlers.error`` is unchanged as ``'w'``. We could do some further refinement by factoring out the common location for the log files: .. code-block:: cfg-body :caption: logging.cfg (partial) defs: { base_file_handler: { class: `logging:FileHandler`, formatter: 'brief', mode: 'w', encoding: 'utf-8' } log_prefix: 'run/', }, handlers: { file: ${defs.base_file_handler} + { level: 'INFO', filename: ${defs.log_prefix} + 'server.log', mode: 'a', }, error: ${defs.base_file_handler} + { level: 'ERROR', filename: ${defs.log_prefix} + 'server-errors.log', }, debug: ${defs.base_file_handler} + { level: 'DEBUG', filename: ${defs.log_prefix} + 'server-debug.log', } } with the same result as before. It *is* slightly more verbose than before, but the location of all files can be changed in just one place now, as opposed to three, as it was before. A larger example - Django configuration ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Let's look at a slightly larger example: .. include:: log1.cfg.txt This is the analogue of the configuration dictionary for the "fairly complex logging setup" documented in `Django 2.2 `__. Let's assume we have both Django and the following on Python's ``sys.path``: .. code-block:: python3 :caption: project/logging.py import logging class SpecialFilter(logging.Filter): def __init__(self, foo): self.foo = foo def filter(self, record): return True We can load the configuration easily enough: .. code-block:: pycon >>> import config, logging.config >>> cfg = config.Config('log1.cfg') ... >>> logging.config.dictConfig(cfg.as_dict()) >>> logging.getLogger('django').handlers [ (INFO)>] >>> logging.getLogger('django.request').handlers [] >>> logging.getLogger('myproject.custom').handlers [ (INFO)>, ] Classes +++++++ The ``Location`` class ~~~~~~~~~~~~~~~~~~~~~~ This represents a source location and has two integer attributes, ``line`` and ``column``. The line must be positive and the column non-negative (newlines have an ending column of zero, as column 1 would have the first character of the next line). The ``ConfigError`` class ~~~~~~~~~~~~~~~~~~~~~~~~~ This is an ``Exception`` subclass which has an error message in the ``message`` attribute and a ``Location`` instance in the ``location`` attribute indicating the position in the source of the error, where appropriate. (Some errors don't have a location, and in such cases the ``location`` attribute of the instance will be set to ``None``.) The ``Config`` class ~~~~~~~~~~~~~~~~~~~~ This class implements access to a CFG configuration. You'll generally interface to CFG files using this class. When you pass in a stream or file path to the constructor the CFG source in the stream or file is parsed and converted into an internal form which can then be accessed in a manner analogous to a Python dictionary. .. index:: single: Config; Python class .. class:: Config .. cssclass:: class-members-heading Attributes .. attribute:: path The path to the file from which the configuration has been read. You won't usually need to set this, unless you want to load a configuration from text that references included files. In that case, set the path to a value whose parent directory contains the included files, or specify relevant directories using :attr:`include_path`. When a configuration is loaded from a file, this attribute is automatically set. .. attribute:: include_path A list of directories which is searched for included sub-configurations if the parent directory of :attr:`path` (or the current directory, if `path` isn't set) doesn't contain them. .. attribute:: context A mapping of strings to objects which is used whenever an identifier is encountered when evaluating an expression in a configuration file. .. attribute:: no_duplicates Whether keys are allowed to be duplicated in mappings. This defaults to ``True`` if not provided -- a :class:`ConfigError` is raised if a duplicate key is seen. If ``False``, then if a duplicate key is seen, its value silently replaces the value associated with the earlier appearance of the same key. .. attribute:: strict_conversions If ``True``, :class:`ConfigError` is raised if a :term:`backtick-string` can't be converted. This defaults to ``True`` if not provided. It's intended to help catch typos in backtick-strings. .. cssclass:: class-members-heading Methods .. index:: single: __init__; Config class .. method:: __init__(stream_or_path, **kwargs) :noindex: Returns a new instance. :param stream_or_path: Either a stream containing the CFG source to be read (which could be an instance of ``io.StringIO`` or a disk file opened in text mode with an appropriate encoding), or the path to a file containing CFG source. It can also be ``None``, in which case a stream needs to be passed to the :meth:`load` method or a file path to the :meth:`load_file` method in order to load a configuration into the instance. :param kwargs: Keyword parameters, see below. :kwarg cache: If set to ``True``, a cache dictionary is initialised and used to cache retrieved values. If not provided, it defaults to ``False``. :kwarg context: A dictionary which is used to look up variables. If not provided, an empty dictionary is used. :kwarg include_path: A list of directories to be searched for included configurations when an include expression is seen. If not provided, an empty list is used. Note that the parent directory of the ``path`` attribute (or the current directory if that isn't set) is always searched for included configurations, before the ``include_path`` is consulted. :kwarg no_duplicates: Whether keys are allowed to be duplicated in mappings. This defaults to ``True`` if not provided -- a :class:`ConfigError` is raised if a duplicate key is seen. If ``False``, then if a duplicate key is seen, its value silently replaces the value associated with the earlier appearance of the same key. :kwarg strict_conversions: If ``True``, back-tick string conversion raises a :class:`ConfigError` if a string can't be converted. This defaults to ``True`` if not provided. It's intended to help catch typos in backtick-strings. :kwarg path: If provided, this will be used to set the :attr:`path` attribute of the instance. You don't need to provide this if `stream_or_path` is a path. .. method:: load(stream) Loads CFG source into this instance. :param stream: A stream containing the CFG source to be read. This could be an instance of ``io.StringIO`` or a disk file opened in text mode with an appropriate encoding. .. cssclass:: mt10 The source must be for a :term:`mapping body` or a :term:`mapping`. All top-level configurations must be of this type, though included sub-configurations could also take the form of a list. .. method:: close() Closes this instance, after which no other methods should be called on it. If :attr:`can_close` is ``True``, the :attr:`stream` will be closed. .. method:: as_dict() Return's this instance's data as a Python dictionary. .. method:: get(key, default_value) :param str key: A key or path into this configuration. :param default_value: A value to return if ``key`` cannot be found. Get an element from this instance using `key`, and returning the corresponding value or `default_value` if the key isn't present. The `key` can either be an actual key or a path. Note that normal indexed access ``cfg[key]`` is also possible, and in this case, if the key or path are not found in the configuration, a ``ConfigError`` is raised. A ``ConfigError`` will also be raised if a path is invalid in some way.