The CFG API for Elixir¶

The CFG reference implementation for the Elixir language assumes an Elixir version of 1.9.1 or later.

Installation¶

You can install the library for use by adding cfg_lib to your list of dependencies in mix.exs:

def deps do
  [
    {:cfg_lib, "~> 0.1.0"}
  ]
end

There’s a minimal example of a program that uses CFG here.

Exploration¶

To explore CFG functionality for Elixir, we use the iex program, which is a Read-Eval-Print-Loop (REPL). You can invoke a shell using

$ iex -S mix

This loads all the dependencies into the shell.

Getting Started with CFG in Elixir¶

A configuration is accessed through the CFG.Config module. This has functions which can be passed a filename or a string which contains the text for the configuration. The text is read in, parsed and converted to an object that you can then query. A simple example:

test0.cfg¶

a: 'Hello, '
b: 'world!'
c: {
  d: 'e'
}
'f.g': 'h'
christmas_morning: `2019-12-25 08:39:49`
now: `Elixir.DateTime:utc_now`
home: `$HOME`
foo: `$FOO|bar`

Loading a configuration¶

The configuration above can be loaded as shown below. In the REPL shell:

iex(1)> alias CFG.Config
CFG.Config
iex(2)> {:ok, cfg} = Config.from_file("test0.cfg")
{:ok, #PID<0.218.0>}

Access elements with keys¶

Accessing elements of the configuration with a simple key is just like using a Hash:

iex(3)> Config.get(cfg, "a")
{:ok, "Hello, "}
iex(4)> Config.get(cfg, "b")
{:ok, "world!"}

Access elements with paths¶

As well as simple keys, elements can also be accessed using path strings:

iex(5)> Config.get(cfg, "c.d")
{:ok, "e"}

Here, the desired value is obtained in a single step, by (under the hood) walking the path c.d – first getting the mapping at key c, and then the value at d in the resulting mapping.

Note that you can have simple keys which look like paths:

iex(6)> Config.get(cfg, "f.g")
{:ok, "h"}

If a key is given that exists in the configuration, it is used as such, and if it is not present in the configuration, an attempt is made to interpret it as a path. Thus, f.g is present and accessed via key, whereas c.d is not an existing key, so is interpreted as a path.

Access to date/time objects¶

You can also get native Elixir date/time objects from a configuration, by using an ISO date/time pattern in a backtick-string:

iex(7)> Config.get(cfg, "christmas_morning")
{:ok, ~U[2019-12-25 08:39:49.000000Z]}

Access to other Elixir objects¶

Access to other Elixir objects is also possible using the backtick-string syntax, provided that they are one of:

Environment variables
Public functions in Erlang or Elixir

iex(8)> {:ok, dt} = Config.get(cfg, "now")
{:ok, ~U[2021-10-16 12:37:37.781391Z]}
iex(9)> DateTime.diff(DateTime.utc_now, dt)
6

Accessing the “now” element of the above configuration invokes the DateTime.utc_now function and returns its value. You can see from the subtraction that the times are practically equivalent (the difference is the time between the two calls to DateTime.utc_now - once when computing the configured value, and once in the interactive session.

Access to environment variables¶

To access an environment variable, use a backtick-string of the form `$VARNAME`:

iex(10)> elem(Config.get(cfg, "home"), 1) == System.get_env("HOME")
true

You can specify a default value to be used if an environment variable isn’t present using the `$VARNAME|default-value` form. Whatever string follows the pipe character (including the empty string) is returned if VARNAME is not a variable in the environment.

iex(11)> Config.get(cfg, "foo")
{:ok, "bar"}

Access to computed values¶

Sometimes, it’s useful to have values computed declaratively in the configuration, rather than imperatively in the code that processes the configuration. For example, an overall time period may be specified and other configuration values are fractions thereof. It may also be desirable to perform other simple calculations declaratively, e.g. concatenation of numerous file names to a base directory to get a final pathname.

test0a.cfg¶

total_period : 100
header_time: 0.3 * ${total_period}
steady_time: 0.5 * ${total_period}
trailer_time: 0.2 * ${total_period}
base_prefix: '/my/app/'
log_file: ${base_prefix} + 'test.log'

When this file is read in, the computed values can be accessed directly:

iex(12)> {:ok, cfg} = Config.from_file("test0a.cfg")
{:ok, #PID<0.218.0>}
iex(13)> Config.get(cfg, "header_time")
{:ok, 30.0}
iex(14)> Config.get(cfg, "steady_time")
{:ok, 50.0}
iex(15)> Config.get(cfg, "trailer_time")
{:ok, 20.0}
iex(16)> Config.get(cfg, "log_file")
{:ok, "/my/app/test.log"}

Including one configuration inside another¶

There are times when it’s useful to include one configuration inside another. For example, consider the following configuration files:

logging.cfg¶

layouts: {
  brief: {
    pattern: '%d [%t] %p %c - %m%n'
  }
},
appenders: {
  file: {
    level: 'INFO',
    layout: 'brief',
    filename: 'run/server.log',
    append: false,
    charset: 'UTF-8'
  },
  error: {
    level: 'ERROR',
    layout: 'brief',
    filename: 'run/server-errors.log',
    append: false,
    charset: 'UTF-8'
  },
  debug: {
    level: 'DEBUG',
    layout: 'brief',
    filename: 'run/server-debug.log',
    append: false,
    charset: 'UTF-8'
  }
},
loggers: {
  mylib: {
    level: 'INFO'
  }
  'mylib.detail': {
    level: 'DEBUG'
  }
},
root: {
  handlers: ['file', 'error', 'debug'],
  level: 'WARNING'
}

redirects.cfg¶

cookies: {
  url: 'http://www.allaboutcookies.org/',
  permanent: false
},
freeotp: {
  url: 'https://freeotp.github.io/',
  permanent: false
},
'google-auth': {
  url: 'https://play.google.com/store/apps/details?id=com.google.android.apps.authenticator2',
  permanent: false
}

main.cfg¶

secret: 'some random application secret',
port: 8000,
sitename: 'My Test Site',
default_access: 'public',
ignore_trailing_slashes: true,
site_options: {
  want_ipinfo: false,
  show_form: true,
  cookie_bar: true
},
debug: true,
captcha_length: 4,
captcha_timeout: 5,
session_timeout: 7 * 24 * 60 * 60,  # 7 days in seconds
redirects: @'redirects.cfg',
email: {
  sender: 'no-reply@my-domain.com',
  host: 'smtp.my-domain.com:587',
  user: 'smtp-user',
  password: 'smtp-pwd'
}
logging: @'logging.cfg'

The main.cfg contents have been kept to the highest-level values, within logging and redirection configuration relegated to other files logging.cfg and redirects.cfg which are then included in main.cfg. This allows the high-level configuration to be more readable at a glance, and even allows the separate configuration files to be e.g. maintained by different people.

The contents of the sub-configurations are easily accessible from the main configuration just as if they had been defined in the same file:

iex(17)> {:ok, cfg}= Config.from_file("main.cfg")
{:ok, #PID<0.218.0>}
iex(18)> Config.get(cfg, "logging.layouts.brief.pattern")
{:ok, "%d [%t] %p %c - %m%n"}
iex(19)> Config.get(cfg, "redirects.freeotp.url")
{:ok, "https://freeotp.github.io/"}
iex(20)> Config.get(cfg, "redirects.freeotp.permanent")
{:ok, false}

Avoiding unnecessary repetition¶

Don’t Repeat Yourself (DRY) is a useful principle to follow. CFG can help with this. You may have noticed that the logging.cfg file above has some repetitive elements:

logging.cfg (partial)¶

appenders: {
  file: {
    level: 'INFO',
    layout: 'brief',
    filename: 'run/server.log',
    append: true,
    charset: 'UTF-8'
  },
  error: {
    level: 'ERROR',
    layout: 'brief',
    filename: 'run/server-errors.log',
    append: false,
    charset: 'UTF-8'
  },
  debug: {
    level: 'DEBUG',
    layout: 'brief',
    filename: 'run/server-debug.log',
    append: false,
    charset: 'UTF-8'
  }
},

This portion could be rewritten as:

logging.cfg (partial)¶

defs: {
  base_appender: {
    layout: 'brief',
    append: false,
    charset: 'UTF-8'
  }
},
appenders: {
  file: ${defs.base_appender} + {
    level: 'INFO',
    filename: 'run/server.log',
    append: true,
  },
  error: ${defs.base_appender} + {
    level: 'ERROR',
    filename: 'run/server-errors.log',
  },
  debug: ${defs.base_appender} + {
    level: 'DEBUG',
    filename: 'run/server-debug.log',
  }
},

where the common elements are separated out and just referenced where they are needed. We find it useful to put all things which will be reused like this in one place in the condiguration, so we always know where to go to make changes. The key used is conventionally defs or base, though it can be anything you like.

Access is just as before, and provides the same results:

iex(21)> Config.get(cfg, "logging.appenders.file.append")
{:ok, true}
iex(22)> Config.get(cfg, "logging.appenders.file.filename")
{:ok, "run/server.log"}
iex(23)> Config.get(cfg, "logging.appenders.file.level")
{:ok, "INFO"}
iex(24)> Config.get(cfg, "logging.appenders.debug.level")
{:ok, "DEBUG"}
iex(25)> Config.get(cfg, "logging.appenders.debug.filename")
{:ok, "run/server-debug.log"}
iex(26)> Config.get(cfg, "logging.appenders.debug.append")
{:ok, false}

The definition of logging.appenders.file as ${defs.base_appender} + { level: 'INFO', filename: 'run/server.log', append: true } has resulted in an evaluation which first fetches the defs.base_appender value, which is a mapping, and “adds” to that the literal mapping which defines the level, filename and append keys. The + operation for mappings is implemented as a copy of the left-hand side merged with the right-hand side. Note that the append value for logging.appenders.file is overridden by the right-hand side to true, whereas that for e.g. logging.appenders.error is unchanged as false.

We could do some further refinement by factoring out the common location for the log files:

logging.cfg (partial)¶

defs: {
  base_appender: {
    layout: 'brief',
    append: false,
    charset: 'UTF-8'
  }
  log_prefix: 'run/',
},
appenders: {
  file: ${defs.base_appender} + {
    level: 'INFO',
    filename: ${defs.log_prefix} + 'server.log',
    append: true,
  },
  error: ${defs.base_appender} + {
    level: 'ERROR',
    filename: ${defs.log_prefix} + 'server-errors.log',
  },
  debug: ${defs.base_appender} + {
    level: 'DEBUG',
    filename: ${defs.log_prefix} + 'server-debug.log',
  }
}

with the same result as before. It is slightly more verbose than before, but the location of all files can be changed in just one place now, as opposed to three, as it was before.

Modules¶

The CFG.Config module implements code to model a CFG configuration. You’ll generally interface to CFG files using this module. When you pass in a source string or file path to the from_source / from_file functions, the CFG source in the source string or file is parsed and converted into an internal form that can be queried. The CFG.RecognizerError module represents errors returned from the API. The CFG.Location module represents a location in CFG source code.

CFG.Location¶

CFG.Location

This represents a location in CFG source.

Struct Members

line : pos_integer = 1: The source line.

column : non_negative_integer = 1: The source column. Newlines end with a zero column; the first character in the next line would be at column 1.

CFG.RecognizerError¶

CFG.RecognizerError

This represents an error which occurred when processing CFG.

Struct Members

reason : atom

An indication of what the error is. The following values are currently in use:

:invalid_escape - an invalid escape sequence was detected in a string.
:unterminated_backtick - a backtick-string is unterminated.
:newlines_not_allowed - newlines aren’t allowed in strings other than multi-line strings.
:unterminated_string - a quoted string is unterminated.
:bad_number - a number is badly formed.
:bad_octal_constant - a number which looks like an octal constant is badly formed.
:unexpected_char - an unexpected character was encountered.
:unexpected_token - an unexpected token was encountered.
:unexpected_token_for_value - an unexpected token was encountered when looking for a value.
:unexpected_token_for_atom - an unexpected token was encountered when looking for an atomic value.
:bad_key_value_separator - a bad key/value separator was encountered.
:unexpected_for_key - an unexpected token was encountered when looking for a key in a mapping.
:unexpected_token_for_container - an unexpected token was encountered when parsing a container.
:text_after_container - there is trailing text following text for a valid container.
:invalid_index - an array or slice index is invalid.
:unexpected_token_for_expression - an unexpected token was encountered when looking for an expression.
:must_be_mapping - a top-level configuration must be a mapping.
:invalid_path - a CFG path is invalid.
:invalid_path_extra - there is text following what looks like a valid CFG path.
:no_configuration - no configuration has been loaded.
:not_found - the specified key or path was not found in this configuration.
:invalid_step - an invalid step (zero) was specified.
:unexpected_path_start - a CFG path doesn’t begin as expected (with an identifier).
:cannot_evaluate - an expression cannot be evaluated.
:string_expected - a string was expected, but not found.
:include_not_found - an included configuration was not found.
:cannot_add - an addition cannot be performed.
:cannot_negate - a negation cannot be performed.
:cannot_subtract - a subtraction cannot be performed.
:cannot_multiply - a multiplication cannot be performed.
:cannot_divide - a division cannot be performed.
:cannot_integer_divide - an integer division cannot be performed.
:cannot_compute_modulo - a modulo operation cannot be performed.
:cannot_left_shift - a left shift cannot be performed.
:cannot_right_shift - a right shift cannot be performed.
:cannot_raise_to_power - raise to power operation cannot be performed.
:cannot_bitwise_or - a bitwise-or operation cannot be performed.
:cannot_bitwise_and - a bitwise-and operation cannot be performed.
:cannot_bitwise_xor - a bitwise-xor operation cannot be performed.
:unknown_variable - a variable is undefined or no context was provided.
:conversion_failure - a string conversion operation cannot be performed.
:circular_reference - a circular reference was detected when resolving references.
:not_implemented - a feature is not implemented.

location : nil | Location: The optional location of the error in the source. Some errors may have no location, such as :no_configuration.

detail : any: Optional additional information about the error. For example, an :invalid_index error would indicate the invalid index value in the detail.

CFG.Config¶

CFG.Config

This represents a single configuration (which may contain nested configurations).

Struct Members

no_duplicates : boolean = true: Whether duplicates keys are allowed when parsing a mapping or mapping body. If not allowed and duplicates are seen, a RecognizerError is returned.

strict_conversions : boolean = true: If true, conversion of backtick-strings returns an error if conversion fails for any reason. If false, the string itself is returned if it can’t be converted. It’s intended to help catch typos in backtick-strings.

context : map: A mapping within which to look up identifiers during evaluation of AST nodes.

cached : boolean = false: If true, an internal cache is used.

include_path : list(binary): A list of directories to be searched for included configurations when an include expression is seen. Note that the include_path is only needed if the included configuration file is not in the same place as the including configuration file.

path : binary: The path to the file from which this instance’s configuration has been loaded.

Functions

Where a return type of {atom, value} is given, the return value is either one of {:ok, success_value} or {:error, error_value} where the error_value is a RecognizerError. If a return type of atom is given, then the returned atom is usually :ok, unless otherwise specified.

is_config(v) : boolean

See if a value is a configuration.

Parameters:	v – The value to check.

from_source(s) : {atom, value}

Return a new instance with the configuration read from the provided string.

Parameters:	s – A string containing the configuration source.
Errors:	`RecognizerError` – If there is an error in the configuration syntax.

from_file(path) : {atom, value}

Return a new instance with the configuration read from the specified file.

Parameters:	path – A string containing the path to the configuration source.
Errors:	`RecognizerError` – If there is an error in the configuration syntax or an I/O error when reading the source.

load_file(this, path) : {atom, value}

Load this instance with the configuration read from a file, replacing any existing configuration.

Parameters:	this – The configuration to operate on. path – A string specifying the path to the file which contains the configuration.
Errors:	`RecognizerError` – If there is an I/O error when reading the source or an error in the configuration syntax.

get(this, key, default = :MISSING) : {atom, value}

This function gets a value from the configuration. The value of key can be a simple key (identifier) or a valid path string. If a default value is specified and the key or path not found in the configuration, the specified default is returned.

Parameters:	this – The configuration to operate on. key – A string describing a simple key or a path in the configuration.
Errors:	`invalid_path` – If key is not present in the data and it cannot be parsed as a path. `invalid_path_extra` – If there is a valid path followed by extraneous data. `invalid_container` – If, while traversing a path, a key or index is not of the appropriate type for its container `invalid_index` – If the index is out of range for the container.

add_include(this, paths, append = true) : atom

Append or prepend a list of directories to the include path of a configuration.

Parameters:	this – The configuration to operate on. paths – A list of strings which represent directories, which are to be added to the include path. append – If `true`, the paths are appended, otherwise prepended, to the existing include path.

set_include(this, paths) : atom

Set the include path of a configuration to the specified list of directories.

Parameters:	this – The configuration to operate on. paths – A list of strings which represent directories, which are to become the include path.

get_include(this) : list(binary)

Get the include path of a configuration.

Parameters:	this – The configuration to operate on.
Returns:	A list of strings which represent directories, which are the configuration’s include path.

as_dict(this) : {atom, value}

Get a configuration as a map, recursing into included configurations and nested maps.

Parameters:	this – The configuration to operate on.

is_cached(this) : boolean

See if a configuration is cached.

Parameters:	this – The configuration to operate on.

set_cached(this, cached) : boolean

Set if a configuration is cached.

Parameters:	this – The configuration to operate on. cached – If `true`, a cache will be created and populated. If `false`, any existing cache will be disabled.

is_no_duplicates(this) : boolean

See if a configuration disallows duplicate keys.

Parameters:	this – The configuration to operate on.

set_no_duplicates(this, no_dupes) : boolean

Set if a configuration disallows duplicate keys.

Parameters:	this – The configuration to operate on. no_dupes – If `true`, any subsequent `load_file` will fail if the source has duplicate keys. If `false`, then duplicate keys are allowed and later values for duplicate keys will silently overwrite earlier values.

set_path(this, path) : atom

Set the path this configuration was loaded from. You normally don’t need to call this because it will be called by load_file or from_file, but if you load a configuration from text which references included configurations, you might find this useful.

Parameters:	this – The configuration to operate on. path – The path to set.