The CFG API for JavaScript

The CFG reference implementation for JavaScript is written and tested using Node v12 or later. It’s implemented in a module called config.

Installation

The package can be installed for use from the NPM package registry using the package name cfg-lib:

$ npm install cfg-lib

There’s a minimal example of a program and project that uses CFG here.

Exploration

To explore CFG functionality for JavaScript, we can just use the node Read-Eval-Print-Loop (REPL). You can invoke it using

$ node

Getting Started with CFG in JavaScript

A configuration is represented by an instance of the Config() class. The constructor for this class can be passed a filename or a stream which contains the text for the configuration. The text is read in, parsed and converted to an object that you can then query. A simple example:

test0.cfg

a: 'Hello, '
b: 'world!'
c: {
  d: 'e'
}
'f.g': 'h'
christmas_morning: `2019-12-25 08:39:49`
home: `$HOME`
foo: `$FOO|bar`

Loading a configuration

The configuration above can be loaded as shown below. In the REPL shell:

> const config = require('cfg-lib');
undefined
> let cfg = new config.Config("test0.cfg");
undefined

Usage in a browser

In a browser, API elements are exposed in the CFG namespace:

let stream = CFG.makeStream('abc: 1');
let cfg = new CFG.Config(stream);
let d = cfg.asDict();

As you have no access to a filesystem in the browser, a stream needs to be constructed from a string value and a configuration created from that. You can use the makeStream() function to do this. The above code would result in d having a value of the object { abc: 1}, as you might expect.

You’ll need to include the browser-specific version of the library from a CDN using a URL such as https://cdn.jsdelivr.net/npm/cfg-lib@0.1.1-4/dist/config.min.js - search the CDN for the latest version, which will be of the form X.Y.Z-N, where -N is optional and relates to versions updated due to packaging problems rather than any change to the actual functionality of the package.

Access elements with keys

Accessing elements of the configuration is done using the get method:

> cfg.get('a')
'Hello, '
> cfg.get('b')
'world!'

Access elements with paths

As well as simple keys, elements can also be accessed using path strings:

> cfg.get('c.d')
'e'

Here, the desired value is obtained in a single step, by (under the hood) walking the path c.d – first getting the mapping at key c, and then the value at d in the resulting mapping.

Note that you can have simple keys which look like paths:

> cfg.get('f.g')
'h'

If a key is given that exists in the configuration, it is used as such, and if it is not present in the configuration, an attempt is made to interpret it as a path. Thus, f.g is present and accessed via key, whereas c.d is not an existing key, so is interpreted as a path.

Access to date/time objects

You can also get native date/time objects from a configuration, by using an ISO date/time pattern in a backtick-string:

> cfg.get('christmas_morning')
2019-12-25T08:39:49.000Z

Access to environment variables

To access an environment variable, use a backtick-string of the form `$VARNAME`:

> cfg.get('home')
'/home/vinay'

You can specify a default value to be used if an environment variable isn’t present using the `$VARNAME|default-value` form. Whatever string follows the pipe character (including the empty string) is returned if VARNAME is not a variable in the environment.

> cfg.get('foo')
'bar'

Access to computed values

Sometimes, it’s useful to have values computed declaratively in the configuration, rather than imperatively in the code that processes the configuration. For example, an overall time period may be specified and other configuration values are fractions thereof. It may also be desirable to perform other simple calculations declaratively, e.g. concatenation of numerous file names to a base directory to get a final pathname.

test0a.cfg

total_period : 100
header_time: 0.3 * ${total_period}
steady_time: 0.5 * ${total_period}
trailer_time: 0.2 * ${total_period}
base_prefix: '/my/app/'
log_file: ${base_prefix} + 'test.log'

When this file is read in, the computed values can be accessed directly:

> cfg.get('header_time')
30
> cfg.get('steady_time')
50
> cfg.get('trailer_time')
20
> cfg.get('log_file')
'/my/app/test.log'

Including one configuration inside another

There are times when it’s useful to include one configuration inside another. For example, consider the following configuration files:

logging.cfg

layouts: {
  brief: {
    pattern: '%d [%t] %p %c - %m%n'
  }
},
appenders: {
  file: {
    layout: 'brief',
    append: false,
    charset: 'UTF-8'
    level: 'INFO',
    filename: 'run/server.log',
    append: true,
  },
  error: {
    layout: 'brief',
    append: false,
    charset: 'UTF-8'
    level: 'ERROR',
    filename: 'run/server-errors.log',
  },
  debug: {
    layout: 'brief',
    append: false,
    charset: 'UTF-8'
    level: 'DEBUG',
    filename: 'run/server-debug.log',
  }
}
loggers: {
  mylib: {
    level: 'INFO'
  }
  'mylib.detail': {
    level: 'DEBUG'
  }
},
root: {
  handlers: ['file', 'error', 'debug'],
  level: 'WARNING'
}

redirects.cfg

cookies: {
  url: 'http://www.allaboutcookies.org/',
  permanent: false
},
freeotp: {
  url: 'https://freeotp.github.io/',
  permanent: false
},
'google-auth': {
  url: 'https://play.google.com/store/apps/details?id=com.google.android.apps.authenticator2',
  permanent: false
}

main.cfg

secret: 'some random application secret',
port: 8000,
sitename: 'My Test Site',
default_access: 'public',
ignore_trailing_slashes: true,
site_options: {
  want_ipinfo: false,
  show_form: true,
  cookie_bar: true
},
connection: 'postgres+pool://db_user:db_pwd@localhost:5432/db_name',
debug: true,
captcha_length: 4,
captcha_timeout: 5,
session_timeout: 7 * 24 * 60 * 60,  # 7 days in seconds
redirects: @'redirects.cfg',
email: {
  sender: 'no-reply@my-domain.com',
  host: 'smtp.my-domain.com:587',
  user: 'smtp-user',
  password: 'smtp-pwd'
}
logging: @'logging.cfg'

The main.cfg contents have been kept to the highest-level values, within logging and redirection configuration relegated to other files logging.cfg and redirects.cfg which are then included in main.cfg. This allows the high-level configuration to be more readable at a glance, and even allows the separate configuration files to be e.g. maintained by different people.

The contents of the sub-configurations are easily accessible from the main configuration just as if they had been defined in the same file:

> cfg.get('logging.appenders.file.filename')
'run/server.log'
> cfg.get('redirects.freeotp.url')
'https://freeotp.github.io/'
> cfg.get('redirects.freeotp.permanent')
false

Avoiding unnecessary repetition

Don’t Repeat Yourself (DRY) is a useful principle to follow. CFG can help with this. You may have noticed that the logging.cfg file above has some repetitive elements:

logging.cfg (partial)

appenders: {
  file: {
    layout: 'brief',
    append: false,
    charset: 'UTF-8'
    level: 'INFO',
    filename: 'run/server.log',
    append: true,
  },
  error: {
    layout: 'brief',
    append: false,
    charset: 'UTF-8'
    level: 'ERROR',
    filename: 'run/server-errors.log',
  },
  debug: {
    layout: 'brief',
    append: false,
    charset: 'UTF-8'
    level: 'DEBUG',
    filename: 'run/server-debug.log',
  }
}

This portion could be rewritten as:

logging.cfg (partial)

defs: {
  base_appender: {
    layout: 'brief',
    append: false,
    charset: 'UTF-8'
  }
},
appenders: {
  file: ${defs.base_appender} + {
    level: 'INFO',
    filename: 'run/server.log',
    append: true,
  },
  error: ${defs.base_appender} + {
    level: 'ERROR',
    filename: 'run/server-errors.log',
  },
  debug: ${defs.base_appender} + {
    level: 'DEBUG',
    filename: 'run/server-debug.log',
  }
}

where the common elements are separated out and just referenced where they are needed. We find it useful to put all things which will be reused like this in one place in the condiguration, so we always know where to go to make changes. The key used is conventionally defs or base, though it can be anything you like.

Access is just as before, and provides the same results:

> cfg.get('logging.appenders.file.level')
'INFO'
> cfg.get('logging.appenders.file.layout')
'brief'
> cfg.get('logging.appenders.file.append')
true
> cfg.get('logging.appenders.file.filename')
'run/server.log'
> cfg.get('logging.appenders.error.append')
false
> cfg.get('logging.appenders.error.filename')
'run/server-errors.log'

The definition of logging.appenders.file as ${defs.base_appender} + { level: 'INFO', filename: 'run/server.log', append: true } has resulted in an evaluation which first fetches the defs.base_appender value, which is a mapping, and “adds” to that the literal mapping which defines the level, filename and append keys. The + operation for mappings is implemented as a copy of the left-hand side merged with the right-hand side. Note that the append value for logging.appenders.file is overridden by the right-hand side to true, whereas that for e.g. logging.appenders.error is unchanged as false.

We could do some further refinement by factoring out the common location for the log files:

logging.cfg (partial)

defs: {
  base_appender: {
    layout: 'brief',
    append: false,
    charset: 'UTF-8'
  }
  log_prefix: 'run/',
},
layouts: {
  brief: {
    pattern: '%d [%t] %p %c - %m%n'
  }
},
appenders: {
  file: ${defs.base_appender} + {
    level: 'INFO',
    filename: ${defs.log_prefix} + 'server.log',
    append: true,
  },
  error: ${defs.base_appender} + {
    level: 'ERROR',
    filename: ${defs.log_prefix} + 'server-errors.log',
  },
  debug: ${defs.base_appender} + {
    level: 'DEBUG',
    filename: ${defs.log_prefix} + 'server-debug.log',
  }
}

with the same result as before. It is slightly more verbose than before, but the location of all files can be changed in just one place now, as opposed to three, as it was before.

Classes

The Config class

This class implements a CFG configuration. You’ll generally interface to CFG files using this class. When you pass in a stream or file path to the constructor or the Config.load() / Config.loadFile() methods, the CFG source in the stream or file is parsed and converted into an internal form that can be queried.

class Config()

Variables

Config.noDuplicates: bool = true

Whether duplicates keys are allowed when parsing a mapping or mapping body. If not allowed and duplicates are seen, a ConfigException is thrown.

Config.strictConversions: bool = true

If true, an exception is thrown if a backtick-string can’t be converted. This defaults to true if not provided. It’s intended to help catch typos in backtick-strings.

Config.context: object

A mapping within which to look up identifiers during evaluation of AST nodes.

Config.cached: bool = false

If true, an internal cache is used.

Config.includePath: string[]

A list of directories which is searched for included sub-configurations if the parent directory of path (or the current directory, if path isn’t set) doesn’t contain them.

Config.path: string = null

The path to the file from which this instance’s configuration has been loaded.

Constructors

Config.Config(pathOrStream, options)

Constructs an instance.

Arguments:

• pathOrStream – An optional path or stream. If undefined, an instance is created with no actual configuration loaded. A call to load() or loadFile() would be needed to actually make a usable instance. If a string is provided, it is assumed to be a path to a configuration file, and that file is used to initialize the configuration. If a stream is provided, it is used to obtain the configuration text.
• options – An optional object with attributes as shown above under the heading of Attributes. The values for the following keys are read and copied to the instance:
  • noDuplicates
  • strictConversions
  • includePath
  • cached

Methods

Config.load(stream)

Load this instance with the configuration read from the provided reader.

Arguments:

• stream – A range from which to read the bytes of the configuration.

Exception ConfigException:

If there is an I/O error when reading the source or an error in the configuration syntax.

Config.loadFile(path)

Load this instance with the configuration read from the file at the provided path.

Arguments:

• path – A string specifying the location of the file which contains the configuration.

Exception ConfigException:

If there is an I/O error when reading the source or an error in the configuration syntax.

Config.get(string key)

This method implements the key access operator, e.g. mapping[key]. The value of key can be a simple key (identifier) or a valid path string.

Arguments:

• key – A string describing a simple key or a path in the configuration.

Throws:

• InvalidPathException() – If key is not present in the data and it cannot be parsed as a path.
• BadIndexException() – If, while traversing a path, a key or index is not of the appropriate type for its container, or if it isn’t in the required range.
• ConfigException() – If a key is not found or some other semantic error occurs (for example, an operation involving incompatible types).

Functions

makeStream(s)

Make a stream from a string. This is useful when constructing configurations in a browser, where you have no access to a filesystem. The string could be a literal, a computed value or obtained across the network via an AJAX request.

Arguments:

• s (str()) – The string, which must be valid CFG source.

Returns:

A stream object. The stream can be passed to the Config() constructor.