The CFG API for Go¶

The CFG reference implementation for Go is written and tested using Go >= 1.13. It’s implemented in a module called config.

Installation¶

You can use this package using go get github.com/vsajip/go-cfg-lib/config and then importing github.com/vsajip/go-cfg-lib/config in your code.

There’s a minimal example of a program that uses CFG here.

Exploration¶

To explore CFG functionality for Go, we use the gore Read-Eval-Print-Loop (REPL), which is available from here. Once installed, you can invoke a shell using

$ gore

Note that gore requires a version of Go that supports Go modules. Note: We use a slightly modified version for this documentation, which prints the type of the objects explored in addition to printing their values.

Getting Started with CFG in Go¶

A configuration is represented by an instance of the Config struct. A reference to one can be obtained using either the NewConfig() or FromFile() functions. The former creates an instance with no configuration loaded, while the latter initialises using a configuration in a specified file: the text is read in, parsed and converted to an object that you can then query. A simple example:

test0.cfg¶

a: 'Hello, '
b: 'world!'
c: {
  d: 'e'
}
'f.g': 'h'
christmas_morning: `2019-12-25 08:39:49`
home: `$HOME`
foo: `$FOO|bar`

Loading a configuration¶

The configuration above can be loaded as shown below. In the REPL shell:

gore> :import os
gore> :import config
gore> os.Chdir(os.Getenv("PWD"))
<nil>(<nil>)
gore> cfg, err := config.FromFile("test0.cfg")
*config.Config(Config("test0.cfg" [7 items]))
<nil>(<nil>)

The os.Chdir(os.Getenv("PWD")) dance is needed because of the way gore works, and it isn’t relevant to this example. The <nil>(<nil>) printed is just the error returned from the function.

Access elements with keys¶

Accessing elements of the configuration with a simple key is not much harder than using a map[string]Any:

gore> cfg.Get("a")
string(Hello, )
<nil>(<nil>)
gore> cfg.Get("b")
string(world!)
<nil>(<nil>)

You can see the types and values of the returned objects are as expected.

Access elements with paths¶

As well as simple keys, elements can also be accessed using path strings:

gore> cfg.Get("c.d")
string(e)
<nil>(<nil>)

Here, the desired value is obtained in a single step, by (under the hood) walking the path c.d – first getting the mapping at key c, and then the value at d in the resulting mapping.

Note that you can have simple keys which look like paths:

gore> cfg.Get("f.g")
string(h)
<nil>(<nil>)

If a key is given that exists in the configuration, it is used as such, and if it is not present in the configuration, an attempt is made to interpret it as a path. Thus, f.g is present and accessed via key, whereas c.d is not an existing key, so is interpreted as a path.

Access to date/time objects¶

You can also get native Go date/time objects from a configuration, by using an ISO date/time pattern in a backtick-string:

gore> cfg.Get("christmas_morning")
time.Time(2019-12-25 08:39:49 +0000 UTC)
<nil>(<nil>)

Access to environment variables¶

To access an environment variable, use a backtick-string of the form `$VARNAME`:

gore> cfg.Get("home")
string(/home/vinay)
<nil>(<nil>)

You can specify a default value to be used if an environment variable isn’t present using the `$VARNAME|default-value` form. Whatever string follows the pipe character (including the empty string) is returned if VARNAME is not a variable in the environment.

gore> cfg.Get("foo")
string(bar)
<nil>(<nil>)

Access to computed values¶

Sometimes, it’s useful to have values computed declaratively in the configuration, rather than imperatively in the code that processes the configuration. For example, an overall time period may be specified and other configuration values are fractions thereof. It may also be desirable to perform other simple calculations declaratively, e.g. concatenation of numerous file names to a base directory to get a final pathname.

test0a.cfg¶

total_period : 100
header_time: 0.3 * ${total_period}
steady_time: 0.5 * ${total_period}
trailer_time: 0.2 * ${total_period}
base_prefix: '/my/app/'
log_file: ${base_prefix} + 'test.log'

When this file is read in, the computed values can be accessed directly:

gore> cfg.Get("header_time")
float64(30)
<nil>(<nil>)
gore> cfg.Get("steady_time")
float64(50)
<nil>(<nil>)
gore> cfg.Get("trailer_time")
float64(20)
<nil>(<nil>)
gore> cfg.Get("log_file")
string(/my/app/test.log)
<nil>(<nil>)

Including one configuration inside another¶

There are times when it’s useful to include one configuration inside another. For example, consider the following configuration files:

logging.cfg¶

layouts: {
  brief: {
    pattern: '%d [%t] %p %c - %m%n'
  }
},
appenders: {
  file: {
    layout: 'brief',
    append: false,
    charset: 'UTF-8'
    level: 'INFO',
    filename: 'run/server.log',
    append: true,
  },
  error: {
    layout: 'brief',
    append: false,
    charset: 'UTF-8'
    level: 'ERROR',
    filename: 'run/server-errors.log',
  },
  debug: {
    layout: 'brief',
    append: false,
    charset: 'UTF-8'
    level: 'DEBUG',
    filename: 'run/server-debug.log',
  }
}
loggers: {
  mylib: {
    level: 'INFO'
  }
  'mylib.detail': {
    level: 'DEBUG'
  }
},
root: {
  handlers: ['file', 'error', 'debug'],
  level: 'WARNING'
}

redirects.cfg¶

cookies: {
  url: 'http://www.allaboutcookies.org/',
  permanent: false
},
freeotp: {
  url: 'https://freeotp.github.io/',
  permanent: false
},
'google-auth': {
  url: 'https://play.google.com/store/apps/details?id=com.google.android.apps.authenticator2',
  permanent: false
}

main.cfg¶

secret: 'some random application secret',
port: 8000,
sitename: 'My Test Site',
default_access: 'public',
ignore_trailing_slashes: true,
site_options: {
  want_ipinfo: false,
  show_form: true,
  cookie_bar: true
},
connection: 'postgres+pool://db_user:db_pwd@localhost:5432/db_name',
debug: true,
captcha_length: 4,
captcha_timeout: 5,
session_timeout: 7 * 24 * 60 * 60,  # 7 days in seconds
redirects: @'redirects.cfg',
email: {
  sender: 'no-reply@my-domain.com',
  host: 'smtp.my-domain.com:587',
  user: 'smtp-user',
  password: 'smtp-pwd'
}
logging: @'logging.cfg'

The main.cfg contents have been kept to the highest-level values, within logging and redirection configuration relegated to other files logging.cfg and redirects.cfg which are then included in main.cfg. This allows the high-level configuration to be more readable at a glance, and even allows the separate configuration files to be e.g. maintained by different people.

The contents of the sub-configurations are easily accessible from the main configuration just as if they had been defined in the same file:

gore> cfg.Get("logging.appenders.file.filename")
string(run/server.log)
<nil>(<nil>)
gore> cfg.Get("redirects.freeotp.url")
string(https://freeotp.github.io/)
<nil>(<nil>)
gore> cfg.Get("redirects.freeotp.permanent")
bool(false)
<nil>(<nil>)

Avoiding unnecessary repetition¶

Don’t Repeat Yourself (DRY) is a useful principle to follow. CFG can help with this. You may have noticed that the logging.cfg file above has some repetitive elements:

logging.cfg (partial)¶

appenders: {
  file: {
    layout: 'brief',
    append: false,
    charset: 'UTF-8'
    level: 'INFO',
    filename: 'run/server.log',
    append: true,
  },
  error: {
    layout: 'brief',
    append: false,
    charset: 'UTF-8'
    level: 'ERROR',
    filename: 'run/server-errors.log',
  },
  debug: {
    layout: 'brief',
    append: false,
    charset: 'UTF-8'
    level: 'DEBUG',
    filename: 'run/server-debug.log',
  }
}

This portion could be rewritten as:

logging.cfg (partial)¶

defs: {
  base_appender: {
    layout: 'brief',
    append: false,
    charset: 'UTF-8'
  }
},
appenders: {
  file: ${defs.base_appender} + {
    level: 'INFO',
    filename: 'run/server.log',
    append: true,
  },
  error: ${defs.base_appender} + {
    level: 'ERROR',
    filename: 'run/server-errors.log',
  },
  debug: ${defs.base_appender} + {
    level: 'DEBUG',
    filename: 'run/server-debug.log',
  }
}

where the common elements are separated out and just referenced where they are needed. We find it useful to put all things which will be reused like this in one place in the condiguration, so we always know where to go to make changes. The key used is conventionally defs or base, though it can be anything you like.

Access is just as before, and provides the same results:

gore> cfg.Get("logging.appenders.file.level")
string(INFO)
<nil>(<nil>)
gore> cfg.Get("logging.appenders.file.layout")
string(brief)
<nil>(<nil>)
gore> cfg.Get("logging.appenders.file.append")
bool(true)
<nil>(<nil>)
gore> cfg.Get("logging.appenders.file.filename")
string(run/server.log)
<nil>(<nil>)
gore> cfg.Get("logging.appenders.error.append")
bool(false)
<nil>(<nil>)
gore> cfg.Get("logging.appenders.error.filename")
string(run/server-errors.log)
<nil>(<nil>)

The definition of logging.appenders.file as ${defs.base_appender} + { level: 'INFO', filename: 'run/server.log', append: true } has resulted in an evaluation which first fetches the defs.base_appender value, which is a mapping, and “adds” to that the literal mapping which defines the level, filename and append keys. The + operation for mappings is implemented as a copy of the left-hand side merged with the right-hand side. Note that the append value for logging.appenders.file is overridden by the right-hand side to true, whereas that for e.g. logging.appenders.error is unchanged as false.

We could do some further refinement by factoring out the common location for the log files:

logging.cfg (partial)¶

defs: {
  base_appender: {
    layout: 'brief',
    append: false,
    charset: 'UTF-8'
  }
  log_prefix: 'run/',
},
layouts: {
  brief: {
    pattern: '%d [%t] %p %c - %m%n'
  }
},
appenders: {
  file: ${defs.base_appender} + {
    level: 'INFO',
    filename: ${defs.log_prefix} + 'server.log',
    append: true,
  },
  error: ${defs.base_appender} + {
    level: 'ERROR',
    filename: ${defs.log_prefix} + 'server-errors.log',
  },
  debug: ${defs.base_appender} + {
    level: 'DEBUG',
    filename: ${defs.log_prefix} + 'server-debug.log',
  }
}

with the same result as before. It is slightly more verbose than before, but the location of all files can be changed in just one place now, as opposed to three, as it was before.

Types¶

Any¶: This is a type alias for interface{}.

Sequence¶: This is a type alias for []Any.

Mapping¶: This is a type alias for map[string]Any.

Config¶

This struct represents a configuration.

Fields

NoDuplicates¶: This is a bool which should be true (the default) if no duplicate keys are allowed when loading a configuration. If not allowed and duplicates are seen, an error will be returned. If duplicates are allowed, values seen later will overwrite values seen earlier for any keys that are duplicated.

StrictConversions¶: This is a bool which should be true (the default) if special string conversions should always succeed. If true and a conversion can’t be done, an error is returned.

IncludePath¶: This is a []string of directories which is searched for included sub-configurations if the parent directory of Path (or the current directory, if Path isn’t set) doesn’t contain them.

Path¶: This is a string which should be the path to the file where the configuration was loaded from.

Context¶: This is a *Mapping which can hold a mapping between identifiers and values. It’s used to look up identifiers during evaluation of AST nodes.

Methods

(self *Config) Load(reader *io.Reader) → error¶

This loads the configuration from the specified stream.

Parameters:	reader – The stream to read from.

(self *Config) LoadFile(path string) → error¶

This loads the configuration from the specified file.

Parameters:	path – The path to the file to read from.

(self *Config) Get(key string) → (Any, error)¶

Parameters:	key – The particular configuration entry which is being sought. If the key is not in the data, an attempt will be made to parse it as a path.
Returns:	The value at the specified key or path, if present.

(self *Config) GetWithDefault(key string, defaultValue Any) → (Any, error)¶: This gets the value at a specified key. If the key is not in the data, an attempt will be made to parse it as a path. If a key isn’t found, the defaultValue is returned.

(self *Config) AsDict() → (map[string]Any, error)¶: This converts the configuration into a map.

Functions¶

NewConfig() → *Config¶: Returns a new instance of Config with default values for all fields.

FromFile(path string) → (*Config, error)¶

Returns a configuration loaded from the specified file.

Parameters:	path – The path to the configuration file to be loaded.
Returns:	A reference to the loaded configuration.

The CFG API for Go

The CFG API for Go

Table Of Contents

The CFG API for Go¶

Installation¶

Exploration¶

Getting Started with CFG in Go¶

Loading a configuration¶

Access elements with keys¶

Access elements with paths¶

Access to date/time objects¶

Access to environment variables¶

Access to computed values¶

Including one configuration inside another¶

Avoiding unnecessary repetition¶

Types¶

Functions¶