Bibliothek für Konfigurationen

Software must be configurable to be flexible. A configuration defines parameters and settings for software. Usually, the settings are stored in a configuration file that the software reads. But how do we ensure that a configuration is complete and valid? That all aspects that need to be configured are actually configured? That there are sensible defaults for values not explicitly configured? And that the values entered in the configuration are actually sensible values?

To avoid answering these questions anew for each project, we have developed a library for configurations for Clojure and ClojureScript that we have been using successfully for many years - and present in this article.

Configuration

Following typical Clojure style, we represent configurations as nested key-value maps.¹ Such configuration maps consist of settings and sections. Here‘s a simple example of a configuration for an application with a web server:

{:log-level :info
 :webserver {:host "0.0.0.0"
             :port 80}}

The configuration contains the :log-level setting for configuring the application‘s minimum logging level, which is set to the value :info. It also contains a section :webserver for configuring the web server‘s properties. The :webserver section is itself a configuration map with settings :host for the listen host (here the web server is accessible from outside by specifying "0.0.0.0") and the listen port, namely the standard HTTP port 80.

You can already sense a possible pitfall with configurations: An application typically expects very specific configured values, which it then interprets and responds to accordingly. In our example, the application assumes that the value for the logging level is actually a keyword like :info and not, say, a string "info" or "INFO". And the web server port should be a number 80 and not a string "80" and so on.

And programmers can specify exactly such constraints with our library and then check configurations against these constraints, ensuring that the application receives a valid and understandable configuration.

Configuration Schema

To do this, the application defines a configuration schema that it expects. Such a configuration schema is called a schema in our library. Now let‘s define the settings, sections, and then the schema for the above example configuration in order. In the code examples, we assume that the namespace active.clojure.config is imported as config.

Let‘s start with the definition of the logging level setting:

(def log-level-setting
  (config/setting
   :log-level
   "Minimal log level, defaults to :error."
   (config/one-of-range #{:trace :debug :info :warn :error :fatal}
                        :error)))

The call to config/setting expects the setting‘s keyword as its first argument, here it‘s :log-level. Next is a mandatory string documenting the setting – clear descriptions are very helpful for reference, showing users what they can configure and when the configuration causes errors. As the third argument, we define the valid value range for the setting, just called a range in our library. The value range for the log level is a one-of-range, meaning a valid value is one of the specified keywords :trace, :debug, :info, :warn, :error, or :fatal. Additionally, the range allows specifying a default if the setting is not explicitly configured. Here it‘s :error – this fact is also helpfully described in the description text.

Following the same pattern, we now write the settings for the web server. First for the host:

(def webserver-host-setting
  (config/setting
   :host
   "The address the webserver listens on, defaults to 0.0.0.0."
   (config/default-string-range "0.0.0.0")))

The setting for the web server host should be a string, so we use a string range. In this case, the range additionally allows specifying a default "0.0.0.0", so we use default-string-range to define this. Our library already contains a variety of ranges for commonly used value ranges, and it‘s easy to define custom ranges for more specialized value ranges. We‘ll see another built-in range with the next configuration setting.

The definition of the configuration setting for the port looks like this:

(def webserver-port-setting
  (config/setting
   :port
   "The port the webserver listens on, defaults to 3000."
   (config/integer-between-range 0 65535 3000)))

The configuration expects the port as an integer, so we use an integer range. However, in TCP networks there are only a limited number of ports, namely from 0 to 65535, so we restrict the possible value range to that with integer-between-range. The third argument to integer-between-range is the default if the setting was not explicitly made. Here our application uses port 3000.

These two settings make up the web server section – we bundle them together in a schema with an appropriate description:

(def webserver-schema
 (config/schema
  "Configuration settings for the web server"
  webserver-host-setting
  webserver-port-setting))

And with this schema, we can now define the web server section:

(def webserver-section
  (config/section
   :webserver
   webserver-schema))

No description is needed here, since the description is already in the schema.

Now we have everything we need for our complete configuration schema:

(def schema
 (config/schema
  "Configuration for our application"
  log-level-setting
  webserver-section))

Validation and Normalization

With this, we can now check configurations against our schema for validity:

(config/normalize&check-config-object
  schema
  {:log-level :info
   :webserver {:host "0.0.0.0"
               :port 80}})

The call returns the configuration that was passed in, meaning that the passed configuration is valid and complete.

Incomplete configurations are completed with defaults: The call

(config/normalize&check-config-object
  schema
  {:webserver {:host "0.0.0.0"}})

returns the completed configuration with the defined defaults:

{:log-level :error
 :webserver {:host "0.0.0.0"
             :port 3000}}

Finally, an erroneous configuration

(config/normalize&check-config-object
  schema
  {:webserver {:port "80"}})

returns a data structure called RangeError that describes the error in the configuration via a path to the erroneous configuration setting, the incorrect value, and the actually expected value range. This gives users the ability to understand and fix the problem. The representation of the above RangeError printed with str looks like this:

"Range error at path [:webserver :port]: value \"80\" is not in range integer between 0 and 65535"

Accessing Configuration Settings

Now we could read the settings from the nested configuration map using the settings keywords. But this isn‘t a particularly robust approach, since the Clojure compiler won‘t even notice possible typos in the keywords – in case of doubt, such access to a non-existent key simply returns nil. And that in turn could confuse or disrupt our application – which we want to prevent through robust configuration.

Therefore, we don‘t want to access the configuration map directly with keywords, but rather use safer mechanisms available in our library:

• We don‘t handle the map directly, but work with a special data type called Configuration.

• We use the settings and sections bound to variables to access the values – this way the compiler notices possible typos at compile time.

• We use functions for access that ensure a validated configuration.

We‘ll show what this looks like in practice shortly. First, let‘s bind our example configuration to a name for our next experiments and explanations:

(def c
  (config/make-configuration
    schema
    {:log-level :info
     :webserver {:host "0.0.0.0"
                 :port 80}}))

From this configuration, we can now read the values for the settings using the access function:

(config/access c log-level-setting)

returns :info.

And reading from nested configurations works like this:

(config/access c webserver-port-setting webserver-section)

This returns 80.

It‘s also possible to extract an entire section and read values from it:

(let [webserver-config (config/section-subconfig c webserver-section)]
  (config/access webserver-config webserver-port-setting))

This enables configuration of different application components without too much coupling.

Lenses on Configuration Settings

Lenses play a major role in our daily work – and therefore also in this blog – because they have great practical utility. Therefore, our configuration library naturally knows how to work with lenses, for example for accessing settings. The code

(let [log-level-lens (config/access-lens log-level-setting)]
  (log-level-lens c))

returns :info, just like the direct access above. Noteworthy is that we can construct the lens without the configuration – a very elegant way to write selectors.

The same works for nested configurations:

(let [webserver-port-lens (config/access-lens webserver-port-setting
                                              webserver-section)]
  (webserver-port-lens c))

This returns 80 as expected.

Projection of Configuration Settings

Avoiding or at least reducing coupling is the most important mantra for good software. One method for decoupling is to use different data structures for different areas of the application, even if they are very similar.²

Therefore, it‘s worthwhile in software to separate settings from configuration – for example by introducing a data structure that represents the settings but is not the Configuration data structure. Our library makes this easy through an interplay of records and projection lenses.

We can define the settings data structure we want to use in our application as a record:

(define-record-type Settings
  {:projection-lens settings-projection-lens}
  make-settings
  settings?
  [log-level settings-log-level?
   webserver-host settings-webserver-host
   webserver-port settings-webserver-port])

And define a projection lens between this record and the configuration:

(def configuration->settings
  (settings-projection-lens
    (config/access-lens log-level-setting)
    (config/access-lens webserver-host-setting webserver-section)
    (config/access-lens webserver-port-setting webserver-section)))

With this compact definition, we can now translate a configuration into settings³:

(configuration->settings c)

Profiles

Another useful feature is support for profiles. There are often variants of a configuration due to a particular environment or deployment. Our library helps with this too. For example, we can configure a profile for test environments that adjusts some aspects compared to the production environment. To do this, we add a profile named :test under :profiles in our example configuration. The complete configuration looks like this:

{:log-level :info
 :webserver {:host "0.0.0.0"
             :port 80}
 :profiles
 {:test {:log-level :debug}}}

In the test profile, we‘ve configured that we want to see debug logs in the test environment.

When reading the configuration, we can then „mix in“ the settings for this profile; the profile-specific settings then override the general settings. To do this, we call normalize&check-config-object additionally with a list of profiles to consider:

(config/normalize&check-config-object
  schema
  [:test]
  {:log-level :info
   :webserver {:host "0.0.0.0"
               :port 80}
   :profiles
   {:test {:log-level :debug}}})

This then returns the completed configuration for the test environment with logging level :debug:

{:log-level :debug
 :webserver {:host "0.0.0.0"
             :port 80}}

Conclusion

An important foundation for flexible applications is robust configuration. The presented library enables this robust handling of configuration. The library has been successfully used in production in all our Clojure projects for many years.