Heredia, Costa Rica 2022-09-11

Series: One Configuration for all your JavaScript, Article 1

Traditionally, different frameworks or libraries in JavaScript provide either their own mechanism for data configuration, or refer you to a preferred or popular configuration mechanism.

For example, in the NodeJS world we have the popular config or dotenv packages. The first one uses JSON files while the second one uses plain text files with value pairs to define data configuration values. Both handle the idea of application environment and provide a mechanism to select the correct configuration based on the current application environment.

So what's the problem here? Not really a problem per-se with those packages, it is that niether offer a complete solution or really work under any project type using any arbitrary framework or library.

Introduction

The two packages mentioned above are packages meant for NodeJS, I would say in my limited experience. They are even written as CommonJS modules (as seen here for config). Sure, tools like RequireJS can actually make CommonJS work in the browser, and many times that works.

So in a nutshell, you have configuration packages for one project type, and another set of configuration packages for another project type, and you have to go learn each one of them just because what works in NodeJS may or may not work in, say, the browser.

But what if I told you that you no longer need to depend on external tools? What if I told you that you can get all your configuration needs covered and more with one package? And even more, what if I told you that this package works exactly the same, without the involvement of tools like RequireJS, in NodeJS and browser projects such as ReactJS? Now we're talking, right?

Such package exists. It is wj-config and comes packed with features (more than config), yet is lighter than config. At the time of this writing, wj-config was weighing 74.9kb as reported by NPMJS, while config was weighing 93.5kb; but note that wj-config has a 38kb readme in it, while config has 2 readme's weighing a total around the 28kb (History.md and README.md). Therefore the total size for code is around 36.9kb for wj-config and 65.5kb for config. Almost half the size.

Technical Introduction

Source Language

The first thing to note here is that wj-config is written 100% in TypeScript and transpiled to ES Modules. ES Modules are the standard for JavaScript modules according to the official ECMAScript specification.

According to the Can I Use website and MDN, browsers have included extensive module support through the course of the years, making it a good choice. Furthermore, NodeJS has supported them for a very good long time now as well, so no issues there either.

This means that the package, as is, can be used quickly in the most popular server-side JavaScript runner NodeJS as well as used in browser applications. This covers a lot of ground in the JavaScript world. Even if I don't say so explicitly here, wj-config has a very good chance of working as-is in other server-side JavaScript engines. I just don't mention them because I haven't tested them. I apologize, I just don't have the time.

Zero Dependencies

Because wj-config has no dependencies, the package is free to set its own standard and is free to transpile to ES Modules without a single worry in its little head.

Having no dependencies also makes it a neutral addition into any framework or library. It doesn't come with uninvited guests or freeloaders. It does its thing without having to ask for help from anywhere else.

Modern API

The configuration package provides a builder with fluent syntax to easily define the data sources as well as the wanted features. It does not include anything the consumer doesn't want and will accept a multitude of data sources for your convenience.

The build process is also asynchronous, complying with current trends to create responsive and cost-effective applications. This is particularly important in the JavaScript world because JavaScript is single-threaded. Asynchronous building helps your application work faster.

The Main Course: Configuring an Application

The main objective of a configuration package like wj-config is to provide centralized access to all configured data an application may need at any point in the source code. This comes from very long ago where developers warned us all about the evils of magic numbers or strings in source code. Long story short: Don't hardcode data and instead put it in a configuration file or something somewhere outside the source code. This is what wj-config and other configuration packages help you do.

Your First Configuration Module

NOTE: The code presented in this section is not specific to NodeJS. As stated in the introduction, this works everywhere. It can be in a NodeJS application, or it can be in a ReactJS application. Yes, without change.

To see how simple it is to build a configuration object, see the following module, which I always choose to name config.js (or config.mjs if you're into the module-specific file extension):

import wjConfig from 'wj-config';

// Configuration data.  We'll see how to use a JSON file later.
const data = '{ "app": { "title": "My Application powered by wj-config" }, "logging": { "minLevel": "info" } }';

const config = wjConfig()
    .addJson(data)
    .build();

// Top-level await.  See below for notes around this.
export default await config;

What does the above do, exactly? Well, not much, really:

1. Calls wjConfig() to instantiate a new configuration builder object.
2. Instructs the builder to use the JSON stored in the data variable as source of configuration data.
3. Instructs the build to start building the configuration object (remember, build() is asynchronous).
4. Exports the resulting configuration object by awaiting the promise returned by build() using a top-level await.

Ok, so what's the difference between that and the following?

const myConfig = JSON.parse('{ "app": { "title": "My Application powered by wj-config" }, "logging": { "minLevel": "info" } }');

export default myConfig;

Actually, there's NO difference. So why bother with wj-config? Because the above is not a realistic example, and if it ever is, please, by all means, don't use wj-config or any other configuration package.

The example above was simplistic enough just to demonstrate the general procedure on how to use wj-config. The real stuff is coming, but allow me to interject a note about top-level await here.

Top-Level Await and Why Is Needed

The JavaScript feature called top-level await is something that was recently added to the ECMAScript specification and nowadays all major browsers support (see this Can I Use page).

This is needed because wj-config allows for asynchronous data sources. This is not evident in our first example because all we are providing is some static JSON string as configuration data, but in reality our projects will consume data from files read from disk or fetched from the web. These operations are best handled asynchronously and in fact, the supporting API around those operations is asynchronous. Just take fetch() as a clear example.

Your first REALISTIC Configuration Module

Software systems always live in what is commonly referred to as environments. This is a concept that comes naturally from the need for having copies of a system for testing, development, stress testing, feature previews, demos, and production. In other words, we usually don't have just the production system running. We normally have at least one more copy running somewhere for our own private benefit.

Well, with environment differences come configuration differences. For example, a server-side JavaScript application might need a database server name to change between different environments. Otherwise we would be messing with the production data during our testing or active development tasks. I'm sure you get the idea, so I will shut up about environments altogether now and assume you understand this topic.

wj-config has the ability to alter what I call a master configuration source with a subset configuration source that is dependent on the environment, or really, anything else a developer can think of.

In reality, wj-config is really more general than being environment-aware. It really isn't. It just wants you to give it as many data sources as you want and those will be merged as one, where subsequently-specified data sources override the values present in previously-specified data sources. wj-config doesn't really care if data source X represents an environment or not, it is just used to set values and even override values on all preceding data sources.

So why did we talk about environments? Because it is virtually the main reason for configuration overrides 99% of the time and I promised you a realistic example.

So without further ado, here's the first REALISTIC example:

import wjConfig from 'wj-config';
// Use these lines in ReactJS; NodeJS requires an extra piece of syntax as shown below.
import maingConfig from './config.json';
import devConfig from './config.dev.json';
// NodeJS version.  Sorry, this is out of wj-config's control.
import mainConfig from "./config.json" assert {type: 'json'};
import devConfig from "./config.dev.json" assert {type: 'json'};

const config = wjConfig()
    .addObject(mainConfig)
    .addObject(devConfig)
    .build();

export default await config;

The code explanation would be:

1. We obtain the builder as before, calling wjConfig().
2. We instruct the builder that, as first source of configuration data, we will use the object stored in mainConfig, which is the result of importing a JSON file.
3. We instruct the builder that, as second source of configuration data, we will use the object stored in devConfig, which is the result of importing another JSON file, which by the name you can probably assume it is configuration values specific to the dev environment.
4. As before, we build() and await.

So what's the result's appearance? What are we obtaining? Well, we obtain an object with the properties that were defined in BOTH JSON files. Let's see an example.

Let's say this is the main configuration file, config.json:

{
    "app": {
        "title": "My Application"
    },
    "logging": {
        "minLevel": "info"
    }
}

Similarly, let's write down the contents of config.dev.json:

{
    "logging": {
        "minLevel": "debug",
        "includeExtraData": true
    }
}

The result of importing our config.js (or config.mjs) module would be the following object:

{
    "app": {
        "title": "My Application"
    },
    "logging": {
        "minLevel": "debug",
        "includeExtraData": true
    }
}

These are the highlights:

1. The resulting configuration object has the logging.includeExtraData configuration value not present in config.json. This is because it is present in config.dev.json.
2. The value of logging.minLevel is "debug" because that's the defined value in config.dev.json, and that object was specified as second data source, meaning its values will override previously existing values coming from previous data sources, which in this example is just one previous data source: config.json.

This, avid reader, is the magic provided by wj-config. And yes, to be fair, it is also done by the more popular config package. But stay tuned, because wj-config comes with a lot more.

By extension, you can infer that more data sources can further contribute to shape the final configuration object. There is no bound to the number of data sources that can be used. Merge away until your heart is content.

In the following articles in the series we will be covering more features of wj-config, such as the Environment class, data source value tracing and the most amazing feature of them all: Automagically creating REST URL's from configuration data.