[Day 4] JS in Pipeline (4): Configs and Secrets

The goal of this series is to introduce some best practices in the local development environment and create a CI/CD pipeline for NodeJS applications.

GitHub repo for this article series: https://github.com/jeanycyang/js-in-pipeline

In this article, we will explore the best practices of configs and secrets.

Use .env file

Currently, our db Container in the docker-compose.yaml file looks like:

db:
    image: mysql:5.7
    environment:
      MYSQL_ROOT_PASSWORD: V4TynEq8RsfvyW7JxXGUpjD9MSrbaB9W
      MYSQL_USER: giftcodeserver
      MYSQL_PASSWORD: m5kCtWQ2G8Zr9VxD79R8LMCfmzX6SAWZ
      MYSQL_DATABASE: giftcodeserver
    ports:
      - "3306:3306"

And our JS code:

const sequelize = new Sequelize('giftcodeserver', 'giftcodeserver', 'm5kCtWQ2G8Zr9VxD79R8LMCfmzX6SAWZ', {  
   host: 'db', // the service name set in docker-compose.yaml
   dialect: 'mysql',
});

Every time you change your MySQL user, password or database, you need to update multiple places.

We can use .env to store our environment variables — Just create a .env file in the project directory.

# .env
MYSQL_ROOT_PASSWORD=V4TynEq8RsfvyW7JxXGUpjD9MSrbaB9W
MYSQL_USER=giftcodeserver
MYSQL_PASSWORD=m5kCtWQ2G8Zr9VxD79R8LMCfmzX6SAWZ
MYSQL_DATABASE=giftcodeserver
MYSQL_HOST=db # this one is for server, MySQL container doesn't need it

Note that .env file uses = to sperate key and value — the format is KEY=VALUE.

Next step, we use a npm package called dotenv.

// db.js
require('dotenv').config(); // by default, it loads .env file
const sequelize = new Sequelize(process.env['MYSQL_DATABASE'], process.env['MYSQL_USER'], process.env['MYSQL_PASSWORD'], {  
   host: process.env['MYSQL_HOST'],
   dialect: 'mysql',
});

Then change our docker-compose.yaml:

db:
    image: mysql:5.7
    env_file:
      - ./.env
    ports:
      - "3306:3306"

And it is done!

Config.js

Later, we find that we need to use an environment variable in other files. Let’s say, this environment variable is called API_KEY.

Require dotenv everywhere you need it — it doesn’t seem to be a good idea. Loading a npm package takes time; Reading .env also takes time (It uses file system!).

Therefore, we can import dotenv only once. Let’s create a file called config.js.

// ./config.js
require('dotenv').config();
function getRequiredEnvVar(envVarName) {
if (!process.env[envVarName]) throw new Error(`${envVarName} is required but not provided!`);
  return process.env[envVarName];
}
module.exports = {
  MYSQL_USER: process.env['MYSQL_USER'] || 'giftcodeserver',
  MYSQL_PASSWORD: getRequiredEnvVar('MYSQL_PASSWORD'),
  MYSQL_DATABASE: process.env['MYSQL_DATABASE'] || 'giftcodeserver',
MYSQL_HOST: process.env['MYSQL_HOST'] || 'localhost',

  API_KEY: getRequiredEnvVar('API_KEY'),
};
// ./api/someAPI.js
const { API_KEY } = require('../config');
...

Now you can import config.js anywhere you want.

An extra benefit of using config.js file is, you can set a default value if it isn’t provided in the .env file. Also, you can do some checks, e.g. throwing an error if a required environment variable is not provided.

The Limitation of .env file

Even though .env file works well with a local development environment, I would recommend using .env ONLY in a development environment.

First, for the production environment or other environments, you are NOT mounting your local volume to the Container. So when you build your Docker image, you will copy a .env file into your Docker image. If you need to change your environment variables, you will need to rebuild an image.

Second, if you have more than one environment, you will need to have multiple .env files, such as /prod/.env, /stage/.env, /test/.env , dev/.env. And you then need to build different Docker images even though their commit/codebase is exactly the same.

Thus, you shouldn’t copy a .env file into your image; you can pass your environment variables to the Docker Container using other ways, e.g. if you use AWS ECS, you can set your environment variables in the ECS Task definition.

Secrets

In a local development environment, it might not be a problem even if you put your secrets (e.g. MySQL passwords, API tokens) in your file.
But when it comes to production environment, the best practice is always: never put your secrets in your code or passing your secrets using environment variables.

You can get your secrets via a secrets management tool such as Vault or AWS KMS.

Introducing secrets management tools would take a full article. Currently, you can see it as a nice-to-have; let’s just pass our MYSQL_PASSWORD via an environment variable. (I will write another article for this! :) )

In the next article, we will take about linting, testing, and Git Hooks. They will save you lots of time and make your development process much better!