We are going to explore the reasons why Pandora exists and what problems we could solve with Pandora. You will see one fragment of an application settings done with Pandora so you could get a general feeling about it.

Introduction

In my professional career as a developer I have been involved in applications written primarily in c#. Almost every application used standard configuration settings stored in web.config or app.config files. SlowCheetah was a very good tool which helped a lot with XML transformations of the *.config files by generating settings for the different environments such as DEVELOPMENT -> TEST -> STAGING -> PRODUCTION. Eventually all these good things were integrated in Visual Studio. However inside these files we often put sensitive data like passwords, database connection strings or third party application’s account which are exposed to all people who have access to the source code. It is just not a good idea to give all that production power in the hands of a developer who just joined the team and checked out the source code. Here is a MSDN post which probably addresses this issue.

Another big struggle which I have had during the years was the pure management of the settings. Sometimes I have just missed one connection string in the abundance of all *.config files.

For small applications using the tools which .NET provides for application settings out the box is sufficient and practical. It is very easy to create some advanced configurations using CI tools as well. However, when your application grows the settings management could become one really annoying problem to deal with every day.

Some applications store the application settings directly inside the application’s database which for me personally is a big NO NO NO. Like, should I create a migration script for this settings? Ohh, if I do this the setting probably will be stored in the source code with all migrations. So we just need to create an UI and not forget to change the setting after every deployment. Nah, lets hit the production database directly and execute this SQL statement… :)

I am sure that I am not the first who faces this problem and probably there are very creative solutions which address this problem. One which I really like is Pandora

Pandora

When I started Pandora with my friends there were few design concepts which we wanted to implement:

• Easy to use/write configurations
• Separate the application and its settings
• Schemaless setting keys
• Reduce duplicate configuration settings
• Support for multiple environments
• Support for distributed applications
• Security and user/role based settings

Lets see a concrete example with an imaginary application called NMX. It is a chat application with the cool feature to restrict the number of private channels a user may create. For example, if I buy a basic package of NMX my users could start up to 5 private channels. There are 3 available packages: Free, Basic, Enterprise. This should be enough from business perspective. On the technical side the production environment consists of a cluster of 3 machines. On every machine we host a website and a service. This means that in the source code we have at least 2 project each of them with a configuration file. So we should have at least two *.config files which look like this:

• website: web.config

<?xml version="1.0"?>
<configuration>
  <connectionStrings>
   <add name="db" connectionString="10.10.10.10/prod/db" />
  </connectionStrings>
  <appSettings>
    <add key="channel_package" value="basic" />
  </appSettings>
</configuration>

• service: app.config

<?xml version="1.0"?>
<configuration>
  <connectionStrings>
   <add name="db" connectionString="10.10.10.10/prod/db" />
  </connectionStrings>
  <appSettings>
    <add key="tenant_specific_key" value="Client1 value" />
    <add key="channel_package" value="basic" />
  </appSettings>
</configuration>

This looks fine, we could apply some transformations if we have a second environment… Wait! We already have a second one. The DEVELOPMENT environment is very important and it should be very flexible, keep it healthy. Now it becomes a bit harder to introduce a TEST environment (3rd) but with some advanced techniques described here and here you could do it. When teams start to have these problems they were solving them in many different ways. I have seen in many places developers to share resources, trying to work around this issue, such as database during development however I think this leads to a lot more problems slowing down the feature development and team growth. Also note the key tenant_specific_key. Here we could have different clients using our NMX application and somehow we need to provide client specific values. May be there are better places to put such configurations such as inside client’s database and let the client configure it through the UI but this is not always the case.

Probably you already noticed but there are some configuration files missing for the different environments and clients. Unfortunately you have to use your imagination what files and what tools you need in order to support such software development environment.

Now, lets see the same a configuration with Pandora:

• website: website.json

{
    "name": "website",
    "references": [
        { "jar": "db.json" },
        { "jar": "channel_package.json" }
    ],
}

• service: service.json

{
    "name": "service",
    "references": [
        { "jar": "db.json" },
        { "jar": "channel_package.json" }
    ],
    "defaults": {
        "tenant_specific_key": "available only in website"
    },
    "clusters": {
        "development": {
        },
        "test": {
            "tenant_specific_key": "available only in website for QA",
        },
        "client1": {
            "tenant_specific_key": "available only in website for Client1",
        },
        "client2": {
            "tenant_specific_key": "available only in website for Client2",
        }
    },
    "machines": {
        "developer1": {
            "cluster": "development",
            "tenant_specific_key": "hacked"
        }
    }
}

• service: db.json

{
    "name": "service",
    "defaults": {
        "db": ""
    },
    "clusters": {
        "development": {
            "db": "local.com/test/db",
        },
        "test": {
            "db": "1.1.1.1/test/db",
        },
        "client1": {
            "db": "10.10.10.10/prod/db",
        },
        "client2": {
            "db": "20.20.20.20/prod/db",
        }
    },
    "machines": {
        "developer1": {
            "cluster": "development",
            "db": "docker:8888/dev/yessssss"
        },
        "developer2": {
            "cluster": "development",
            "db": "GG"
        }
    }
}

• service: channel_package.json

{
    "name": "ChannelPackage",
    "defaults": {
        "channel_package": "free"
    },
    "clusters": {
        "client2": {
            "channel_package": "enterprise"
        }
    },
    "machines": {
        "developer2": {
            "channel_package": "enterprise"
        }
    }
}

Here we go. Try to read this configuration without any hints. Let me know which parts of these json files look confusing to you and why so that we could work together to make it better.

What is next

In the next posts we will learn more about Pandora, how to use it, how it works internally.

Software is fun! Happy coding!