Configuration – Part 1

Background
I’ve done quite a bit of consulting over the years. In that time I’ve been asked, more than once, to create a service for hosting shared configuration settings. Technical support folks love shared configuration services because, with them, they no longer need to gain physical access to individual configuration files in order to tweak their production environment. It’s a big time saver and it’s almost always safer than making XML or JSON changes by hand.

To be sure, there are plenty of 3rd party solutions that do that sort of thing, but, like most 3rd party tools, those are usually too big, too expensive, too complicated and almost always designed to force your business to adapt to their software, instead of the other way around. A custom service can be a better approach, at least if you’re willing to spend a bit of time to get things going. A bare bones configuration service is easily a one or two afternoon project. No, really, no more than that for most developers. Of course, adding things like security, fail-over, logging, a GUI, testing, etc, etc, will take longer, but that’s a given. What I usually do is bring up a working core service then I add those other features as needed, and as my development schedule permits.

So, given that I’ve done this sort of project more than a handful of times now, I though I might demonstrate how to put together a working configuration service using common C# tools and a bit of ingenuity. Feel free to follow along, if you like. Let’s get started!

Architecture
My approach usually depends on my customer’s needs but for this article I’ll assume that I should produce a working configuration service that supports a REST interface, and runs on a variety of Operating Systems. To meet those goals I’ll choose C# as my language and .Net Core 2.1 as my framework. I’ll also leverage some existing Microsoft NUGET packages to make the development go a little faster. The product we create should run on Windows, Mac OS, or Linux, and because it supports REST, it should be consumable by a wide variety of clients.

The Project
We’ll start in Visual Studio 2017 by hitting Ctrl+Shift+N to bring up the New Project Dialog:

I’ll name my project CG.Configuration.Host, but you can choose any name that tickles your fancy. The important thing for this dialog is that you choose the ASP.NET Core Web Application project type.

If you don’t have an option for ASP.NET Core Web Application then go into the Visual Studio Installer and make sure the .NET Core cross-platform development option is selected:

Press the OK button on the Add New Project dialog and it should bring up the New ASP.NET Core Web Application dialog:

For a production system you’ll probably want to consider some sort of security, which you can easily add here by clicking the Change Authentication button. For the purposes of this demo though, we’ll leave security off. Make sure the API option is selected and hit the OK button.

The result should be a project structure, in Visual Studio, that looks a bit like this:

We’ll be primarily making changes to the ValuesController.cs and the Startup.cs files. But first, we need to add a few NUGET packages. To do that, start by right clicking on the project, in the Solution Explorer window, and choose the “Manage NuGet Packages …” menu option.

That should bring up the NuGet Package Manager Window. Make sure the “Browse” tab is selected then enter “Microsoft.Extensions.Configuration” in the search window and select that package in the results window, then press the Install button and accept the license.

A bit farther down in that same window, you should see the “Microsoft.Extensions.Configuration.Json” package. Go ahead and select that one as well, then click the install button again.

That’s all the NUGET packages we’ll need, for now. Later, we’ll add another package and we’ll need to eventually write a custom provider for the Microsoft.Extensions.Configuration package, but let’s not get ahead of ourselves …

 

The Code

Now that we have the project created, and the NUGET packages added, we can start actually writing our configuration service. Let’s start by making changes to the Startup.cs file.

Here is a listing of the changes we’ll make to that file:

Most of this code was generated by the Visual Studio template. We added the bit in the ConfigureServices method, that looks like this:

This bit of code registers a service with the .NET Core dependency injector, for the IConfiguration type. That type is used by the Microsoft.Extensions.Configuration NUGET package, and is the part of the reason this configuration service will be so simple to write. So, after we do this, any controller that has a need for an IConfiguration object, at runtime, will have that instance resolved using this bit of registration code. The code itself simply creates a configuration builder, then adds a JSON source file to the builder, for configuration settings. Finally, the IConfiguration instance is created using the Build method on the builder.

The next change we’ll make is to the ValuesController.cs file, which is located under the Controllers folder. That file already contains a working sample controller that was generated along with the rest of the project, by Visual Studio. We’ll replace most of that code with our custom configuration values controller. Here is the code for that:

So let’s walk through the code here. The class constructor accepts a single parameter of type IConfiguration. If you recall, we registered that type in the Startup.cs file. So, when the .NET Core runtime tries to create an instance of ValuesController, it relies on the built-in dependency injector to supply the IConfiguration object for the parameter. We save that value to a field, in the constructor. Later, when a caller asks for a collection of KeyValuePair objects, representing the values of our shared configuration, we’ll simply defer to our IConfiguration instance, and we’re almost done.

The only thing we really have left to do is to create a JSON file to hold our shared configuration values. Let’s do that now. Hit Ctrl+Shift+A key combination to bring up the Add New Item dialog, then scroll down until you see the JSON file option. Give the json file the name of “config.json”, since that’s what we told the builder to look for, in the Startup.cs file. Press the OK button and you should see an empty JSON file. Go ahead and add a few configuration items to the file, so we’ll have something to see when we run the REST controller. It should look like this when you’re done:

The only thing left to do now is to change the properties for the JSON file, so it will be available to the Web Application, at runtime. To do that, save any changes to the JSON file, then go find the config.json file in the Solution Explorer window, select it, then hit Alt+Enter. You should see the properties window for that file.

Change the “Copy to Output Directory” property to “Copy if newer”. That tells Visual Studio to copy the file to the build directory, as needed, which makes it available to the Web Application at runtime.

And now, we’re done. Don’t believe me? Go ahead and run the project. You should see something like this for the output (I use Chrome so you’re output may look different if you use another browser):

So, what have we done so far? We’ve created a REST based Web Application, in .NET Core, that serves up a collection of shared configuration values. Pretty cool, right? It didn’t take us very long to get to this point, just like I promised. At this point our project meets our original requirements. It could do more than this, obviously, but this is a pretty good starting point. Next we’ll add several nice to have features, such as the ability to change configuration values and send out notifications whenever values are changed. Those changes are quite a bit more complicated than anything we’ve done, so far. Don’t worry though, I’ll walk through everything one step at a time.

 

Changing Data

We’ll need to start by altering the code in the ValuesController.cs file. To modify existing values let’s add a Put method to the controller:

This method uses the IConfiguration object to modify the value of an existing setting. Because of the way the Microsoft configuration extensions work, this method will also add a new setting if the key isn’t already in the collection. We’ll leverage that ability for our Post method:

So the implementation of our Put and Post methods are identical, and that’s OK because we’re keeping things simple for the article. Later, we can add some additional validation logic for both methods, but for now, this is good enough.

 

Change Notifications

You may already have anticipated our next little problem. If the shared configuration data is now changeable, and we have clients using those values at runtime, then how do we notify them whenever a value changes? We’ll use SignalR for those notifications, but, before we do, let’s stop and think about how our clients should interact with our service. The choices we make in that regard will have a dramatic effect on the overall usability of our project.

The most obvious way for any client to interact with our service is through a WebClient instance hitting our REST endpoint. After all, we made the REST interface so we may as well use it, right? That  approach works but if we choose that route then any client wanting change notifications will have to do two things: (1) connect to our REST endpoint and, (2) connect to our SignalR endpoint. That little bit of exposed complexity may not seem like much, but, it bothers me, so, I decided to leverage the architecture of the Microsoft.Extensions.Configuration package by writing a custom Provider to wrap up all the messy implementation details. Using my custom provider, any (.NET) client can interact with our configuration library without ever writing a single line of WebClient or SignalR related code. In fact, the code will look exactly like it would if they were simply interacting with the Microsoft.Extensions.Configuration package and reading their configuration values from a local file. That’s important because it means they can use the same code to hit a local JSON file during development and then hit the shared configuration source in QA or testing. How will I handle all of that? Let’s look.

New Provider

To make the new provider more usable let’s add it to another Class Library project. To do that, use the Ctrl+Shift+N key combination to bring up the New Project dialog.

Make sure the .NET Standard project type is chosen. Then, make sure the Solution dropdown contains “Add to solution”. Finally, name your new Class Library appropriately. Here I’ve named mine “CG.Configuration.RestApi”. Press the OK button to add the new project to your solution. Once that’s done. You have a standard empty project with an empty Class1.cs file. Delete that file, since we won’t be needing it.

Our new project will also need the “Microsoft.Extensions.Configuration” NUGET package added, so go back into the NuGet Package Manager, for our new project, and add that package now. While you’re at it, also add the SignalR package that we’ll eventually need: “Microsoft.AspNetCore.SignalR.Client”. Take note that there are multiple flavors of SignalR client packages available. Make sure to add the right one, or the notifications may not work correctly. Finally, add the “Newtonsoft.Json” package as well, since we’ll need that to work with JSON data.

Next, for our custom provider to work, we’ll need to create five new source files:

  1. A provider class, which we’ll put into a file named RestApiConfigurationProvider.cs
  2. A source interface, which we’ll put into a file named IRestApiConfigurationSource.cs
  3. A source class, which we’ll put into a file named RestApiConfigurationSource.cs
  4. An extension class, which we’ll put into a file named ConfigurationBuilderExtensions.cs

Adding each file is as simple as using the Ctrl+Shit+A key combination (with the provider project highlighted, not the web application project), and then choosing the right name and type, each time, in the Add New Item dialog. Afterwards, the project should look like this:

Let’s start with the source interface, which should be modified to look like this:

We’ll need these two properties, Url and Port, to make the SignalR connection back to our configuration service. Next, modify the source class like this:

This class is mostly just an implementation of the IRestApiConfigurationSource interface, but there is one point of interest: the class has a Build method to it, this is how the Microsoft package relies on the concrete source object to create an associated provider instance at runtime. In our case, the method simply returns a new instance of our RestApiConfigurationProvider class.

Before we look at the code for the provider class, let’s quickly look at the code for the extension class. That code looks like this:

This little bit of code simply adds a RestApiConfigurationSource to the builder and then returns that source instance to the caller, so the caller can then set any properties needed, for configuration purposes. Placing this code in an extension class isn’t strictly required but it is cleaner, in that in removes those few lines of code from the caller’s code, and replaces it with a single new method that shows up on the builder.

So, let’s look at the code for the provider class now. That code should be modified to look like this:

This is, by far, the most complicated part of the entire project, so far. Let’s go through the code one method at a time.

We’ll start with the constructor. It supplies the source reference from the Provider and stores that object in a property named “Source”. We’ll need that reference later to get at the “Url” and “Port” settings in other methods. Next, the constructor creates an address for the SignalR connection, using the properties gathered from the source. Then it creates a hub builder and plugs that address in before building a SignalR hub object and storing it in a property named “Hub”. Finally, the constructor wires up a handler for the “notify” method, that will reload the provider’s internal data whenever it receives a poke from the configuration service, via the SignalR connection. Lastly, the SignalR connection is started so the provider is actively listening for notifications of changed configuration values.

The next method is the Load method. This method is responsible for loading an internal data collection with configuration values. It uses a WebClient connection back to our configuration REST service to get those values. It starts by building the address to our configuration service. It then creates a WebClient instance and calls the DownloadString method on that web client to get the list of configuration values, as JSON. This is the same data we say in the browser, when we started the configuration REST service. Once the JSON data is downloaded. The method adds each key-value pair to an internal collection it inherited from the ConfigurationProvider class, named “Data”.

The last method is the Set method. This method is called whenever the provider tries to either insert a new value or modify an existing value. We start by opening a WebClient connection to our configuration REST service, then we serialize the new or modified key-value pair into JSON and send it to our POST method, on our REST controller. For now, we send everything as a POST. Later, we could add some intelligence to decide whether to call POST or PUT on the underlying service.

That about it for the provider. Remember, it’s job is to leverage the existing Microsoft.Extensions.Configuration NUGET package, by interfacing with our REST service internally, while making it seem like nothing unusual is happening to the client. That means, for .NET clients anyway, that they’ll only need an IConfiguration instance to interact with our configuration REST service and they’ll never need to know anything at all about WebClients or SignalR hubs.

We’ll write a simple client application and demonstrate how to use this custom provider in a bit. For now though, let’s go back and make sure our REST service can send out SignalR notifications.

 

Adding SignalR to the service

So, we’ve added a SignalR backchannel for notifying the world whenever we modify a configuration value through our REST service. The next step is to add the same SignalR support to our original REST service and make sure it’s sending out notifications correctly. Let go do that now.

Start by adding a SignalR NUGET package to the original REST web application project. Make sure you have that project selected then bring up the NuGet Package Manager and add the “Microsoft.AspNetCore.SignalR” package. Again, just like with the SignalR client, there are lots of different SignalR flavors so make sure you add the right package or things might go awry.

The next step is to make a SignalR hub class. Start by selecting the REST project in the Solution Explorer Window, in Visual Studio, then hitting the Ctrl+Shift+A key combination to add a new class. Call it whatever you like but I called mine “NotifyHub”. The code for that class should look like this:

The class has a single method, named “SendAsnc”, that is used to fire off the “notify” notification through SignalR.

The next step is to modify the code in the Startup.cs file. Here is the way that code should look after adding support for SignalR:

The last line is what we added. It registers SignalR types with the .NET Core dependency injector.

The next change looks like this:

Again, the last line is what we added. That registers a route for SignalR use, and tells SignalR to use our custom Hub whenever it handles requests on that route.

The next change is to send out notifications whenever we change the configuration data through our REST interface. To do that, we need to add a small change to the ValuesController.cs file.

Here are the changes to that file:

First, we add a property to contain a SignalR hub context. We rely on the .NET Core dependency injector to make that object for us, and it responds by injecting it as a constructor parameter – just like it was already doing for the IConfiguration object. In the constructor, we store the parameter in the property, for later use.

Next, we modify the POST and PUT methods on the controller:

The added line of code is that which sends out the call through SignalR. Now, whenever we modify the shared collection of configuration values, SignalR will, through our custom provider, notify our clients that the change has taken place. Client can then respond by reloading their local configuration settings and possibly even restarting themselves.

 

Example Client

So far, we’ve built a REST based web application that serves up a shared collection of configuration values, gives clients the ability to modify those configuration values, or add new configuration values, and sends out notification values whenever a configuration value is changed through the REST interface.

But, we’ve yet to demonstrate how to actually all of these new gadgets in a client application. Let’s do that now.

Start by hitting the Ctrl+Shift+N key combination to bring up the New Project dialog. Make sure the Console App (.NET Framework) project type is chosen. Then, make sure the Solution dropdown contains “Add to solution”. Finally, name your new project appropriately. Here I’ve named mine “CG.Configuration.ExampleClient”. Press the OK button to add the new project to your solution.

Once you’ve done that, select your new project in the Solution Explorer window, in Visual Studio, then open the NuGet Package Manager window and add the “Microsoft.Extensions.Configuration” and “Microsoft.AspNetCore.SignalR.Client” packages to the new project.

The next step is to add our custom provider project as a reference. To do that, right click on the example project in the Solution Explorer window, then choose the “Add Reference” menu choice.

That brings up the Reference Manager dialog. Select the “Projects” tab, then check the CG.Configuration.RestApi project (or whatever you called it in your project). Afterwards, hit the OK button to add the reference.

Now we just need to add some example code to our project. Let’s modify the Program.cs file like this:

This code creates a cancellation token source and uses it, along with a cancellation handler on the console, to cancel a waiting task whenever the caller presses the Ctrl+C key combination. That’s got nothing to do with demonstrating our configuration code, it’s just to get the sample application to run reliably.

The code we’re interested in is the bit that creates the ConfigurationBuilder object, then uses our AddRestApi extension method to inject our custom source. That way, when the builder’s Build method is called, the result is an IConfiguration object that internally contains an instance of our custom RestApiConfigurationProvider class. If you want to know how Microsoft’s builder works, and how it goes about creating an instance of the custom provider, look back at my Builder related articles. My builder is a bit difference than Microsoft’s, but, they are both general variations on the classic “Builder” design pattern. Reading about my Builder should give you a good start at understanding Microsoft’s Builder.

After the IConfiguration instance is created, we have a block of code that looks like this:

What this is doing is, wiring up two handlers that will be fired whenever the Microsoft configuration object detects a change to it’s underlying data – that means the data in our custom provider, the data it got from our REST service. What happens is, the configuration library maintains a change token that is fired whenever the underlying data is changed. That code is part of the Microsoft library. It isn’t mine. I did, however, happily leverage it here to propagate any and all change notifications from my custom provider to any clients who might care about it.

So, when the Microsoft library fires a change token, this handler tells it to do two things: (1) it reloads the change token so it will work with the next change, and (2) it calls some method in our example code – which in our case simply dumps the value of a configuration setting to the console. In a real production project, that code would probably refresh any local configuration values and, possibly, restart the project.

Note that the code for the “Dump” method simple writes the value of the “SampleSection:Key1” configuration key to the console. However, it will do that each time the value is modified in the REST controller. That’s not very useful but it does demonstrate an end-to-end notification from REST controller to client, all without the client having to know anything at all about WebClient or SignalR code.

 

Wrapup

This article is longer than I had anticipated it might be. I have more to say about the topic but I think I’ll leave it for a second article.

So next time, I’ll demonstrate how to run the example project and show off the notifications through the SignalR backchannel. I’ll also demonstrate how to write another custom provider to deal with saving value changes to a backing store (the Microsoft library is designed to be lightweight, so by itself, it doesn’t write changes to disk). Finally, I’ll talk about the missing Delete functionality, which isn’t support by the Microsoft library, but which I’ve added myself with a simple workaround.

Extra points to any readers who pickup on on the missing Delete functionality …  :o)

See you next time.

 

The project for this article can be downloaded HERE: CG.Configuration.Host