CI / CD Pipeline – Part 2

Photo by Martin Brechtl on Unsplash

In this series of articles I am discussing the CI / CD process I use to automatically build and deploy any changes I make to my various NUGET projects. We finished last time by creating a Visual Studio .NET Standard project for our NUGET package, then creating a GITHUB repository for that project. Finally, we connected the GITHUB and Visual Studio project together so that we can track changes to the project, later on.

This time, we’ll continue by choosing a tool for a build server and then connecting that tool to our GITHUB repository.

There are tons of choices out there, for a build server. I personally use APPVEYOR because it integrates well with GITHUB, is free (for open source projects, like mine), and is fairly easy to work with. If you decide to use APPVEYOR as well then you’ll need to start by creating an account on that website now. If you already have an account, you’ll want to log in.

I’ll assume that you’re now logged into your APPVEYOR account and are sitting at the projects page. Hit the “New Project” button:

AVNewProject

That takes you to a page that lists all your GITHUB projects. Choose the one that we want to build and click the “Add” button:

AppBuild

That creates a new default build for the project and directs you to the build page for that project:

nobuild

From this point the APPVEYOR build exists and is connected to the GITHUB repository, but, it’s not really configured to do anything terribly useful. Certainly, we’ll need to change a few things to get where we want to be, which is, to have a build that’s specific for our project and will deploy our package to the NUGET site. We could use the very nice settings tab on this site:

AVsettings

This is the page I used to configure my first project with. It allows for a ridiculous number of configuration options. My only problem with this approach is that we can’t version control anything we change here. That fact has implications for creating a repeatable build, later on. Instead, at this point, I usually add a .yaml file to my Visual Studio project and then check everything into GITHUB. That way, if I need to make changes to my build parameters at some point in the future, that change will be properly version controlled, just like the rest of the project.

So, let’s go back to Visual Studio, but keep the APPVEYOR page open in your browser. Select the solution name, in the Solution Explorer window, in Visual Studio. After that, choose the “Add”, then “New Item” menu items. Untitled-2

Once that’s done, it should display the “New Item” dialog

appveyoryaml

Choose the “Text File” type, then name the new file “appveyor.yaml”, then click the “Add” button.

The result should be a new file named “appveyor.yaml”, in a folder named “Solution Items”in your Visual Studio solution. The file itself is still blank so let’s add something to it. Add this bit of YAML:

version: ‘{build}’

image: Visual Studio 2017

branches:

only:

– master

configuration: Release

before_build:

– cmd: before_build.cmd

build:

project: CG.Walkthrough.sln

verbosity: minimal

artifacts:

– path: .\CG.Walkthrough\bin\$(Configuration)\*.nupkg

name: nuget-packages

deploy:

– provider: NuGet

api_key:

secure: [your API key here]

artifact: nuget-packages

Your project name might be different than mine so use your name in place of mine. Also, the api_key I’m showing here, under api_key, secure, section, will need some attention before it will work for us. This needs to be the encrypted API key from your NUGET account. We’ll need to encrypt that information before we add it here, since anyone can look at the contents of this file once we save it to GITHUB. Let’s go look at that part next.

You’ll need an account on NUGET.ORG for this next part. If you don’t have an account go there now and create one. If you do have an account, go ahead and login now.

Once inside your NUGET account, look for your account name at the top of the page, select it with the mouse, then choose the “API Keys” menu option.

ng-apikeys

That will take you to a page with your API key(s) on it. There will be an option to create a new API key. Choose that so it open up, revealing all the various options for API keys.

ngapikey-create

Some of the options will depend on your needs, but, certainly you’ll want to name this API key … Something related to your CI / CD pipeline might be a good choice. For my needs, I chose the “Push” option, then I chose the “Push new packages and package versions” option, then I entered a glob pattern of “*”. Finally, I clicked the big blue “create” button.

Once you do that, you’ve created a key that can be used by your APPVEYOR build to push packages to your NUGET account. That way, you don’t have to deploy the packages manually, every time you make a change to your project. The new key is still in the NUGET website though, so let’s go get a copy of it.

Open the “Manage” section of the page and you should see your new API key listed:

managekeys

Next, click the “Copy” button , to copy your raw API key to your clipboard. Once that’s done, we need to head back over to the APPVEYOR site, to encrypt this key so we can safely use it in our appveyor.yaml file. So, bring up your APPVEYOR site again now. Click on your account name, at the top of the page, then select “Encrypt Data” from the menu:

blah

That brings you to a page where you can encrypt sensitive data, for use in your yaml files:

blah2

This step is important, by the way. If you don’t perform this step, and simply copy your raw NUGET API key to your yaml file, you’ll end up giving anyone who finds it, a key to do very bad things to your NUGET packages, or even to your NUGET account.

Once you’re on this page, past the raw API key from your clipboard into the field named “Value to encrypt”, then press the big blue “Encrypt” button. After you do that, your encrypted API key is available in the field marked “Encrypted value”.

Highlight that encrypted key value and copy it to the clipboard. Then, go back to Visual Studio and paste that encrypted text into the yaml file, in the api_key, secure, section, where I had the text [your api key here]:

version: ‘{build}’

image: Visual Studio 2017

branches:

only:

– master

configuration: Release

before_build:

– cmd: before_build.cmd

build:

project: CG.Walkthrough.sln

verbosity: minimal

artifacts:

– path: .\CG.Walkthrough\bin\$(Configuration)\*.nupkg

name: nuget-packages

deploy:

– provider: NuGet

api_key:

secure: UNA9vyx74GLzFtfSt4v/Kk6FWGRp5esS4kH7FY044JAlmlRu468rZgvdBVHV4d9K

artifact: nuget-packages

The result should look like this, except the actual value should be your API key, not the one I’ve shows here. This key is for demonstration purposes only.

Let me explain a bit about the purpose of this yaml file. This file contains all the APPVEYOR settings that we could have chosen using their web interface, but, instead we’re going to supply all our options using this yaml file instead. We’re going to do it this way because the yaml file can be included in the GIT repository and versioned, with comments, so we can properly track any build related changes, later on. APPVEYOR will look for this file in the solution root, which is why we added it to the “Solution Items” folder, and not directory to our CG.Walkthrough project.

Next we need to go back into Visual Studio again, right click the Solution Items folder, choose the “Add” and “New Items” menu choices again, and this time, choose a “Text File” type, with a name of “before_build.cmd”. Hit the “Add” button to add that file to the solution.

beforebuildcmd

Once we have the “before_build.cmd” file added, we need to edit it and add the following text:

nuget restore

Let me explain this line of script. In order for APPVEYOR to properly build the solution on one of their build servers, they first need to know about any NUGET packages that the project might need. Now, for our project, we don’t need any NUGET packages, but, the MSBUILD engine that APPVEYOR uses to build with won’t know that until we do a NUGET restore operation. That restore operation creates a empty “project.assets.json” file that the MSBUILD engine uses to load any NUGET packages with, before the build operation starts. So, by telling APPVEYOR to perform a nuget restore, we’re also ensuring that the underlying MSBUILD engine will have all the files it needs to build our project on the APPVEYOR build server.

Now that we’ve edited a yaml file and a cmd file, and made them both part of our project, we need to check all the changes into GITHUB so we can kick off a new build. Let’s do that now. Go back to Visual Studio and open the “Team Explorer” window, click the “Changes” tab and you should see something like this:

changes

Add a comment for the checkin then press the big grey “commit all” button. You should see an informational popup with the number for the GITHUB commit:

checkinnumber

There’s only one thing left to do now, which is to sync our remote repository with the one on the GITHUB site. To do that, click the “Sync” link on the informational popup. Afterwards, you should see the following screen:

sync

From here, click the link under “Outgoing Commits”, labeled “Push”, to push our changes to the server (yes, it really does require us to jump through this many hoops to check in a change …)

After we get to this point, let’s head back to the APPVEYOR site. Don’t worry, the switching back and forth is almost over …

When we checked our changes into GIT, through the Visual Studio UI, it informed APPVEYOR, behind the scenes, that a new version was ready to be built. APPVEYOR then pulled that version down and built it on one of their build servers. It looked something like this:

AVBuild

You can see the results of your own automated builds by going to APPVEYOR, then clicking the “Builds” tab, then choosing your project from the list. You’ll see something that looks like the screen shot above.

We still have one last website to go look at. We want to make sure that the APPVEYOR build actually published our NUGET package to our NUGET account. To verify that, log into your NUGET account, click on your name at the top of the page, then choose the “Manage Packages” menu choice.

publishedpackages

You should see something like the image above.

Final Thoughts

So what have we done while jumping through all these hoops? We have created a Visual Studio project that builds a NUGET package (Admittedly, our class library doesn’t do much but then again, this is just a walk through). We have created a GITHUB repository and connected that repository to our project so that we can track any changes to it, over time. We have created an APPVEYOR project and connected it to out GITHUB repository so that our project will be automatically built anytime we check in changes. Finally, we have created a NUGET page and connected it to our APPVEYOR build so that any NUGET packages that are automatically built will also be automatically deployed and made available to anyone on the interwebz.

That’s a complete CI / CD pipeline and all it took from us was a bit of flipping back and forth between browser pages and Visual Studio, and, of course, creating a few web accounts along with way.

So what else can we do with our pipeline? Stick around because I plan to add some things to our CI / CD pipeline that will enhance it’s utility and hopefully demonstrate how easy it is to customize things to fit your specific needs.

See ya next time.