Home .NET Updating the CI/CD process: teamcity

Updating the CI/CD process: teamcity

by admin

Updating the CI/CD process: teamcity
This is the second article in a series on updating CI/CD processes.Up to this point, preparations were made to set up the new tools, namely planning, daily rallies, resolving disagreements, in general, everything without which you can’t build a proper workflow. And now, all issues are taken care of, we are sure that we can continue development and our code will not fall into a black hole at the beginning of summer.
Let me remind you of the table of contents of the list of articles, as well as a brief summary of the previous part.
Part 1: what is, why don’t like it, planning, a little bash. I would call this part near-technical.
Part 2: teamcity.
Part 3: octopus deploy.
Part 4: Behind the scenes. Troubling moments, plans for the future, maybe an FAQ. Probably could also be called near-technical.
Provided the customer with the proof-of-concept for the new update delivery system, pointed out the shortcomings of the existing system, chose the basis for the new system, made a migration plan, and converted our mercurial repository to git. Also, the previous installment outlined features of the project that would affect the entire process.
As mentioned earlier, the existing solution was not up to date. Initially, there was probably one build set up in ccnet. The very first build that started it all (like a big bang, yeah). Then, apparently, the person who set it all up, pressed ctrl+c, ctrl+v, tweaked a couple of lines, and got a new configuration. And all subsequent time, it was done enough times that the sources on the build server took up more than 60G. And that’s just the sources! And then there are logs, build artifacts, temporary files, and so on. This alone made me think. Looking at the old system we could see the following: for each module the build step was performed. In fact the same kernel was built. And this build was stored for each module (together with sources) on the server. The build results (90% identical) were flushed into local repositories (git deployment), and naturally also took space. This was something we could and should have gotten away from.

General settings

At this point it is assumed that teamcity and octopus are already configured. We do not stop at the installation and configuration stage in detail. For convenience, we write the url and api-token of octopus in the teamcity env, which, by the way, for the root project looks like this :
Updating the CI/CD process: teamcity
env.apiUserName and env.apiUserPassword are accesses to api user teamcity (will be needed later), env.buildPath and env.sourcePath – are the default paths to the build and source files, respectively, env.octoApiKey and env.octoUrl – token and url of the octopus. To these also. env.teamcityUrl added. Well, and env.tt.exe – Path to text transform utility. This is the utility that generates all the configurations.
From third-party plugins, only Octopus Deploy integration is installed.
In order not to overload the already enormous amount of text, only those steps in which there is something worthy of attention will be discussed in detail. It is assumed that the rest is clear from the data presented and the reader’s knowledge. Anyway, if you have any questions, I’ll be happy to answer them.

Partitioning into projects, versioning, names

As a result, we have the following scheme of projects (by image, because there is a problem with displaying the multi-level list):
Updating the CI/CD process: teamcity
We will version the packages as follows: major.minor.patch, where major is still 1, minor is the build counter of the Build configuration, patch is the build counter for the Deploy configuration (we don’t need it for the Build configuration, so it is always 0).
Note : This describes the old versioning. We are now redoing it so that it is based on git tag.
Updating the CI/CD process: teamcity
For example, here 1 is major, 95 is minor, and 1 is patch.
In the new process for the modules will be a single build, which will then be deployed to each module. That is, the sources are stored in a single instance for each environment, the result of the build, too, as it is common to all modules. Unique configurations and libraries are packaged in an individual package during the deplug phase. This will make efficient use of disk space and reduce the delivery time of each package.
There was a separate short rally on the naming convention of configuration files in the preparation phase. It used to be (just was), but not always the same. In addition, the name of the configuration did not always contain enough information. As a result, all names of configurations had the following form :
ConfigurationType.Environment.ModName.config where ConfigurationType can be AppSettings, Web, etc. Environment – development, test, staging, production. ModName – unique name of the module.


All Modules projects work on the same principle.
Every environment has a Build configuration that :

  1. Performs nuget restore (NuGet installer, nuget restore)
  2. Generates configs (Command Line, text transform)
  3. Buildit Solution (Visual studio sln)
  4. Takes files from env.buildPath to zip and flushes it to Octopus (Octopus deploy: push packages)
  5. Creating release (without deploying) (Octopus deploy create release)
  6. Resets patch versions (powershell)

There are also deploy configurations that work based on the pattern and do the following :

  1. Takes the configs that were generated during the build stage (step 2) (Octopus deploy push packages)
  2. Core release deplot (which was generated in step 6 of the build) (Octopus deploy: deploy release)
  3. Deploy individual release (which was created in step 1 of the current configuration) (Octopus deploy: deploy release)
  4. Displays a list of all domains for this module in the log (step for tester and developer convenience). (Powershell).

For each deploy configuration, a build configuration has been added as a dependency. This gives you the ability to automatically start a build if it is not up-to-date, as well as the ability to reset patch versions when you start a deploy.
Deploy_all configuration. This is actually a build chain which describes that you must first run the build if it is not up to date and then deploy all mods in parallel.
It looks something like this :
Updating the CI/CD process: teamcity
Let’s break down some of the steps in a little more detail.

Build.General settings

Updating the CI/CD process: teamcity
Only the build number format is unusual. To be explained later.


Updating the CI/CD process: teamcity
Of interest here are :
env.coreProjectName – Is set for the entire project. Needed for project identification in octopus.
env.Environment – Set for Modules subproject. Needed to select the correct configs depending on the environment.
env.octoChannel – The channel in the octopus into which the packet arrives. Matches the env.Environment.
env.serviceName – is the name of the solushen. Basically, the variable is added for easy visualization.
env.tenantRoleTag – The tenant tag in octopus. See octopus for details.


Updating the CI/CD process: teamcity
Add to the archive with the name %env.serviceName%.%build.number%.zip files from the build result directory, while excluding all configurations, and dlls from the bin/native folder. The latter were manually added to the web servers once and no one will touch them again. If necessary, they will also update them manually (for this plan to write a separate script, but let’s be honest). This is due to the fact that these libraries "hold" the pool of IIS, and for a successful deplaya it must be stopped and then run, which takes some time. By excluding these libraries from the deplot, we can skip the step of stopping the pool, which will reduce the deplot time and, accordingly, the down-time.


Updating the CI/CD process: teamcity
I think everything is clear here, except why the Environment field is empty. Which env a packet goes to, the octopus decides based on the pre-release tag (configured in the channels). This tag is contained in the %build.number% (the same -staging on the screenshot in Build.General). This was more convenient in this case. You should also pay attention to the checkbox "Show deployment process". If it is not checked, Timcity considers the build as successful as soon as the octopus started (but not finished!). That is, if the checkbox is not set and something went wrong at the stage of deployment, we will not know about it without looking into the interface of octopus.


Nothing unusual, just powershell script and information from teamcity documentation.

$pair = "%env.apiUserName%:%env.apiUserPassword%"$encodedCreds = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes($pair))$basicAuthValue = "Basic $encodedCreds"$depUri="%env.teamcityUrl%/app/rest/buildTypes?locator=snapshotDependency:(from:(build:(id:%teamcity.build.id%)), includeInitial:false)"# Get all dependencies of main build$result = (Invoke-RestMethod -Headers @{'Origin'='http://localhost:8090'; "Authorization"="$basicAuthValue"} -Uri $depUri)foreach ($i in $result.buildTypes.buildType.Count) { $buildId += $result.buildTypes.buildType.id }$buildId = $buildId | Where-Object { $_ -ne 'Module_DeployAll' }# Reset all child build counters to 1. This build counters are patch versions in every build. (1.build.patch)for ($i=0; $i -lt $buildId.Count; $i++) {$buildCounterUri = "%env.teamcityUrl%/httpAuth/app/rest/buildTypes/"+$buildId[$i]+"/settings/buildNumberCounter"Invoke-RestMethod -Method Put -Headers @{'accept'='text/plain'; 'Origin'='http://localhost:8090'; "Authorization"="$basicAuthValue"} -Uri $buildCounterUri -ContentType "text/plain" -Body 1}

Here we need an api user in timcity (that’s who the envs are created for). We log in, take all configurations which depend on Build configuration and reset their build counter to 1. That is, in fact, for each deployment patch version is a measure of how many times the same build has been deployed to a particular module. If it’s not 1 then something went wrong with the process of deploating or running the module on the current version and needs attention.


Updating the CI/CD process: teamcity
Version Format : 1.%dep.Module_Build.build.counter%.%build.counter%-staging where
%dep.Module_Build.build.counter% – teamcity system variable whose value corresponds to the build counter of the corresponding Build configuration (must be added in dependencies). In fact here 1 is the major version, %dep.Module_Build.build.counter% – minor, and %build.counter% – patch. staging is a pre-release tag.


Updating the CI/CD process: teamcity
When a new module is added, only the configuration for its deplot is added and the value of the variable env.modName All other variables are inherited from parent projects. Variable env.tenantModTag for the octopus is generated from env.modName


Updating the CI/CD process: teamcity
Deploy core package.
Release number latest – use the latest available release for this environment.
Tenant tag – client/organization tag. See section octopus for details.
Additional cli parameters. Here we specify the channel our packet will go to and additional parameters. This will also be explained in the octopus section.


Similar to Deploy.2, only the individual module package is uploaded.


This step will also be detailed later, because it gets the data from octopus.
Configurations for Company website and Special module are built on the same principle, and the main points in them are the same, so I do not see the point in describing them in the same detail. They are the same, you don’t need to deploy them many times, and in fact there are nuget restore, text transform, msbuild, octopus push, octopus create release + deploy.
More about unique configurations. For some mods may not be some environments (eg, staging), they have to be deployed to other servers, or you have to manually set the client ID (take the base config and change some fields in it using powershell). That is, they work on the same principle as the basic modules. Their uniqueness is that for them the steps are added/changed or the server changes and they do not fit in the template, so they take a little more time to create. Fortunately, there aren’t many of those.

Teamcity: Results

Introducing teamcity allowed us to build the application in a more optimal way. It now takes much less time: instead of building the same code every time, we build it once. We also use disk space, network traffic and computational time more optimally. Previously, the number of packages to be deplocated (repositories with artifacts) was equal to the number of modules. Each package weighed about 100M in compressed form. Now we have the number of packages one more than the number of modules, with one package weighing about the same 100M and the rest about 500K. Plus a friendlier interface, easy to use logs, and most importantly, less time to maintain the build system and add configurations. Now adding a new module takes from 15 minutes to half an hour.
It is worth mentioning about the templates, because it is the templating allows us to save time on creating new configurations and their subsequent maintenance. All configurations for interaction with octopus (deploy) are based on one template. Firstly, it allows to unify and therefore simplify the process. Secondly, as already mentioned, it allows you to save time on adding new configurations: connect a template, change one variable, done. And most importantly, templates simplify maintenance. All changes made in the template are inherited by the configurations based on it. That means if we need to add a step to all configurations at once, we don’t need to click through them all, just add that step to the template.
Naturally, the teamcity setup was done in parallel with the octopus setup. It’s just not possible to describe it in a text in parallel, because there’s a great chance of confusion. Therefore, the description was split and in this part only the steps that provide CI are described. The process of setting up octopus will be described in the next one.

You may also like