Microsoft DevOps FastTrack & Azure DevOps Migrations

ethan-weil-262745-unsplash

While writing this post Microsoft has re-branded VSTS (Visual Studio Team Services) towards Azure DevOps. I have reflected the new name in this post so that it is up-to-date with latest naming and documentation references.

Introduction

Recently I have completed a very nice project and it finished with a migration weekend brining several TFS collections towards Azure DevOps. This writeup helps me share my experiences with running the DevOps FastTrack Program as well as my approach on migrating from TFS to Azure DevOps.

Microsoft FastTrack Program

The Microsoft FastTrack (DevOps Accelerator) program is a program for customers that qualify. The FastTrack program consists of a two-week engagement, run by Microsoft selected and Microsoft trained consultants. Together with Rene van Osnabrugge and myself Xpirit has two!

Continue reading “Microsoft DevOps FastTrack & Azure DevOps Migrations”

Provision a VSTS Agent using an ARM Linux Custom Script Extension

There are many ways to get VSTS Agents deployed to a machine. You can find more on that here: https://docs.microsoft.com/en-us/vsts/pipelines/agents/agents?view=vsts. In this post you will find a way to deploy a VSTS Agent on a Linux Azure VM through an ARM template. For this we use a Custom Script Extension.

I intentionally left out the creation of the Linux VM in this post. I used a Packer script for this while my collegue Manuel Riezebosch created a very convenient VSTS Task for that! See this here: https://marketplace.visualstudio.com/items?itemName=riezebosch.Packer

To deploy the agent a couple of steps are involved;

  1. Get the download URL for the agent; blogged here: https://wp.me/p34BgL-81
  2. Encode a Linux script, to install the agent
  3. Use a Linux ARM Custom Script Extension in your ARM template

To create the encoded script I used another Inline PowerShell Task in VSTS. The full script can be found here: https://github.com/JasperGilhuis/VSTS-RestAPI/blob/master/Get-EncodedAgentDeployScript-Linux.ps1

To clarify the details I expanded the script a bit:

1. curl -s $(AgentDownloadUrl) > /tmp/agent.tar.gz;
2. for i in `seq 1 $(AgentsPerVM)`;
3. do mkdir /agent$i &&
4. cd /agent$i &&
5. tar zxf /tmp/agent.tar.gz -C . &&
6. chmod -R 777 . &&
7. sudo -u $(AdminUserName) ./config.sh --unattended --url $(VSTSAccount) --auth pat --token $(PersonalAccessToken) --pool $(AgentPool) --agent $(AgentName)$i --work ./_work --runAsService &&
8. ./svc.sh install &&
9. ./svc.sh start;
10. done;

The following lines comment the script above on line by line basis;
1. Download the VSTS Agent, save in the tmp folder
2. Loop for the desired number of agents
3. Create the agent directory
4. Go to agent specific directory
5. Unpack the agent in folder
6. Set permissions for the directory so that users can access it
7. For the user, configure the agent for to the specified VSTS account, using a PAT and named Pool and provided Agent name.
8. During the configuration a svc.sh file is generated. This needs to be run to install the service.
9. After installation the service can be started using the start method.
10. Done one loop

This script needs to be passed to the ARM template. The Custom Script Extension allows us to send a base64 encoded script. So we encrypt the script first and then encode it.

$Bytes = [System.Text.Encoding]::UTF8.GetBytes($script)
$EncodedText =[Convert]::ToBase64String($Bytes)

The encoded script is stored in a VSTS Variable

Write-Host "##vso[task.setvariable variable=EncodedScript;issecret=true]$EncodedText"

This script can be passed to a section in an ARM template through a parameter for the ARM template. The template can be deployed using the Azure Resource Group Deployment task.

In the ARM template you can add a section that executes the provided script. The section can be found here: https://github.com/JasperGilhuis/VSTS-RestAPI/blob/master/ARM-Linux-Custom-Script-Extension-Snippet.json

More details on the current Custom Script Extensions can be found here: https://github.com/Azure/custom-script-extension-linux/blob/master/README.md and here: https://docs.microsoft.com/en-us/azure/virtual-machines/extensions/custom-script-linux

Getting the latest VSTS Agent Download URL for your account

This week I have been playing to automatically provision a VSTS Agent on a Linux Machine. One thing i noticed is that in separate VSTS accounts the latest agent is not always the agent your account supports.

There may be little risk but this PowerShell script, that I use in an Inline PowerShell script in a Task during my provisioning release, helps to get the URL for the account your are targeting. Convenient and checked.

The script requires a few parameters;

  • PersonalAccesToken – A PAT for the VSTS account you are targeting
  • VSTSAccount – The https://account.visualstudio.com url
  • AgentType – The REST API calls for the Agent Type requested, this could be one of three values; “linux-x64”, “windows-x64” or “osx-x64”

The script updates a variable, AgentDownloadUrl, that can be used in the pipeline.

View/Download the script here: https://github.com/JasperGilhuis/VSTS-RestAPI/blob/master/Get-LatestAgentDownload.ps1

 

 

Adding a Team Administrator through the VSTS Rest API

In many projects I come across there is a desire to add a Team Administrator to a VSTS Project. While there is allot of quality documentation, there is no clear route to add a Team Administator to a VSTS Project.

I investigated what calls the VSTS Web UI makes to add a team administrator and constructed a script that does exactly that.

The UI uses a simple method call this method: https://account.visualstudio.com/TeamPermissions/_api/_identity/AddTeamAdmins?__v=5 where it posts a piece of JSON. This basically consists of the Team ID and the user that you want to add.

However to construct this message you need to do several calls to get the required information. It involves getting all the Groups, Users and the users StorageKey to be able to add the administrator.

I created a script containing all the methods and supportive functions that can be found in my GitHub account here: https://github.com/JasperGilhuis/VSTS-RestAPI

Debugging packages from your VSTS Package Management Feed with Visual Studio 2017

In this blogpost I am going to show you how you can debug packages that reside in a Package Management Feed in Visual Studio Team Services (VSTS).

Setting the stage
The package represents a custom build package that can serve any generic functionality used by other applications. The package has its own solution and is automatically pushed to the Package Management feed by a CI Build.

In another solution we have an command line application that consumes this package and we like to have the ability to be able to debug through that code without having to do any other plumbing. We want to treat the package as if we don’t know where the source is.

Continue reading “Debugging packages from your VSTS Package Management Feed with Visual Studio 2017”

Automatically retain a VSTS release

In some environments it can be convenient to retain production releases automatically. Richard Zaat and I worked on this together. Our objective was to retain a release automatically after it has been succesfully enrolled to an environment. To achieve this we wanted to utilize the PowerShell task to minimize the effort.

First we created a demo release pipeline containing one environment. The Release does not do anything and does not have any artifacts. To the environment we only added the PowerShell task.

We have configured to have it use the Preview 2.* version but this works for version 1.* too. The script we have is the following;

$baseurl = $env:SYSTEM_TEAMFOUNDATIONSERVERURI
$baseurl += $env:SYSTEM_TEAMPROJECT + "/_apis"
$uri = "$baseurl/release/releases/$($env:Release_ReleaseID)?api-version=3.0-preview.2"

$accesstoken = "Bearer $env:System_AccessToken"

Invoke-RestMethod -Uri $uri -Method Patch -ContentType "application/json" -Headers @{Authorization = $accesstoken} -Body "{keepforever:true}"

In the script we construct the URL for VSTS Release Management, together with a ‘template’ to call the Release REST API service, passing the current Release ID. It also constructs the Bearer token to be able to call the REST API authenticated. The last line invokes the contructed REST API call. The call sets the ‘KeepForever‘ attribute of the release. This will exempt it from the release retention policies.

In the release definition the “Agent Phase” needs to be configured to “Allow scripts to access OAuth token”. Listed under ‘Additional Options’ section. This will allow the script to use the $env:System_AccessToken.

The last thing to do is to make sure that the agent account has the “Manage Releases” permissions. This can be done very specifically or for all release definitions.

A few links to usefull resources

VSTS Release API overview
https://www.visualstudio.com/en-us/docs/integrate/api/rm/releases

VSTS Release Retention polocies
https://www.visualstudio.com/en-us/docs/build/concepts/policies/retention#release

Interacting with VSTS and the Rest API’s
https://roadtoalm.com/2017/05/01/only-trigger-a-release-when-the-build-changed/

Enjoy!