Paul Selles

Computers and cats

Tag Archives: TFS

TFS API: TFS User Email Address Lookup and Reverse Lookup

Occasionally I need to develop a tool that requires sending and receiving emails to and from developers. To do this I will need to lookup email addresses based on a TFS user’s display or account name and vice versa. Lucky for us this is painfully easy to do using IIdentityManagementService.ReadIdentity and the TeamFoundationIdentity class [1][2]:

private static readonly string TeamProjectUriString = "http://tfs.yourtfsurl.com";

// Team Project Collection getter
private static TfsTeamProjectCollection _tfsTeamProjectCollection;
public static TfsTeamProjectCollection TfsTeamProjectCollection
{
    get
    {
        return _tfsTeamProjectCollection ??
               (_tfsTeamProjectCollection = new TfsTeamProjectCollection(new Uri(TeamProjectUriString)));
    }
}

// Identity Management Service getter
private static IIdentityManagementService _identityManagementService;
public static IIdentityManagementService IdentityManagementService
{
    get
    {
        return _identityManagementService ??
               (_identityManagementService = TfsTeamProjectCollection.GetService<IIdentityManagementService>());
    }
}

// Get Email Address from TFS Account or Display Name
public static string GetEmailAddress(string userName)
{
    TeamFoundationIdentity teamFoundationIdentity =
        IdentityManagementService.ReadIdentity(
            IdentitySearchFactor.AccountName | IdentitySearchFactor.DisplayName,
            userName,
            MembershipQuery.None,
            ReadIdentityOptions.None);

    return teamFoundationIdentity.GetAttribute("Mail", null);
}

// Get TFS Display Name from and Email Address
public static string GetDisplayName(string emailAddress)
{
    TeamFoundationIdentity teamFoundationIdentity =
        IdentityManagementService.ReadIdentity(
            IdentitySearchFactor.MailAddress,
            emailAddress,
            MembershipQuery.None,
            ReadIdentityOptions.None);

    return teamFoundationIdentity.DisplayName;
}

References

[1] IIdentityManagementService.ReadIdentity Method. MSDN

[2] TeamFoundationIdentity Class. MSDN

Advertisements

Renaming File with Team Foundation Server API: Extending Workspace to Rename a Large Group of Files

The Problem

We have TFS check-in policies that will enforce naming conventions for certain files. The policy will allow the developer to programmatically rename any files that do not match the acceptable naming convention. Behind the scene the renaming is done using the WorkSpace.PendRename method [1]. This policy works fine for one or two files but for a large number of files we start having performance issues. Anyone who has experience renaming multiple files on TFS knows that it is a painfully slow ordeal. So how can we speed it up.

The Solution

I have solved this issue through the use of extending methods in the Microsoft.TeamFoundation.VersionControl.Client.Workspace class, and incorporating Task Parallelism [2][3][4]. Since the original Workspace.PendRename returns the number of files that it has renamed as an Int32, we want to make sure that we preserve that functionality.  I have only created a solution for the Workspace.PendRename(string,string) case since this is the only variation of the method that I am currently using; to expand this to all Workspace.PendRename implementations is trivial.

This the extended method:

public static class WorkSpaceExtension
{
    // Extend PendRename(String, String)
    public static int PendRename(this Workspace workspace,
        string[] oldPaths,
        string[] newPaths)
    {
        // Make sure that the oldPath and new Paths match
        if (oldPaths.Count() != newPaths.Count())
            throw new ArgumentException("Every oldPath must have corresponding newPath");

        // This list will contain all our tasks
        List pendRenameTasks = new List();

        // Loop throuh all newPath and oldPath pairs
        for (int i = 0; i < oldPaths.Count(); i++)
        {
            // Add each new task in out task container
            pendRenameTasks.Add(Task.Factory.StartNew((Object obj) =>
            {
                // We are going to pass the current old path and the
                // current new path into the path as an anonymous type
                var path = (dynamic)obj;
                return workspace.PendRename(path.oldPath, path.newPath);
            }, new { oldPath = oldPaths[i], newPath = newPaths[i] }));
        }

        // Wait for all tasks to complete
        Task.WaitAll(pendRenameTasks.ToArray());

        // Sum up the result of all the original PendRename method
        int taskCount=0;
        foreach (Task pendRenameTask in pendRenameTasks)
        {
            taskCount += pendRenameTask.Result;
        }

        // Return the number of file names changed
        return taskCount;
    }
}

References

[1] Workspace.PendRename Method. MSDN Library.

[2] Extension Methods (C# Programming Guide). MSDN Library.

[3] WorkSpace Class. MSDN Library.

[4] Extension Methods (C# Programming Guide). MSDN Library.

Tfs Build: Query Build Table for historical Build Status, Compilation Status, and Test Status

If you are having trouble finding historical build data using the TFS API, build information can be found in your Team Foundation Server Databases [1]. The table Tbl_Build keeps a log of all your historical builds including, build number, start and stop times, controller id, drop location, and statuses. This can be and extremely powerful tool to look up Build Id values to look up historic build logs which contain detailed reports (as covered in Tfs Build Log: Querying Build Log Data).

You can find the Tbl_Build table within your Collection database (Tfs_YourTeamProjectCollection, where YourTeamProjectCollection is the name of your Team Project Collection.) The Status columns that we are interested in are BuildStatus, CompilationStatus, and TestStatus as seen in the SQl Server Management Studio screen capture below:

Tbl_Build_Columns

The BuildStatus column values map to the Microsoft.TeamFoundation.Build.Client BuildStatus Enumeration [2]. Here is the Database specific table:

BuildStatus Description
1 InProgress
2 Succeeded
4 PartiallySucceeded
8 Failed
16 Stopped

The CompliationStatus and TestStatus columns are trickier to deal with than BuildStatus column. Where the BuildStatus is pretty well self explanatory, both the CompliationStatus and TestStatus are heavily based on user definitions within the Build Process Template. That is to say that a Complitation/Test failure may not mean what you think that it means out of the box. Please review your Build Process Template and see how the CompliationStatus and TestStatus are set.

Moving forward, both CompliationStatus and TestStatus columns map to the Microsoft.TeamFoundation.Build.Client BuildPhaseStatus Enumeration [3]. Here are the Database specific table:

CompliationStatus Description
1 Failed
2 Succeeded
NULL Unknown
TestStatus Description
1 Failed
2 Succeeded
NULL Unknown

We can now query the database for historical build information, for example we can look for all failed builds between Jan 5th and Jan 18th of this year:

SELECT * FROM [Tfs_YouTeamProjectCollection].[dbo].[Tbl_Build] 
WHERE [BuildStatus]=8 
AND [StartTime] BETWEEN '1/5/2013 12:00:00 AM' 
	AND '1/18/2013 12:00:00 AM'

Be sure to check this out my previous post Tfs Build Log: Querying Build Log Data to learn how to programmatically read historical build data.

References

[1] Team Foundation Server Databases. MSDN Library.

[2] BuildStatus Enumeration. MSDN Libary.

[3] BuildPhaseStatus Enumeration. MSDN Libary.

Tfs Build Log: Querying Build Log Data

If you have ever wanted to programmatically parse past or current Tfs Build log data, then this post is for you.

 

Background

My company uses Tfs to implement a Rolling CI Build which often involves multiple Changesets per build. For the most part this is smooth sailing, however when the build breaks we want to to quickly be able to pinpoint the source and notify the guilty party. With multiple Associated Changesets per build, with multiple files checked-in by multiple developers, how to we efficiently pin-point to source of the break. We also do not want to parse the MSBuild log files since we are not creating log files for our Rolling CI Builds. Our solution then become querying and parsing the xml build information from the Project Collection SQL Database.

All the build log information is available on the Project Collection Database where the data is recorded one activity at a time so we are able to query all the way up to the current build activity. All this data can be found in the SQL Database, appropriately named Tfs_YourTeamProjectCollection, table Tbl_BuildInformation; where YourTeamProjectCollection is the name of your Team Project Collection. The build activity will appear as independent XML nodes, divided up into 16 different types.

 

Examining Tbl_BuildInformation

As mentioned above the build information will be located the Tbl_BuildInformation table and the build activity will be appear of 16 different types of XML nodes. Firstly let’s take a look at the the available columns:

dbo.tbl_BuildInformation_Columns

and a query result for a single build:

tlb_BuildInformation_query 

Here is what you need to know about the columns:

  • BuildId: Should be self explanatory it is typically the last octet of the IBuildDetail.BuildNumber (BuildDefinitionName.1.0.0.12345) or the Uri Fragment in the IBuildDetail.BuildUri (vstfs:///Build/Build/12345.)[1][2]
  • NodeId: Id given to the Node on that specific node
  • ParentId: This is the hierarchy that in the build log, this id represents it’s parent NodeId
  • NodeType: Identify what type of node the row is (1 to 16)
  • LastModifiedDate: Self explanatory
  • Fields: Contains the XML data that we are interested in.

 

Understanding NodeTypes

Even though there are 16 different NodeTypes I am interested in only 5 of them. I had some difficulties finding documentation on the NodeTypes, so here is are examples of the XML Fields Column of each the NodeTypes that I use. The structure of all the NodeTypes are the same, and any value can be retrieved with the the XPath string “/Fields/Field[@Name=’AttributeValueOfInterest’][Value]”. For the sake of simplicity, I will only be referencing the ‘AttributeValueOfInterest’ from here on out.

NodeType 4: Project or Solution Compilation Results

This NodeType contains the MSBuild results for a Project or Solution component of your Build Targets. It provides the number of build errors or warnings for specific build. The Build Agent Local Path and the Server Path of the Project or Solution being built. Expect to see one entry for each Project or Solution Built.

Example Fields Column XML:

<Fields>
  <Field Name="CompilationErrors">
    <Value>0</Value>
  </Field>
  <Field Name="CompilationWarnings">
    <Value>0</Value>
  </Field>
  <Field Name="FinishTime">
    <Value>2013-10-30T20:01:00.3073686Z</Value>
  </Field>
  <Field Name="Flavor">
    <Value>Release</Value>
  </Field>
  <Field Name="LocalPath">
    <Value>C:\Builds\57\Path\To\Your\Project\Or\Solution.csproj</Value>
  </Field>
  <Field Name="Platform">
    <Value>AnyCPU</Value>
  </Field>
  <Field Name="ServerPath">
    <Value>$/Path/To/Your/Project/Or/Solution.csproj<Value>
  </Field>
  <Field Name="StartTime">
    <Value>2013-10-30T20:00:32.4797462Z</Value>
  </Field>
  <Field Name="StaticAnalysisErrors">
    <Value>0</Value>
  </Field>
  <Field Name="StaticAnalysisWarnings">
    <Value>0</Value>
  </Field>
  <Field Name="TargetNames">
    <Value />
  </Field>
</Fields>

NodeType 6: Total Compilation Results

This NodeType contains the MSBuild results for all components of your Build Targets. It provides the number of build errors or warnings for specific build.

Example Fields Column XML:

<Fields>
  <Field Name="Platform">
    <Value>Any CPU</Value>
  </Field>
  <Field Name="Flavor">
    <Value>Release</Value>
  </Field>
  <Field Name="TotalCompilationErrors">
    <Value>16</Value>
  </Field>
  <Field Name="TotalCompilationWarnings">
    <Value>9</Value>
  </Field>
  <Field Name="TotalStaticAnalysisErrors">
    <Value>0</Value>
  </Field>
  <Field Name="TotalStaticAnalysisWarnings">
    <Value>0</Value>
  </Field>
</Fields>

NodeType 7: Associated Changeset

This NodeType contains contains the information of an individual Changeset from the list of Associated Changesets. It contains the ChangsetId, who checked it in, and the the check-in comments.

Example Fields Column XML:

<Fields>
  <Field Name="ChangesetId">
    <Value>12345</Value>
  </Field>
  <Field Name="ChangesetUri">
    <Value>vstfs:///VersionControl/Changeset/12345</Value>
  </Field>
  <Field Name="CheckedInBy">
    <Value>Paul Selles</Value>
  </Field>
  <Field Name="Comment">
    <Value>Check-in comments.</Value>
  </Field>
</Fields>

NodeType 8: Build Log Errors

This NodeType contains contains any errors that are created during any build activity. These can be anything from compilation errors, network errors, to user created errors through the WriteBuildError class in the build process template[3]. As an example I will post the Fields Column XML for a compilation error. It is important to note that for error types other than compilation error, the AttributeValueOfInterest File, ServerPath, LineNumber, EndLineNumber are not present.

Example Fields Column XML:

<Fields>
  <Field Name="Code">
    <Value>BC30002</Value>
  </Field>
  <Field Name="EndLineNumber">
    <Value>270</Value>
  </Field>
  <Field Name="ErrorType">
    <Value>Compilation</Value>
  </Field>
  <Field Name="File">
    <Value>C:\Builds\57\Path\To\Your\Project\Or\CodeFile.cs</Value>
  </Field>
  <Field Name="LineNumber">
    <Value>270</Value>
  </Field>
  <Field Name="Message">
    <Value>Type 'EmailSet' is not defined.</Value>
  </Field>
  <Field Name="ServerPath">
    <Value>$/RQ/Path/To/Your/Project/Or/CodeFile.cs;C51077</Value>
  </Field>
  <Field Name="Timestamp">
    <Value>2013-10-30T20:10:34.6821707Z</Value>
  </Field>
</Fields>

NodeType 10: Build Log Warnings

This NodeType contains any warnings that are created during any build activity. This is very similar to NodeType 8, build activity errors. The build process template class for user created warning is WriteBuildWarning [3]. And as for the errors above, the posted example of the Fields Column XML for a compilation error. It is important to note that for warnings types other than compilation warnings , the AttributeValueOfInterest File, ServerPath, LineNumber, EndLineNumber are not present.

Example Fields Column XML:

<Fields>
  <Field Name="CompilationErrors">
    <Value>0</Value>
  </Field>
  <Field Name="CompilationWarnings">
    <Value>0</Value>
  </Field>
  <Field Name="FinishTime">
    <Value>2013-10-30T20:01:00.3073686Z</Value>
  </Field>
  <Field Name="Flavor">
    <Value>Release</Value>
  </Field>
  <Field Name="LocalPath">
    <Value>C:\Builds\57\Path\To\Your\Project\Or\Solution.csproj</Value>
  </Field>
  <Field Name="Platform">
    <Value>AnyCPU</Value>
  </Field>
  <Field Name="ServerPath">
    <Value>$/Path/To/Your/Project/Or/Solution.csproj<Value>
  </Field>
  <Field Name="StartTime">
    <Value>2013-10-30T20:00:32.4797462Z</Value>
  </Field>
  <Field Name="StaticAnalysisErrors">
    <Value>0</Value>
  </Field>
  <Field Name="StaticAnalysisWarnings">
    <Value>0</Value>
  </Field>
  <Field Name="TargetNames">
    <Value />
  </Field>
</Fields>

 

Querying The Database

In order to get these values, we will query this database with the current BuildId. As mentioned above, this is typically be the last octet of the IBuildDetail.BuildNumber (BuildDefinitionName.1.0.0.12345) or the Uri Fragment in the IBuildDetail.BuildUri (vstfs:///Build/Build/12345.)[1][2] Best practice is to confirm the BuildId in the table tbl_Build using the above mentioned number, 12345, in the BuildUri column. Once we have confirmed that we have to correct BuildId, we can go ahead and get the relevant build information with the following query:

SELECT * FROM [Tfs_YourTeamProjectCollection].[dbo].[Tbl_BuildInformation] WHERE [BuildId]=BuildId AND [NodeType] IN (4,6,7,8,10)

 

References

[1] IBuildDetail.BuildNumber Property. MSDN Library

[2] IBuildDetail.Uri Property. MSDN Library

[3] WriteBuildError. MSDN Library

[4] WriteBuildWarning. MSDN Libary

TFS and Git Can Play Together

If you find yourself in a situation where you are using both TFS and Git for source control then you may be tempted to duplicate your efforts and write separate scripts for both. Before you get carried away, hear me out. My company uses both TFS and Git, and I have created a couple of scripts to help interact with both environments.

All the scripts that I have written find the project root directory and do all the work from there. Consequently, all my builds, testing, and deployments will be defined relative to the project root directory.  The root directory for our Git project is based off the TopLevel, and for TFS the root directory is based off the ProjectCollection.

Since we are dealing with Git and TFS I feel like I should provide a solution for both bash and PowerShell.

First bash:

#!/bin/bash

# Find and move into project directory
if [ $(git rev-parse --is-inside-work-tree 2> /dev/null) ]
then 
	echo -e "33[1;35mBuilding in git33[0m"
	path=$(git rev-parse --show-toplevel)
else
	echo -e "33[1;35mBuilding in tfs33[0m"
	tfpath=`C:/Program\ Files\ \(x86\)/Microsoft\ Visual\ Studio\ 11.0/Common7/IDE/tf.exe info $/`
	tfpath=${tfpath##*Local path : }
	tfpath=${tfpath%% Server path:*}
	path=$tfpath
fi

pushd $path

## Your script goes here
# Go ahead build, test, deploy
# Seriouly enjoy yourself

# Return to starting directory
popd

In the above script it may be unnecessary to include the full path to tf.exe if it already exists in your windows environment %PATH%. I find it helpful to include bash related scripts within my bash environment $PATH. A simple way to include custom scripts is the navigate to your scripts directory and type in the following command:

> echo “export PATH=\${PATH}:${PWD}” >> ~/.bashrc

This will append to your .bashrc script which is used when you open a new interactive shell.

Now for the PowerShell solution:

function Get-VsCommonTools
{

	# Array of valid VS Common Tools environment variable versions,
	# ensure that the path is not null.
	$VsCommonToolsPaths = @(@($env:VS110COMNTOOLS,$env:VS100COMNTOOLS) | Where-Object {$_ -ne $null})
	if ($VsCommonToolsPaths.Count -ne 0) {$VsCommonToolsPaths[0]}
	else {Write-Error "Unable to find Visual Studio Common Tool Path."}
}
################################################################################
function Get-Tf
{

	$TfPath = Get-VsCommonTools

	if (Test-Path "${TfPath}..\IDE\tf.exe")
	{
		return "${TfPath}..\IDE\tf.exe"
	}

	Write-Error "Could not find tf.exe"
}
################################################################################
function Get-Git
{

	# Array of valid VS Common Tools environment variable versions,
	# ensure that the path is not null.
	$ProgramFilesPaths = @(@($env:PROGRAMFILES,${env:PROGRAMFILES(X86)}) | Where-Object {Test-Path (Join-Path $_ "\Git\cmd\git.exe")})
	if ($ProgramFilesPaths.Count -ne 0) {(Join-Path $ProgramFilesPaths[0] "\Git\cmd\git.exe")}
	else {Write-Error "Unable to find Git Path."}
}
################################################################################
function Get-RootDirectory
{
	$Tf = Get-Tf
	$Git = Get-Git

	if (& $Git rev-parse --is-inside-work-tree 2> NULL) { $RootDirectory = & $Git rev-parse --show-toplevel }
	else { if ([String](& $Tf info $/) -match "\s*Local\s*path\s*:\s*(?[^\s]+)") { $RootDirectory = $matches["LocalPath"] } }

	if (-not $RootDirectory) { Write-Error "Could not find root directory"; return }

	$RootDirectory
}

In the above script we have four functions. The first function will get the latest visual studio common tools path, as mentioned above. This is unnecessary if tf.exe is within your %PATH% windows environment variable. The next two functions are to get the local path for both tf.exe and git.exe. Once again, these two functions can be referenced in %PATH% instead. The final function Get-RootDirectory will return root directory of the currently active project.

The Powershell script is slightly different than the bash script, though it achieves the same results, to determine if the current working directory is within Git or Tfs. Using these two scripts, I am able to work with one set of build, testing, and deployment scripts for the code in either Git or Tfs. I hope you find them as useful as I have.

%d bloggers like this: