Brian Hartsock's Blog

Every now and again, your build time gets so long you have to investigate what the cause is. Since Visual Studio uses MSBuild, it is possible to get very detailed build information.

The first step is getting into diagnostic mode. This will print out way too much information, but it is the only mode that prints out execution summaries for each project. To get into diagnostics mode, simply go to Tools -> Options -> Projects and Solutions -> Build and Run. From there, you can change the MSBuild verbosity to diagnostic.

Now just build (or rebuild). Then copy the build output to a txt file and save it somewhere.

The only problem with the output is the amount of information (300K lines on a rebuild of a medium size project). It has ton’s of valuable information, but an easy way to process it is needed. The biggest value, in my opinion, is the section at the end of every project where the Summary information about projects, targets, and tasks are displayed. To help retrieve this info, I wrote a quick little Powershell script.

 
param
(
    [string] $file = ""
)
 
if(!(test-path $file)){
    throw "$file not found"
}
 
function accumulate_and_print_project_info(){
    begin{
        $curr_project = ""
        $should_print = $false
    }
    process{
 
 
        if($_ -match "----- (re)?build (all )?started: project: (.*) -----"){
            $curr_project = $matches[3]
            "----- $curr_project -----"
            $should_print = $false
        }elseif($curr_project -and !$should_print -and $_.EndsWith("Summary:")){
            $_
            $should_print = $true
        }elseif($curr_project -and $should_print -and $_ -eq ""){
            $should_print = $false
        }elseif($should_print){
            $_
        }
    }
}
 
gc $file | accumulate_and_print_project_info

The output is going to look something like the following.

----- ControlPanel, Configuration: Development Any CPU -----
Project Performance Summary:
     2672 ms  C:\Users\brian.hartsock\Documents\Visual Studio 2008\Projects\Api\cp3\ControlPanel\ControlPanel.csproj   1 calls
Target Performance Summary:
        0 ms  CreateSatelliteAssemblies                  1 calls
        0 ms  _ComputeNonExistentFileProperty            1 calls
        0 ms  Clean                                      1 calls
        0 ms  _AfterCompileWinFXInternal                 1 calls
        0 ms  BeforeCompile                              1 calls
        0 ms  BeforeClean                                1 calls
        0 ms  AfterBuild                                 1 calls
        0 ms  GetTargetPath                              1 calls
        0 ms  _CheckForCompileOutputs                    1 calls
        0 ms  EntityDeploy                               1 calls
        0 ms  ResolveVCProjectReferences                 1 calls
        0 ms  _SetEmbeddedWin32ManifestProperties        1 calls
        0 ms  StyleCopForceFullAnalysis                  1 calls
        0 ms  CleanPublishFolder                         1 calls
        0 ms  BeforeBuild                                1 calls
        0 ms  CoreResGen                                 1 calls
        0 ms  BeforeRebuild                              1 calls
        0 ms  _CopySourceItemsToOutputDirectory          1 calls
        0 ms  AfterResGen                                1 calls
        0 ms  Build                                      1 calls
        0 ms  SetWin32ManifestProperties                 1 calls
        0 ms  AssignTargetPaths                          1 calls
        0 ms  CoreBuild                                  1 calls
        0 ms  AfterResolveReferences                     1 calls
        0 ms  Compile                                    1 calls
        0 ms  ResGen                                     1 calls
        0 ms  AfterClean                                 1 calls
        0 ms  Rebuild                                    1 calls
        0 ms  PrepareResourceNames                       1 calls
        0 ms  BuildOnlySettings                          1 calls
        0 ms  PrepareResources                           1 calls
        0 ms  CleanReferencedProjects                    1 calls
        0 ms  AfterMarkupCompilePass1                    1 calls
        0 ms  FileClassification                         1 calls
        0 ms  AfterCompile                               1 calls
        0 ms  CompileRdlFiles                            1 calls
        0 ms  BeforeResolveReferences                    1 calls
        0 ms  CreateCustomManifestResourceNames          1 calls
        0 ms  ResolveReferences                          1 calls
        0 ms  DesignTimeMarkupCompilation                1 calls
        0 ms  AfterRebuild                               1 calls
        0 ms  GetReferenceAssemblyPaths                  1 calls
        0 ms  PrepareForRun                              1 calls
        0 ms  AfterCompileWinFX                          1 calls
        0 ms  BeforeResGen                               1 calls
        1 ms  PrepareRdlFiles                            1 calls
        1 ms  GetCopyToOutputDirectoryItems              1 calls
        1 ms  SplitResourcesByCulture                    1 calls
        1 ms  _SplitProjectReferencesByFileExistence     1 calls
        1 ms  _GenerateCompileInputs                     1 calls
        1 ms  _GenerateSatelliteAssemblyInputs           1 calls
        1 ms  GetWinFXPath                               1 calls
        1 ms  EntityClean                                1 calls
        1 ms  _CheckForInvalidConfigurationAndPlatform   1 calls
        1 ms  ResolveProjectReferences                   1 calls
        1 ms  PrepareForBuild                            1 calls
        3 ms  GetFrameworkPaths                          1 calls
        8 ms  CoreClean                                  1 calls
       14 ms  IncrementalClean                           1 calls
       17 ms  CopyFilesToOutputDirectory                 1 calls
       17 ms  _CleanGetCurrentAndPriorFileWrites         1 calls
       66 ms  ResolveAssemblyReferences                  1 calls
      399 ms  _CopyFilesMarkedCopyLocal                  1 calls
      519 ms  StyleCop                                   1 calls
      636 ms  CoreCompile                                1 calls
      974 ms  PostBuildEvent                             1 calls
Task Performance Summary:
        0 ms  Delete                                     2 calls
        0 ms  Message                                    6 calls
        0 ms  EntityDeploy                               1 calls
        0 ms  GetFrameworkSdkPath                        1 calls
        0 ms  CreateProperty                             1 calls
        0 ms  AssignTargetPath                           5 calls
        0 ms  MakeDir                                    2 calls
        1 ms  ConvertToAbsolutePath                      1 calls
        1 ms  AssignCulture                              1 calls
        1 ms  GetWinFXPath                               1 calls
        1 ms  FindAppConfigFile                          1 calls
        1 ms  ReadLinesFromFile                          2 calls
        1 ms  EntityClean                                1 calls
        2 ms  CreateItem                                 3 calls
        2 ms  WriteLinesToFile                           2 calls
        3 ms  GetFrameworkPath                           2 calls
       13 ms  RemoveDuplicates                           3 calls
       20 ms  FindUnderPath                              7 calls
       66 ms  ResolveAssemblyReference                   1 calls
      415 ms  Copy                                       6 calls
      515 ms  StyleCopTask                               1 calls
      635 ms  Csc                                        1 calls
      973 ms  Exec                                       1 calls

Now you should have all the data you need to figure out what is taking so long to build your project.

There are a few different ways to use Powershell from the legacy cmd shell. The most common way is to call it like the following.

> powershell write-host "hello world"

As you can see, the powershell.exe is called with Powershell commands as the parameters. I started noticing some odd behavior though. I have the following script, TestScript.ps1. It has code as follows:

param
(
  $str
)
 
write-host $str

Very simple right. Well guess what happens when I call it from powershell.exe like above?

> powershell .\TestScript.ps1 "hello world"
hello

A little odd. I have a very cordial script but it is a little tongue tied. I didn’t spend a lot of time trying to figure out why this was occurring, I instead used powershell -? to help me find an alternate method, and probably better way to call Powershell scripts from the legacy cmd shell.

> powershell -Command "& { .\TestScript.ps1 'hello world' }"
hello world

This worked like a charm. Note the quotes, as script blocks aren’t interpreted from the cmd shell properly, and will cause odd behavior. And from MSBuild, a little bit of XML escapage and you can easily use Powershell.

  <Target Name="AfterBuild">
    <Exec Command="powershell.exe -command &quot;&amp; {.\Register-EmailApiSnapIn.ps1 &apos;$(TargetPath)&apos;}&quot;" />
  </Target>

Yesterday, I took the plunge, changed my DNS, and am now on Mosso. Dreamhost was getting slow, and having intermittent outages, so I needed the switch. Not to mention Mosso is another division of Rackspace, just like Mailtrust.

I am super happy with Mosso so far. The blog is faster than ever, and I have had almost no problems. What’s even crazier is I am using CDN Tools to host a lot of my static content on the Cloud Files CDN. I could have done this while on Dreamhost, but it was easier since I already have a Mosso account.

Automation

Setting up Mosso is quite different than Dreamhost, since SSH access isn’t permitted. FTP or SFTP is really the only means to get your code uploaded, so automation is a must. Luckily I love automation, so I got to work on an MSBuild script.

Just to give some background on how my Wordpress blog works. I have a SVN checkout of the latest Wordpress tag. I also have a few custom files that I want uploaded, in addition to to the Wordpress code. Instead of interminginly the custom code with Wordpress, I created a custom directory.

My idea for the build was to follow a fairly simple workflow.

• Update Wordpress
• Export Wordpress checkout to a deployment directory
• Copy custom files to deployment directory
• SFTP deployment directory

Here is how the majority of the build script came out. You will need MSBuild Community tasks as well as SVN installed somewhere.

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="Upload">
  <Import Project="lib\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets"/>
  <PropertyGroup>
    <!-- Fill this in with all your info -->
    <CloudSiteFtp></CloudSiteFtp>
    <CloudSiteDbHost></CloudSiteDbHost>
    <CloudSiteDb></CloudSiteDb>
    <CloudSiteDbUser></CloudSiteDbUser>
    <CloudSiteDbPass></CloudSiteDbPass>
    <CloudSiteFtpUser></CloudSiteFtpUser>
    <CloudSiteFtpPass></CloudSiteFtpPass>
    <SvnSource></SvnSource>
    <SvnToolPath></SvnToolPath>
    <DeploymentDir></DeploymentDir>
    <CustomSourceDir></CustomSourceDir>
    <CloudSiteDir></CloudSiteDir>
  </PropertyGroup>
 
  <Target Name="DeleteDeploymentDir">
    <Message Importance="normal" Text="Deleting directory $(DeploymentDir)" />
    <RemoveDir Directories="$(DeploymentDir)" />
    <Message Text="Done." />
    <Message Text="" />
  </Target>
 
  <Target Name="UpdateSource">
    <Message Text="Updating $(SvnSource)..." />
    <SvnUpdate LocalPath="$(SvnSource)" ToolPath="$(SvnToolPath)" />
    <Message Text="Done." />
    <Message Text="" />
  </Target>
 
  <Target Name="ExportSource" DependsOnTargets="DeleteDeploymentDir">
    <Message Text="Exporting $(SvnSource) to $(DeploymentDir)..." />
    <SvnExport LocalPath="$(DeploymentDir)" RepositoryPath="$(SvnSource)" ToolPath="$(SvnToolPath)" />
    <Message Text="Done." />
    <Message Text="" />
  </Target>
 
  <Target Name="MergeCustomSourceCode">
    <Message Text="Copying custom files..." />
    <Exec Command="xcopy $(CustomSourceDir) $(DeploymentDir) /E /Y" />
    <Message Text="Done." />
    <Message Text="" />
  </Target>
  <!-- Upload, BackupDb, and ImportDB targets should go here -->
</Project>

Uploading the data from a Windows machine required a tool for SFTP or FTP. MSBuild Community Tasks have FTP tasks, but I wanted to make sure my data was transfered securely, so I chose SFTP. pscp was the obvious tool because it is the simplest solution to SFTP file transfer on Windows, in my opinion.

<Target Name="Upload" DependsOnTargets="UpdateSource; ExportSource; MergeCustomSourceCode">
  <Exec Command="lib\pscp -r -pw $(CloudSiteFtpPass) $(DeploymentDir)\* $(CloudSiteFtpUser)@$(CloudSiteFtp):$(CloudSiteDir)" />
</Target>

The next step was getting the database working. The database and database user have to be created from the web interface before anything can be done from the client. Once it is created, you can execute commands remotely, so automation is pretty easy. Word of warning, I am one of those crazy’s that run MySQL on my laptop, so access to mysql and mysqldump is required for these tasks to work.

<Target Name="ImportDb">
  <Error Condition="'$(ImportFileName)' == ''" Text="No ImportFileName specificed (msbuild /p:ImportFileName ...)" />
 
  <Exec Command="mysql -h $(CloudSiteDbHost) -u $(CloudSiteDbUser) -p$(CloudSiteDbPass) $(CloudSiteDb) &lt; $(ImportFileName)" />
</Target>
 
<Target Name="BackupDb">
 
  <Error Condition="'$(BackupFile)' == ''" Text="No backupfile specified" />
 
  <Exec Command="mysqldump -h $(CloudSiteDbHost) -u $(CloudSiteDbUser) -p$(CloudSiteDbPass) $(CloudSiteDb) > $(BackupFile)" />
  <Zip Files="$(BackupFile)" ZipFileName="$(BackupFile).zip" />
  <Delete Files="$(BackupFile)" />
</Target>

What next…?

I am still somewhat unhappy that my upload process. It uploads everything, each time. rsync, or something similar, would be awesome if SSH access was allowed. I am pretty sure I can accomplish a dumb rsync over SFTP, but I haven’t devoted enough time to it. The other downside to this is wordpress’s automatic upgrade feature will get overwritten with the next upload. So be wary about automatically upgrading through the wordpress interface.

After that, I want to incorporate data backups before the upload and allow for easy rollback if something fails. After that, I should be set.

Today I wrote my first MsBuild script that uses sqlcmd.exe to execute SQL Server commands remotely. It is awesome. Although it is fairly limited, for tasks like backing up databases, it is very simple and easy.

  <Target Name="Backup">
	<Exec Command='sqlcmd -S $(server) -q "$(sql)"' />
  </Target>