Merge pull request #199 from petabridge/dev

0.7.0 Release

README.md

@@ -1,8 +1,196 @@
 # Incrementalist
 
-Update this readme file with your details.
+Incrementalist is a .NET tool that leverages [libgit2sharp](https://github.com/libgit2/libgit2sharp/) and [Roslyn](https://docs.microsoft.com/en-us/dotnet/csharp/roslyn-sdk/) to compute incremental build steps, to help reduce total build time, for Continuous Integration systems for _large_ .NET solutions.
 
-## Building this solution
+## Installation and Use
+Incrementalist is available in one of two distribution methods:
+
+1. [Incrementalist Library](https://www.nuget.org/packages/Incrementalist/) - a stand-alone .NET Standard 2.0 NuGet package that can be called from a program of your own making or
+2. [Incrementalist.Cmd](https://www.nuget.org/packages/Incrementalist.Cmd/) - a `dotnet tool` that can be run directly from the `dotnet` CLI. **Most users prefer `Incrementalist.Cmd` for running inside their build systems**.
+
+To install Incrementalist and try for yourself, do the following:
+
+```shell
+dotnet tool install --global Incrementalist.Cmd
+```
+
+### `Incrementalist.Cmd` CLI Options
+
+The following CLI options are available on Incrementalist, which you can print out at any time via the `incrementalist --help` command:
+
+```
+  -s, --sln             The name of the Solution file to be analyzed by Incrementalist.
+
+  -f, --file            If specified, writes the output to the named file.
+
+  -l, --folders-only    List affected folders instead of .NET projects
+
+  -b, --branch          Required. (Default: dev) The git branch to compare against. i.e. the `dev` or the
+                        `master` branch.
+
+  -d, --dir             Specify the working directory explicitly. Defaults to using the current working
+                        directory.
+
+  --verbose             (Default: false) Prints out extensive debug logs during operation.
+
+  -t, --timeout         (Default: 2) Specifies the load timeout for the solution in whole minutes.
+                        Defaults to 2 minutes.
+
+  --help                Display this help screen.
+
+  --version             Display version information.
+```
+
+To run a standard Incrementalist build on a project like Akka.NET, we do the following:
+
+```shell
+PS> incrementalist -s ./src/Akka.sln -b dev --file ./bin/output/incrementalist.txt
+```
+
+## How It Works
+
+Incrementalist works by analyizing the `git diff` of each commit in your working branch, comparing it to a base branch (i.e. `dev`) to determine which files have been modified in your changes, and then it uses Roslyn solution analysis to determine the graph of projects that were affected by these changes.
+
+![Incrementalist - how it works](docs/images/incrementalist-how-it-works.png)
+
+This graph analysis produces a text file that looks like this (when we're running it on the [Akka.NET main repository](https://github.com/akkadotnet/akka.net)):
+
+> D:\a\1\s\src\core\Akka\Akka.csproj,D:\a\1\s\src\benchmark\SerializationBenchmarks\SerializationBenchmarks.csproj
+
+* Each line represents one graph of changes detected by `git` and Roslyn - each project will be listed using its absolute path and will be separated by comma;
+* If there are multiple lines in the output file, it means that multiple discrete graphs were detected (two sets of projects that don't directly relate to eachother were updated in the same set of `git` commits.)
+
+This file can be parsed and used inside a build script, such as the [FAKE file](https://fake.build/) we use for running Akka.NET's build system:
+
+```fsharp
+//--------------------------------------------------------------------------------
+// Incrementalist targets
+//--------------------------------------------------------------------------------
+// Pulls the set of all affected projects detected by Incrementalist from the cached file
+let getAffectedProjectsTopology =
+    lazy(
+        log (sprintf "Checking inside %s for changes" incrementalistReport)
+
+        let incrementalistFoundChanges = File.Exists incrementalistReport
+
+        log (sprintf "Found changes via Incrementalist? %b - searched inside %s" incrementalistFoundChanges incrementalistReport)
+        if not incrementalistFoundChanges then None
+        else
+            let sortedItems = (File.ReadAllLines incrementalistReport) |> Seq.map (fun x -> (x.Split ','))
+                              |> Seq.map (fun items -> (items.[0], items))
+            let d = dict sortedItems
+            Some(d)
+    )
+
+let getAffectedProjects =
+    lazy(
+        let finalProjects = getAffectedProjectsTopology.Value
+        match finalProjects with
+        | None -> None
+        | Some p -> Some (p.Values |> Seq.concat)
+    )
+
+Target "ComputeIncrementalChanges" (fun _ ->
+    if runIncrementally then
+        let targetBranch = match getBuildParam "targetBranch" with
+                            | "" -> "dev"
+                            | null -> "dev"
+                            | b -> b
+        let incrementalistPath =
+                let incrementalistDir = toolsDir @@ "incrementalist"
+                let globalTool = tryFindFileOnPath "incrementalist.exe"
+                match globalTool with
+                    | Some t -> t
+                    | None -> if isWindows then findToolInSubPath "incrementalist.exe" incrementalistDir
+                              elif isMacOS then incrementalistDir @@ "incrementalist"
+                              else incrementalistDir @@ "incrementalist"
+
+
+        let args = StringBuilder()
+                |> append "-b"
+                |> append targetBranch
+                |> append "-s"
+                |> append solution
+                |> append "-f"
+                |> append incrementalistReport
+                |> toText
+
+        let result = ExecProcess(fun info ->
+            info.FileName <- incrementalistPath
+            info.WorkingDirectory <- __SOURCE_DIRECTORY__
+            info.Arguments <- args) (System.TimeSpan.FromMinutes 5.0) (* Reasonably long-running task. *)
+
+        if result <> 0 then failwithf "Incrementalist failed. %s" args
+    else
+        log "Skipping Incrementalist - not enabled for this build"
+)
+
+let filterProjects selectedProject =
+    if runIncrementally then
+        let affectedProjects = getAffectedProjects.Value
+
+        (*
+        if affectedProjects.IsSome then
+            log (sprintf "Searching for %s inside [%s]" selectedProject (String.Join(",", affectedProjects.Value)))
+        else
+            log "No affected projects found"
+        *)
+
+        match affectedProjects with
+        | None -> None
+        | Some x when x |> Seq.exists (fun n -> n.Contains (System.IO.Path.GetFileName(string selectedProject))) -> Some selectedProject
+        | _ -> None
+    else
+        log "Not running incrementally"
+        Some selectedProject
+```
+
+And in each one of our build steps where we want to execute incremental builds only (in order to reduce the amount of work per-pull request), we call the `filterProjects` method that uses the `Incrementalist.Cmd` output to determine which projects need to be considered based on these changes:
+
+```fsharp
+//--------------------------------------------------------------------------------
+// Build targets
+//--------------------------------------------------------------------------------
+let skipBuild =
+    lazy(
+        match getAffectedProjects.Value with
+        | None when runIncrementally -> true
+        | _ -> false
+    )
+
+let headProjects =
+    lazy(
+        match getAffectedProjectsTopology.Value with
+        | None when runIncrementally -> [||]
+        | None -> [|solution|]
+        | Some p -> p.Keys |> Seq.toArray
+    )
+
+Target "AssemblyInfo" (fun _ ->
+    XmlPokeInnerText "./src/common.props" "//Project/PropertyGroup/VersionPrefix" releaseNotes.AssemblyVersion
+    XmlPokeInnerText "./src/common.props" "//Project/PropertyGroup/PackageReleaseNotes" (releaseNotes.Notes |> String.concat "\n")
+)
+
+Target "Build" (fun _ ->
+    if not skipBuild.Value then
+        let additionalArgs = if versionSuffix.Length > 0 then [sprintf "/p:VersionSuffix=%s" versionSuffix] else []
+        let buildProject proj =
+            DotNetCli.Build
+                (fun p ->
+                    { p with
+                        Project = proj
+                        Configuration = configuration
+                        AdditionalArgs = additionalArgs })
+
+        match getAffectedProjects.Value with
+        | Some p -> p |> Seq.iter buildProject
+        | None -> buildProject solution // build the entire solution if incrementalist is disabled
+)
+```
+
+This tool is best used for _large_ .NET projects, where the time to complete each build step can take 30+ minutes. Incrementalist can help reduce the average execution time by an order of magnitude for many pull requests.
+
+## Build Instructions
 To run the build script associated with this solution, execute the following:
 
 **Windows**

RELEASE_NOTES.md

@@ -1,6 +1,5 @@
-#### 0.6.0 November 09 2021 ####
-New feature release: 0.6.0
+#### 0.7.0 May 24 2022 ####
 
-* Upgraded to LibGit2 experimental in order to run on newer version of Ubuntu;
-* Added .NET 6.0 `dotnet tool` support; and
-* Upgraded to .NET 6 versions of binaries.
+* Upgraded from Roslyn 3.11 to 4.2.0;
+* Upgraded NuGet.ProjectModel to 6.2.0; and
+* Upgraded Libgit2Sharp to 0.27.0-preview-0182.

src/Incrementalist.Cmd/Incrementalist.Cmd.csproj

@@ -11,7 +11,7 @@
   </PropertyGroup>
 
   <ItemGroup>
-    <PackageReference Include="CommandLineParser" Version="2.8.0" />
+    <PackageReference Include="CommandLineParser" Version="2.9.1" />
     <PackageReference Include="Microsoft.Build.Locator" Version="1.4.1" />
     <PackageReference Include="Microsoft.CodeAnalysis.Workspaces.MSBuild" Version="$(RoslynVersion)" />
     <PackageReference Include="Microsoft.Extensions.Logging.Console" Version="2.2.0" />

src/Incrementalist/Incrementalist.csproj

@@ -9,7 +9,7 @@
 
 
   <ItemGroup>
-    <PackageReference Include="Libgit2Sharp" Version="0.27.0-preview-0156" />
+    <PackageReference Include="Libgit2Sharp" Version="0.27.0-preview-0182" />
     <PackageReference Include="Microsoft.CodeAnalysis.CSharp.Workspaces" Version="$(RoslynVersion)" />
     <PackageReference Include="Microsoft.Extensions.Logging" Version="6.0.0" />
   </ItemGroup>

src/common.props

@@ -1,12 +1,11 @@
 <Project>
   <PropertyGroup>
-    <Copyright>Copyright © 2015-2021 Petabridge</Copyright>
+    <Copyright>Copyright © 2015-2022 Petabridge</Copyright>
     <Authors>Petabridge</Authors>
-    <VersionPrefix>0.6.0</VersionPrefix>
-    <PackageReleaseNotes>New feature release: 0.6.0
-Upgraded to LibGit2 experimental in order to run on newer version of Ubuntu;
-Added .NET 6.0 `dotnet tool` support; and
-Upgraded to .NET 6 versions of binaries.</PackageReleaseNotes>
+    <VersionPrefix>0.7.0</VersionPrefix>
+    <PackageReleaseNotes>Upgraded from Roslyn 3.11 to 4.2.0;
+Upgraded NuGet.ProjectModel to 6.2.0; and
+Upgraded Libgit2Sharp to 0.27.0-preview-0182.</PackageReleaseNotes>
     <tags>build, msbuild, incremental build, roslyn, git</tags>
     <PackageIconUrl>
       https://petabridge.com/images/logo.png
@@ -222,8 +221,8 @@ Upgraded to .NET 6 versions of binaries.</PackageReleaseNotes>
   <PropertyGroup>
     <NBenchVersion>1.2.2</NBenchVersion>
     <XunitVersion>2.4.1</XunitVersion>
-    <TestSdkVersion>17.0.0</TestSdkVersion>
-    <RoslynVersion>3.11.0</RoslynVersion>
-    <NugetVersion>6.0.0</NugetVersion>
+    <TestSdkVersion>17.2.0</TestSdkVersion>
+    <RoslynVersion>4.2.0</RoslynVersion>
+    <NugetVersion>6.2.0</NugetVersion>
   </PropertyGroup>
 </Project>