Migrating databases with EF Core 2.0 during a VSTS release

A while ago I wrote down how to migrate a database using the default tooling with EF Core 1.1. With 2.0 being released last August it was finally time to upgrade projects on which I’m working to the latest bits. A reader had already reported that the same approach as used with 1.1 no longer worked for EF Core 2.0.

Again I ran the update database command with the verbose option and I saw that the command itself has not changed. What has changed however is the location of EF.dll. Even when forcing the package restore to not use any caching and explicitly stating where the packages had to be installed, EF.dll just would not show up. It is however located (at least on my Mac) at /usr/local/share/dotnet/sdk/NuGetFallbackFolder/microsoft.entityframeworkcore.tools.dotnet/2.0.0/tools/netcoreapp2.0/ef.dll.

I tried for an hour or two to get this file in a zip archive on Windows and on macOS. The easy way out would be to just find the dll and add it to source control. Of course I went for another solution: download the EF.dll during a release and use it to update the databases.

I edited the release definition of VSTS and in my first attempt I tried to use the package management task which you get when you install this extension. Unfortunately this is not meant to be used to download public packages from the NuGet servers.

Second attempt was to use the Download file task. It was not able to download the NuGet package I needed giving me a 404 error.

So when everything fails, Powershell will fix it. With just a single line I was able to download the NuGet package containing EF.dll.

I used an inline script, but you could just create a script and add it to source control so you can use variables for the package name.

wget "https://www.nuget.org/api/v2/package/Microsoft.EntityFrameworkCore.Tools.DotNet/2.0.0" -outfile "EF.nupkg"

Then I extract all the files, since the build output contains one or more zip files and a nupkg file is also just a zip archive. My sample project just contained one zip.

Then move the EF.dll to the location where the assembly resides containing the DbContext and migrations. The final step is to execute the update command.

mv ./tools/netcoreapp2.0/ef.dll ./
dotnet exec --depsfile migratetest20.deps.json  --runtimeconfig migratetest20.runtimeconfig.json ef.dll database update --assembly migratetest20.dll --startup-assembly migratetest20.dll

Migratetest20 was my test project.

Migrating databases with EF Core during a VSTS release

I wanted to migrate databases during a release. The default way of doing this is to run code when the application starts. Great for little demo projects but not when you are releasing a real app to production. When you are pushing a new version to production and the migration fails the release should fail, alarms should go off etc. With the “old” Entity Framework I usually wrapped the migration code in a console application. I packaged it alongside the other code and then executed an additional release step to execute the migrations. Since I’ve been using .NET Core on my latest projects I wanted to see if I could use all the default tooling.

If you run the update command with the verbose option you will see that “dotnet exec” is called using “EF.dll” together with a bunch of other options. This is the behaviour I want to replicate during a release.

dotnet ef database update -v

The first thing to figure out is where to find “EF.dll”. This is of course in a NuGet package. By default they are downloaded in your user’s home folder. I was not able to figure out a way to just grab it from there using a hosted build agent. However by modifying the restore command I can specify where the packages need to be downloaded.

dotnet restore --packages /my/location

I then added an additional “Archive files” build step to my build definition. This step creates a zip file and adds it to the build artifacts staging directory. This means it will automatically be picked up with all the other deliverables.

Those are all the necessary changes for the build definition. Now the extra bits in the release definition. I need four additional steps, but it’s also important to select the correct agent. As I’ll be running .NET Core commands I need a VS2017 agent.

The first step is to unarchive the archive which has the appsettings.json with the connectionstring, in my case it’s the output of a project which houses the web api and an Angular app, the other archive we need has the EF Core bits we need.

I then need the correct connectionstring, this is of course a variable in my release environment. The easiest way to replace the connectionstring in appsettings.json of the unarchived files of the web application is to use an extension of VSTS. It’s a two step process, we first tokenize the file so the second step can find the placeholders.

The final step is to do the same command we execute on our development machine. The hard part is figuring out all the parameters. In my case the migrations (and all of the logic) is in a dll named: App.dll while the web application and the connectionstring is from another dll: App.Web.dll

exec --depsfile App.Web.deps.json --runtimeconfig App.Web.runtimeconfig.json ef.dll database update --assembly App.dll --startup-assembly App.Web.dll --verbose --root-namespace MyAppNamespace

And that’s it, now the database is upgraded during a release.

Managing settings for an Angular app with VSTS

Recently I needed to have client side settings in an Angular app. My app had to call some 3rd party webservices and their location was different per environment (test, QA and production), I also had to configure a couple of parameters that were necessary when calling them.

There’s support for this in Angular using “environments”. If you scaffold a new Angular app with angular-cli you’ll get a folder “Environments”.

environments

These files can be used while building the application. Specifying the environment will bundle the environment specific file together with the rest of your application. Which is great but I don’t want to rebuild the application when I go from test to QA and finally to production. Typically settings are changed when deploying to an environment, at least that’s what I usually do with my .NET projects.

Trying to have one unified way of handling the settings gives two challenges: how to bootstrap the Angular app with dynamic settings and how to manage these during deployment.

I added a client-config.json file in my asset folder to contain all the settings and the values which were correct during development.

client-config.json

This file needs to be downloaded before most of the application actually starts so I added a settings provider.

import { Http } from '@angular/http';
import { Injectable } from '@angular/core';
 
@Injectable()
export class SettingsProvider {
 
  private config : any;
 
  constructor(private http: Http) {
  }
 
  public loadConfig() : Promise<any>{
    return this.http.get("assets/client-config.json")
      .map(res => res.json())
      .toPromise()
      .then(settings => this.config = settings);
  }
 
  public get configuration(): any {
    return this.config;
  }
}

Then in my app.module.ts I add a new function to load the configuration.

export function init(settingsProvider: SettingsProvider) {
  return () => settingsProvider.loadConfig();
}

I also changed the NgModule declaration to include

  providers: [{
    'provide': APP_INITIALIZER,
    'useFactory': init,
    'deps': [SettingsProvider],
    'multi': true
  },
  SettingsProvider]

The crux here is the APP_INITIALIZER token which Angular provides to run logic before the application starts. You can read more about it here and here.

With the client side now done, the application still needs to get the correct values when the application is pushed to different environments. This is the easy part as VSTS has support for file transforms and variable substitution. I just add my client-config.json to the list of files that need to be changed. VSTS will replace any variable it finds that can be matched with variables defined for the environment where I deploy.

VSTS Config

The client-config.json is also added to the list of files the Angular build needs to pick up.

 "apps": [
    {
      "root": "src",
      "outDir": "../wwwroot",
      "assets": [
        "assets",
        "favicon.ico",
        "client-config.json"
      ],
      ....
    }]

The end result is a single way to manage environment specific settings for my .NET Core and Angular applications.