Tag Archives: core

Hangfire on .Net Core & Docker

This is going to be a lengthy one, but I did some setup for Hangfire to run in a Docker container (on my Ubuntu server at home)and I thought it’d be pretty exciting to share -given where we are in the .Net lifecycle/ecosystem.

What exactly are we setting up?

So, as part of my software infrastructure at home, I was in need of a job scheduler.  Not because I run a business, but because this is what I do for…um…fun.  I’m starting to have some disparate apps and API’s that are needing some long running, durable job handling, so I selected Hangfire based on their early adoption of Core.

I also completed my Ubuntu server build/reimage this past summer, and I was looking to be able to consistently “Dockerize” my apps, so that was a key learning experience I wanted to take away from this.

So here’s the stack I used to complete this whole thing:

  • Hangfire Job Scheduler
  • Docker – you’ll need the Toolbox if you’re developing on Windows/Mac
  • Hosted on my Server running Ubuntu 16.04 (but you can run the image on your local toolbox instance as a PoC.

The easiest place to start is getting Hangfire up and running.  I’ll  skip over my Postgres and Ubuntu setup, but that stuff is widely covered in other documentation.  I’ll have to assume you have a library for your job store that targets Core (I know MongoDB is dangerously close to finalizing theirs, and they have a Docker Image to boot!).  The one I used is shown below in my project.json.

So, spool up a brand new Asp.Net Core app; I made mine a Web Api with no security.   You can name it Hangfire.Web if you want to exactly follow along, but it really doesn’t matter, as long as you spot the areas where it would need to be changed.

In your program.cs, comment the IIS integration code.  We’ll be running Kestrel on a Linux VM via the Asp.Net Docker Image.

capture

Add your job store connection string to your appsettings.json.

Next up, tweaking your project.json.  I did a few things here, and I’ll post mine for your copy pasting pleasure.  The important parts are removing any IIS packages and pre/post publish scripts/tools.  By default, a new project will come with a couple of IIS publish scripts, and they will break your build/publish if you run only Kestrel in the container.


{
"dependencies": {
"Microsoft.NETCore.App": {
"version": "1.0.1",
"type": "platform"
},
"Microsoft.AspNetCore.Mvc": "1.0.1",
"Microsoft.AspNetCore.Routing": "1.0.1",
"Microsoft.AspNetCore.Server.Kestrel": "1.0.1",
"Microsoft.Extensions.Configuration.EnvironmentVariables": "1.0.0",
"Microsoft.Extensions.Configuration.FileExtensions": "1.0.0",
"Microsoft.Extensions.Configuration.Json": "1.0.0",
"Microsoft.Extensions.Logging": "1.0.0",
"Microsoft.Extensions.Logging.Console": "1.0.0",
"Microsoft.Extensions.Logging.Debug": "1.0.0",
"Microsoft.Extensions.Options.ConfigurationExtensions": "1.0.0",
"Hangfire": "1.6.6",
"Hangfire.PostgreSql.NetCore": "1.4.3",
"Serilog.Extensions.Logging": "1.2.0",
"Serilog.Sinks.Literate": "2.0.0",
"AB.FileStore.Impl.Postgres": "1.0.0",
"ConsoleApp1": "1.0.0"
},

"frameworks": {
"netcoreapp1.0": {
"imports": [
"dotnet5.6",
"portable-net45+win8"
]
}
},

"buildOptions": {
"emitEntryPoint": true,
"preserveCompilationContext": true
},

"runtimeOptions": {
"configProperties": {
"System.GC.Server": true
}
},

"publishOptions": {
"include": [
"wwwroot",
"**/*.cshtml",
"appsettings.json",
"web.config",
"Dockerfile"
]
}
}

You could honestly get rid of most of it for a bare-bones dependency build, but I left alot of defaults since I didn’t mind.
Next, Startup.cs:


public void ConfigureServices(IServiceCollection services)
{
// Add framework services.
services.AddMvc();

services.AddHangfire(options => options
.UseStorage(new PostgreSqlStorage(Configuration["PostgresJobStoreConnectionString"]))
.UseColouredConsoleLogProvider()
);

}

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory, IApplicationLifetime appLifetime)
{
loggerFactory.AddConsole(Configuration.GetSection("Logging"));
loggerFactory.AddDebug();

loggerFactory.AddSerilog();
// Ensure any buffered events are sent at shutdown
appLifetime.ApplicationStopped.Register(Log.CloseAndFlush);

app.UseMvc();
//Hangfire:
//GlobalConfiguration.Configuration
// .UseStorage(new PostgreSqlStorage(Configuration["PostgresJobStoreConnectionString"]));

app.UseHangfireServer();
app.UseHangfireDashboard("/hangfire", new DashboardOptions()
{
Authorization = new List() { new NoAuthFilter() },
StatsPollingInterval = 60000 //can't seem to find the UoM on github - would love to know if this is seconds or ms
}
);
}

And the NoAuth filter for Hangfire:


using Hangfire.Dashboard;
using Hangfire.Annotations;

public class NoAuthFilter : IDashboardAuthorizationFilter
{
public bool Authorize([NotNull] DashboardContext context)
{
return true;
}
}

 

Couple notes:

  • The call to PostgresStorage will depend on your job store.  At the time of writing I found a few Postgres packages out there, but this was the only one that built against .Net Core.
  • Serilog logging was configured, but is completely optional for you.  Feel free to remove it.
  • Why the Authorization and NoAuthFilter?  Hangfire, by default, authorizes its dashboard.  While I admire the philosophy of “secure by default”, it took me extra time to configure a workaround for deploying to a remote server that is still  in a protected environment, and I didn’t want to mess around with plugging in Authorization.  You’d only find that out after you deployed the Hangfire app.
  • Stats polling interval is totally up to you.  I used a (very) long interval since the job store wasn’t really doing anything.  To get the stats I need to consciously navigate to that web page, and when I do, real-time isn’t a critical feature for me.

At this point, you have everything you need to hit F5 and run your Hangfire instance on local.  Now would be a good time to double check your job stores work, because next we’re moving on to…

maxresdefault

DOCKER!

The Big Idea

The idea is, we want to grab the Docker Image for Asp.Net Core, build our source code into it, and be able to run a container from it anywhere.  As you’ll see, we can actually run it locally through Docker Toolbox, and then transfer that image directly to Ubuntu and run it from there!

We’re going to prep our custom image (based on the aspnet core Docker image here).  We do that by creating a Dockerfile, which is a DSL that will instruct Docker on how to layer the images together and merge in your DLL’s.

Note that the Docker for Visual Studio tooling is in preview now, and after experiencing some build issues using it, I chose to just command line my way through.  It’s easy, I promise.

First, create a new file simply called ‘Dockerfile’ (no file extension) in your src/Project folder:

capture
Your Dockerfile:


FROM microsoft/aspnetcore:latest
MAINTAINER Andrew
ARG source=.
WORKDIR /publish
EXPOSE 1000
COPY $source .
ENTRYPOINT ["dotnet", "Hangfire.Web.dll"]

Let’s take a look at what this means.  The ‘FROM’ directive tells Docker to pull an image from the Docker hub.  MAINTAINER is fully optional, and can be left out if you’re paranoid.  ARG, COPY, and WORKDIR work together to set the current folder as a variable, then reference the publish folder from that variable, copying in its contents (which will be your DLL’s in just a moment).  ENTRYPOINT is what Docker will call into once the host boots up the image.  You can call ‘dotnet Hangfire.Web.dll’ straight from your bin folders to double check.  Keep in mind the DLL name in ENTRYPOINT will be whatever you named your project.

To make life a bit harder, I decided to use a specific port via the EXPOSE directive.  I chose an arbitrary number, and wanted to be explicit in my host deployment port assignments.

See that publish folder from above?  We’re going to create that now.  I didn’t want to mess around with publish profiles and Visual Studio settings, so now is where we go into command line mode.  Go ahead and call up the Docker Quickstart terminal.  We can actually call into the dot net core CLI from there, so we’ll do that for brevity.

capture

Make sure kitematic is running your Linux VM.  Mine is going through Virtual Box.  I couldn’t tell you if the process is the same for Hyper-V driven Windows containers.  You might hang at the above screenshot if the Linux VM isn’t detected/running.

‘cd’ into that same project folder where the Dockerfile is and run a dotnet publish.  You can copy mine from here, which just says publish the Release configuration into a new folder called ‘publish’.

capture

cd ‘C:\Users\Andrew\Desktop\ProjectsToKeep\Hangfire\src\Hangfire.Web’

dotnet publish -c Release -o publish

Now, we have freshly built DLL’s.  We call into the Docker CLI which will pull the necessary image, and merge in that folder we referenced.

docker build ./publish -t hangfireweb

The -t argument is a tag.  It’s highly recommended to assign a tag as you can use that name directly in the CLI. If you get errors like “error parsing reference”, then it’s probably related to the tag.  I noticed some issues related to symbols and capital letters.

capture

Bam!  Our image is built!

I can prove it with this command:

docker images

capture

This next command will take a look at the image, and run a new container instance off of it.

docker run -it -d -e “ASPNETCORE_URLS=http://+:1000” -p 1000:1000 –name Hangfire hangfireweb

–name assigns the container a name so we can verify it once it’s live.

-d runs it in a background daemon.

-e will pass in environment variables.  These are variables passed into Docker when its constructing the container, and in this case, Asp.Net defaulted to port 80 (as it should) – but you’ll remember I explicitly instructed the container to only expose port 1000, so I need to also tell Asp.Net to listen on port 1000.  You can view other environment variables for each image on the Docker hub site or in the Toolbox.  Additionally, the -p argument maps the host port to the container port.  In this case, I opened up 1000 and mapped it to 1000.

You’ll get some output, and can confirm the container is up and running with this call:

docker ps

capture

(Keep in mind, if you restart the computer or otherwise stop the container, you can view all containers via:

docker ps -a

You can navigate to the Hangfire UI to make sure everything is dandy –

capture

That’s all!  To run the image from my Ubuntu box I just used docker save and docker load commands. (reference here.)  All you’re really doing is saving the image to a file, and loading it up from another server.  Nothing to it.  You can even keep the Toolbox instance running, spool up a second, and the 2 will compete over the job store.

Hopefully this was helpful!


I’ll sound off with a more urban, electronic/hip-hop fusion along the lines of Griz or Gramatik.  I found the album Brighter Future by Big Gigantic.  This is fun stuff, but will definitely be appreciated by a select few of you.