Structured Logging in ASP.NET Core With log4net

Log4net is a very popular open-source logging library for .NET applications. This library is part of the Apache Logging Services project at the Apache Software Foundation. Log4net provides a flexible way of writing logs to different output targets like files, consoles, databases, emails, etc.

Structured Logging is a better way of logging which involves capturing logs in a structured format like key-value pairs or JSON objects rather than plain text. This makes it easier to query and analyze log data especially while integrating with third-party log management systems. In this article, we are going to learn how to write structured logs with log4net.

So let’s get going. 

Configure the Project

Let’s start by creating a new ASP.NET Core Web API application by running the dotnet new webapi command or using the Visual Studio Project wizard.

The first step is to install the log4net NuGet package. We can do that by running the Install-Package command in the Package Manager Console of Visual Studio:

Install-Package log4net

This will add the log4net library to our application.

Log4net Configuration File

Log4net allows us to fully control how we write logs by using an XML configuration file. So the next step is adding the log4net configuration file. Let’s create a log4net.config XML file in the application’s root folder:

<log4net>
  <appender name="RollingFile" type="log4net.Appender.FileAppender">
    <file value="app.log" />
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%-5p %d{hh:mm:ss} %message%newline" />
    </layout>
  </appender>
   <root>
    <level value="ALL" />
    <appender-ref ref="RollingFile" />
  </root>
</log4net>

In the config file, we define an appender node to configure the appenders. An appender is responsible for writing log messages to a specific output target, such as a file or a database.

Here, we configure a file appender, which will write logs to a file in the location that we specify. Also, we are using a layout node to format the log messages.

Note that we have provided a conversion pattern to format the log event as a string. %-5p denotes the log level which is left justified to a width of five characters. Similarly, %d{hh:mm:ss} denotes the date in the defined format. Finally, we write the message and a new line. 

We’ll discuss appenders and layouts more later in the article.

Similarly, we can use the root node to configure the root logger. Within the root logger, we can set the log level, specify which appenders to use for logging messages, etc.

It is worth noting, in case the log4net.config file is not included as part of the published build, it might be required to set the Copy to Output Directory property of the file to Copy Always to make sure that this is always included while publishing the application.

Add the log4net Service 

The next step is to configure the log4net service and add it to the dependency injection container. Let’s create an extension method for this purpose:

public static class Log4netExtensions
{
    public static void AddLog4net(this IServiceCollection services)
    {            
        XmlConfigurator.Configure(new FileInfo("log4net.config"));                        
        services.AddSingleton(LogManager.GetLogger(typeof(Program)));
    }
}

Here, we use the XmlConfigurator.Configure() method to load the log4net configuration file by specifying the path to the configuration file. After that, we register the log4net service as a singleton instance with the DI container.

After creating the extension method, we call it from the Program class:

public static void Main(string[] args)
{
    var builder = WebApplication.CreateBuilder(args);

    // code removed from brevity

    builder.Services.AddLog4net();
    var app = builder.Build();

    // code removed from brevity

    app.Run();
}

That’s it. This will configure log4net in our application.

Here we have loaded the configuration file manually, created the log4net instance, and added it to the DI container. But to be honest, this is not in line with the ASP.NET Core Logging framework. Ideally, we should try to configure log4net as a logging provider extension. 

The Microsoft.Extensions.Logging.Log4Net.AspNetCore NuGet package allows us to configure Log4net as Microsoft Extensions Logging handler in an ASP.NET Core application. But as of date, it supports only up to .NET 6 and there is no confirmation of support for further versions. 

Another option is to implement the logging provider and extension methods ourselves, which is an advanced topic and hence we are not covering that in this article.

Write Logs With log4net 

With log4net now configured, we can inject it into the controllers and use it to write logs:

public class LogTestController : ControllerBase
{
    private readonly ILog _logger;

    public LogTestController(ILog logger) 
    {  
        _logger = logger ?? throw new ArgumentNullException(nameof(logger)); 
    }

    [HttpGet]
    public IActionResult Get() 
    {
        _logger.Info("This is an informational log.");
        _logger.Error("This is an error log.");

        return Ok("Test success!");
    }
}

We inject the ILog instance into the controller and use it to write the logs. In the Get() method, we add an informational log using the ILog.Info() method and an error log using ILog.Error() method.

Now let’s run the application and navigate to the /logtest endpoint.

Since the endpoint has code to write logs to a log file, it will generate a log file in the path that we specify in the config file, if it does not exist. On opening the file, we can see the messages that we logged:

INFO  08:59:40 This is an informational log.
ERROR 08:59:40 This is an error log.

Create Structured Logging in log4net

We saw how to write logs as plain text messages. However, it is possible to write logs in a structured format such as key-value pairs or JSON objects. Unlike plain log messages, different log management tools can easily parse and analyze structured logs.

Of course, it is possible to pass an object directly into log4net methods:

_logger.Info(new { id = 1, name = "John" });

Then log4net will serialize the object while writing the logs:

INFO 09:42:10 { id = 1, name = John }

However, if we want to log the entire message as JSON, we have to use some additional NuGet packages.

Several NuGet packages allow us to write logs in JSON format with log4net. We are going to use log4net.Ext.Json NuGet package in this example.

First, let’s install the log4net.Ext.Json package:

Install-Package log4net.Ext.Json

Then, let’s modify the config file and change the layout type of our appender and configure the appropriate decorator:

<log4net>
    <appender name="RollingFile" type="log4net.Appender.FileAppender">
        <file value="C:\Temp\app.log" />
        <layout type='log4net.Layout.SerializedLayout, log4net.Ext.Json'>			
            <decorator type='log4net.Layout.Decorators.StandardTypesDecorator, log4net.Ext.Json' />
            <default />
        </layout>
    </appender>
    <root>
        <level value="ALL" />
        <appender-ref ref="RollingFile" />
    </root>
</log4net>

These configurations will format the log messages in a pre-defined JSON format. The application will now log the messages in JSON format:

{
   "date":"2023-04-19T22:07:45.0782319+05:30",
   "level":"INFO",
   "logger":"StructuredLoggingUsingLog4Net.Program",
   "thread":"14",
   "ndc":"(null)",
   "message":"This is an informational log."
}
{
   "date":"2023-04-19T22:07:45.1399917+05:30",
   "level":"ERROR",
   "logger":"StructuredLoggingUsingLog4Net.Program",
   "thread":"14",
   "ndc":"(null)",
   "message":"This is an error log."
}
{
   "date":"2023-04-19T22:07:45.1403932+05:30",
   "level":"INFO",
   "logger":"StructuredLoggingUsingLog4Net.Program",
   "thread":"14",
   "ndc":"(null)",
   "message":"{ id = 1, name = John }"
}

This way we can create structured log messages in JSON format using log4net.

It is possible to customize the fields in the serialized JSON as well as the format by writing our custom layout classes and decorators. It is a bit of an advanced topic and hence we are not covering it in this article.

Different Log Levels in log4net

Log4net has different log levels for classifying and prioritizing the log messages. We can classify the logs based on their priority, severity, etc. Remember how we used Info and Error log levels in our example.

That said, let’s inspect the different log levels in log4net in the order of increasing severity:

• DEBUG: We can use the ILog.Debug() method for writing a log with debug level which is mainly used for debugging purposes. This is a good solution if we want to provide detailed information about the application’s execution during debugging and testing phases.

• INFO: The ILog.Info() method allows us to write a log with information level. This log level is best suited for logging any information about the application as it runs.

• WARN: A warning level log is the best option when we want to indicate that something unexpected has happened, but the application can continue running. For writing a warning-level log, we use the ILog.Warn() method.

• ERROR: We can use error level logs to indicate that some error has occurred and the application failed to perform some tasks. For writing an error log, we can use the ILog.Error() method.

• FATAL: Sometimes some critical errors can occur in our application which can make it crash or become unresponsive. During these times, we can use the ILog.Fatal() method to indicate that a fatal error has occurred.

By using the appropriate log level, we can write logs with different severity, priority, etc. This will help us to easily control and filter the log messages that log4net generates which will be helpful when troubleshooting or analyzing the logs.

Log Appenders in log4net

In log4net, the appenders are responsible for controlling how to write log messages to various destinations, such as a file, console, database, email, etc. We can control how and where the log messages are stored, as well as the message format using appenders.

Conclusion

In this article, we learned how to write unstructured in an ASP.NET Core application using log4net. However, this is the recommended approach, so next we looked at writing structured logs with log4net. Additionally, we discussed different log levels and touched upon what are log appenders in log4net.