This entry is part 2 in a series outlining different solutions for writing logs from an Azure hosted service.

So far the solutions include (subject to change):

1. Synchronizing a log file to blob storage
2. Using a custom log4net appender to write directly to table storage
3. Logging to the Trace log and synchronizing to table storage

These solutions are in order of complexity (in my opinion). Each topic will outline the structure of the solution, how to implement it, and the pros/cons of the solution compared to the others.

2. Using a custom appender to write to Table Storage

Compared to the previous solution this has fewer components, but requires more to setup. In Part 1 the Diagnostics Module did most of the work, the only requirement was a location that contained log files. There were 2 main phases, writing to a file then synchronizing the file to blobs. There was no dependency between the mechanism creating the file (log4net) and the sync process (Diagnostics Module).

In this solution we will create a log4net appender which takes logging events and persists them directly to table storage. Here we will have a direct dependency between log4net and Azure Table Storage. Even with that dependency this solution can be configured into an existing application because the code itself only interacts with the log4net API.

Before I start the implementation, this code is based off a blog post and source code by Vidar Kongsli.

Table Storage Entity

An Entity object is required to store data to Table Storage. This can contain as much or as little data as needed.

public sealed class LogEventEntity : TableServiceEntity
{
    public LogEventEntity()
    {
        var now = DateTime.UtcNow;
        PartitionKey = string.Format("{0:yyyy-MM}", now);
        RowKey = string.Format("{0:dd HH:mm:ss.fff}-{1}",
                                now, Guid.NewGuid());
    }

    public string Identity { get; set; }
    public string ThreadName { get; set; }
    public string LoggerName { get; set; }
    public string Level { get; set; }
    public string Message { get; set; }
    public string Domain { get; set; }
    public string RoleInstance { get; set; }
    public string DeploymentId { get; set; }
}

Some other implementations implement a subclass of Table Service Context but, as I mentioned in a previous post, I don’t think that is necessary.

log4net Appender Implementation

To implement a log4net appender just inherit from AppenderSkeleton. When this appender is activated we need to setup the table storage configuration and create a new table data context instance. For the purposes of this article I’ll assume that this appender will only be used in an Azure hosted service context and use the configuration from the Role Environment. Table Storage can be used from anywhere so to make this implementation more portable just add more configuration.

public class TableStorageAppender : AppenderSkeleton
{
    private string _connectionStringKey =
        "Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString";
    private string _tableName = "LogEntities";
    private TableServiceContext _dataContext;

    public string ConnectionStringKey
    {
        get { return _connectionStringKey; }
        set { _connectionStringKey = value; }
    }

    public string TableName
    {
        get { return _tableName; }
        set { _tableName = value; }
    }

    public override void ActivateOptions()
    {
        base.ActivateOptions();

        var connectionString = RoleEnvironment.GetConfigurationSettingValue(ConnectionStringKey);
        var account = CloudStorageAccount.Parse(connectionString);

        var tableClient = account.CreateCloudTableClient();
        tableClient.CreateTableIfNotExist(TableName);

        _dataContext = tableClient.GetDataServiceContext();
    }

    protected override void Append(LoggingEvent loggingEvent)
    {
        try
        {
            var entity = new LogEventEntity
                                {
                                    RoleInstance = RoleEnvironment.CurrentRoleInstance.Id,
                                    DeploymentId = RoleEnvironment.DeploymentId,
                                    Message = loggingEvent.RenderedMessage,
                                    Level = loggingEvent.Level.Name,
                                    LoggerName = loggingEvent.LoggerName,
                                    Domain = loggingEvent.Domain,
                                    ThreadName = loggingEvent.ThreadName,
                                    Identity = loggingEvent.Identity
                                };

            _dataContext.AddObject(TableName, entity);
            _dataContext.SaveChanges();
        }
        catch (DataServiceRequestException drex )
        {
            ErrorHandler.Error("Could not write log entry", drex);
        }
    }
}

Notice that 2 properties were added, for the configuration key the only reason that the diagnostics string is being used is out of convenience. This solution does not depend on the diagnostics module.

Configuration

Now that we have the storage entity and the appender the final step is to setup the appender in our log4net configuration section. When adding this implementation as a reference note that 3 other assemblies are needed:

Microsoft.WindowsAzure.ServiceRuntime

Microsoft.WindowsAzure.StorageClient

System.Data.Services.Client

To add this appender with no changes to the default configuration add the following to the app.config file

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <configsections>
    <section type="log4net.Config.Log4NetConfigurationSectionHandler, log4net" name="log4net" />
  </configsections>
  <log4net>
    <appender type="AzureLogAppender.TableStorageAppender, AzureLogAppender" name="TableStorageAppender">
    </appender>
    <root>
      <level value="INFO" />
      <appender-ref ref="TableStorageAppender" />
    </root>
  </log4net>
</configuration>

In this example I used a separate assembly for the appender (AzureLogAppender.dll) and referenced the assembly in the role project.

Pros / Cons

Pros: This solution requires some code but gives you full control over the logging component. Compared to the log files and trace logging, there are fewer components and less dependencies. With table storage the log messages can be easily queried for errors or trends.

Cons: The appender is less slightly slower then the out-of-the-box file appenders. Also, if left unmanaged the log table will continue to grow. These issues can be overcome with some thread management and a log truncating process.

So that is the table storage logging solution. The next solution involves a combination of the log4net configuration and diagnostic module to wrote to the Windows Azure Diagnostic Trace logs.

Thank you.