Tag: Azure

Azure WebJobs not logging to Application Insights

~ Nish ~ Leave a comment

With regular App Services, if you enabled Application Insights, your ILogger logs will get written into Application Insights without having to do anything special. This is not the case with WebJobs though. After wasting a few hours trying to figure out why my logs were not showing up, here’s a solution I found that worked.

Step 1

Install the latest version of Microsoft.Azure.WebJobs.Logging.ApplicationInsights.

Step 2

Modify your startup code as follows.

public static void Main(string[] args)
{
    IHost host = Host.CreateDefaultBuilder(args)
        .ConfigureServices(services =>
        {
            services.AddHostedService<Worker>();
        })
        .ConfigureLogging((context, builder) =>
        {
            builder.AddConsole();

            var key = context.Configuration["APPINSIGHTS_INSTRUMENTATIONKEY"];
            if (!string.IsNullOrEmpty(key))
            {
                builder.AddApplicationInsightsWebJobs(
                    options => options.InstrumentationKey = key);
            }
        })
        .Build();

    host.Run();
}

And that’s it. Remember that there’s usually a 1-2 minute delay before your logs show up in App Insights, and it could be as high as 3-4 minutes if it’s the first time you deploy. So, don’t panic if you don’t see the logs right away. Happy debugging!

Implementing asynchronous endpoints in REST with callbacks

~ Nish ~ Leave a comment

(click on the image to view it in full resolution)

This is an extension of the asynchronous endpoint pattern where the consumer can register a callback URL, which is invoked when the task is completed. This only works if the consumer has the option to expose an externally hittable REST endpoint. The advantage is that the consumer does not have to keep polling the service to see if the task has completed.

Once the task completes, the service invokes the callback URL. A common approach is to send just a task id to the callback, and the consumer then invokes a call to the service to fetch the full payload. A less common approach (if sensitive info is not included in the payload) is to send the entire result-set to the callback endpoint.

On the service side, the implementation is done by reacting to the task completed event by invoking the registered callback for the completed task. There are several ways to do this. A simple approach is for the task worker itself to invoke the callback. The disadvantage there is that it’s complex to now support retries if the callback endpoint is down. With AWS, a common approach is to update the data in a DynamoDB table and for a Lambda to be triggered off that, and the Lambda would then invoke the callback. A similar approach for Azure is to have a Function triggered off Table storage or Cosmos DB, which would invoke the callback. Both AWS and Azure have mechanisms to add retry to the function/lambda, to handle scenarios where the callback URI is down.

Implementing asynchronous endpoints in REST

~ Nish ~ Leave a comment

When you have an API endpoint that takes too long to run, it can result in consumers facing time-outs or blocking scenarios. Now while consumers can use background workers to make these long running calls, it’s far more convenient if the service is implemented to support an asynchronous operation. A very common approach is to queue up the work on the service side and to return a task-id to the consumer. The consumer can then periodically poll the API with the task-id which returns a response indicating whether the task has been completed or not. Once the task completes, the consumer then makes a call to retrieve the completed task response. In some implementations, these two operations are combined, so it’s the same endpoint that both returns the pending status as well as the completed result.

On the service side, the implementation is typically done using a queue. The first call generates and persists a task-id which is returned, and the task is queued up in a queue (such as AWS SQS or Azure Queues). You can then trigger off a worker via AWS Lambdas or Azure Functions which performs the task and then marks the task as completed.