Azure Service Bus (4) — Outbox pattern

The last part of the series about Azure Service Bus. We have the basics, producers and consumers, and failure handling. The most subtle problem remains: consistency between the database and the published messages.

The dual-write problem

The code from part 1 looked like this:

public async Task<IActionResult> PlaceOrder(OrderRequest request)
{
    var order = await _orderRepository.CreateAsync(request);      // write 1: DB
    await _publisher.PublishOrderPlacedAsync(new OrderPlacedEvent(order)); // write 2: Service Bus
    return Accepted(order);
}

Two writes in two different systems, without a common transaction. What can go wrong:

• DB succeeds, publish fails — the order exists, but no service learns about it. Stock is not reserved, the email is not sent. Phantom order.
• Publish succeeds, but the process dies before commit (in the reverse order) — services react to an order that does not exist in the DB.

You cannot solve this with try/catch: the process can die anytime between the two writes (deploy, OOM, scale-down — exactly the frequent events in Container Apps). Distributed transactions (2PC) are also not a realistic option: Cosmos DB and Service Bus do not participate in a common transaction.

The Outbox idea: a single atomic write

The solution: you don't write to two systems. You write only once, in the database, both the state and the event to be published — atomically. A separate process (dispatcher) reads unpublished events and sends them to Service Bus.

1. [DB Transaction] save Order + OutboxEvent (atomic)
2. [Dispatcher]    read unpublished OutboxEvents
3. [Dispatcher]    publish to Service Bus
4. [Dispatcher]    mark event as published

If the process dies between steps 1 and 3, the event remains in the outbox and the dispatcher publishes it on the next run. The guarantee: if the state was saved, the event will be published (eventually). The semantics are at-least-once — we will return to this.

Implementation with Cosmos DB

In Cosmos DB, atomicity exists at the partition key level: a transactional batch writes multiple documents atomically if they have the same partition key. We store the outbox event in the same container as the order, on the same partition:

public class OutboxEvent
{
    public string Id { get; set; } = Guid.NewGuid().ToString();
    public string Type { get; set; } = "OutboxEvent";   // discriminator
    public string OrderId { get; set; } = default!;      // common partition key
    public string EventType { get; set; } = default!;    // "OrderPlaced"
    public string Payload { get; set; } = default!;      // event JSON
    public bool Published { get; set; }
    public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
    public DateTime? PublishedAt { get; set; }
}

public async Task CreateOrderAsync(Order order)
{
    var outboxEvent = new OutboxEvent
    {
        OrderId = order.Id,
        EventType = "OrderPlaced",
        Payload = JsonConvert.SerializeObject(new OrderPlacedEvent(order))
    };

    // ATOMIC write: both documents or none
    var batch = _container.CreateTransactionalBatch(new PartitionKey(order.Id))
        .CreateItem(order)
        .CreateItem(outboxEvent);

    var response = await batch.ExecuteAsync();
    if (!response.IsSuccessStatusCode)
        throw new InvalidOperationException(
            $"Saving the order failed: {response.StatusCode}");
}

Here is all the magic: CreateTransactionalBatch guarantees that Order and OutboxEvent are saved together or not at all. The dual-write problem disappeared — there is only one write.

The dispatcher

A BackgroundService that runs periodically, reads unpublished events, and sends them to Service Bus:

public class OutboxDispatcherService : BackgroundService
{
    private readonly Container _container;
    private readonly ServiceBusSender _sender;
    private readonly ILogger<OutboxDispatcherService> _logger;

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            try
            {
                await DispatchPendingAsync(stoppingToken);
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "Error dispatching outbox");
            }

            await Task.Delay(TimeSpan.FromSeconds(5), stoppingToken);
        }
    }

    private async Task DispatchPendingAsync(CancellationToken ct)
    {
        // Cross-partition query -- sort in memory (Cosmos DB lesson!)
        var query = new QueryDefinition(
            "SELECT * FROM c WHERE c.Type = 'OutboxEvent' AND c.Published = false");

        var pending = new List<OutboxEvent>();
        using var iterator = _container.GetItemQueryIterator<OutboxEvent>(query);
        while (iterator.HasMoreResults)
            pending.AddRange(await iterator.ReadNextAsync(ct));

        foreach (var evt in pending.OrderBy(e => e.CreatedAt))
        {
            var message = new ServiceBusMessage(evt.Payload)
            {
                ContentType = "application/json",
                MessageId = evt.Id,          // deduplication on event id
                Subject = evt.EventType,
                CorrelationId = evt.OrderId
            };

            await _sender.SendMessageAsync(message, ct);

            // Patch with EXACT casing of stored document
            await _container.PatchItemAsync<OutboxEvent>(
                evt.Id,
                new PartitionKey(evt.OrderId),
                new[]
                {
                    PatchOperation.Set("/Published", true),
                    PatchOperation.Set("/PublishedAt", DateTime.UtcNow)
                },
                cancellationToken: ct);
        }
    }
}

Two details from the Cosmos DB lessons in the series: the cross-partition query does not use ORDER BY (we sort in memory), and patch operations use exact casing of stored fields (/Published, not /published) — otherwise you get 500 errors.

At-least-once and idempotent consumers

Look carefully at the dispatcher: if the process dies after SendMessageAsync but before the patch, the event remains unpublished in the outbox and will be sent again. The semantics are at-least-once: we guarantee publishing, but possibly multiple times.

Two layers of defense:

1. Duplicate detection in Service Bus

# Deduplication window on MessageId (max 7 days on Standard)
az servicebus queue create \
  --name order-processing \
  --namespace-name my-servicebus \
  --resource-group my-rg \
  --enable-duplicate-detection true \
  --duplicate-detection-history-time-window PT30M

Because MessageId is the deterministic outbox event id, Service Bus automatically discards duplicates sent within the configured window.

2. Idempotency in the consumer

Duplicate detection does not cover everything (limited window, redeliveries after abandonment). The last line of defense is the idempotent consumer — processing the same event twice has the effect of a single processing:

public async Task HandleAsync(OrderPlacedEvent evt, CancellationToken ct)
{
    // Check: have we already processed this event?
    var alreadyProcessed = await _processedEventsRepository
        .ExistsAsync(evt.EventId, ct);
    if (alreadyProcessed)
    {
        _logger.LogInformation("Event {EventId} already processed -- skip", evt.EventId);
        return;
    }

    await _inventoryService.ReserveStockAsync(evt.OrderId, evt.Items, ct);

    // Mark as processed -- ideally in the same transaction as the effect
    await _processedEventsRepository.MarkProcessedAsync(evt.EventId, ct);
}

Ideally, the check and marking are in the same transaction as the business effect (the same transactional batch in Cosmos DB) — otherwise you have recreated the dual-write problem at the consumer level.

Outbox checklist + series conclusion

• State + event saved atomically — transactional batch on the same partition key
• Dispatcher separate from request — BackgroundService with polling, tolerant to failures
• Deterministic MessageId — the outbox event id, for deduplication
• Duplicate detection enabled — first net for at-least-once
• Idempotent consumers — second net, mandatory
• Old events cleaned up — TTL on published documents (Cosmos DB does the cleanup for free)

With this, the Service Bus series is complete: from the sync/async decision, through implementing producers and consumers, through failure handling with DLQ and retry, up to guaranteed consistency with Outbox. Together with the mini-series on Container Apps (where KEDA scales exactly these consumers), you have a complete and resilient asynchronous architecture.

If you have questions or want to discuss how to apply the Outbox pattern in your project, write to me at contact@ludoprogramming.com.