In distributed systems, message communication between services must be ensured with reliability. When we’re working with a microservices architecture and event-driven design, we typically have one primary concern: how to publish messages/events reliably when data changes? That’s when the Outbox Pattern comes in.

What is the Outbox Pattern?

The Outbox Pattern is a technique used to reliably publish events/messages to a message broker (like Kafka, RabbitMQ, etc.) only if the local database transaction succeeds. Instead of publishing messages directly, you:

  1. Save the event to a local Outbox table inside the same transaction that modifies the data.
  2. Use a background process (or another service) to read from the Outbox table and publish the messages.

This ensures atomicity between database changes and event publication.

Implementing the Outbox Pattern in C#

Suppose we have a CreateOrder operation that saves an order and then should notify other services that an order has been created.

Step 1: Define the Outbox Entity

public class OutboxMessage
{
    public Guid Id { get; set; }
    public string EventType { get; set; }
    public string Payload { get; set; }
    public DateTime OccurredOn { get; set; }
    public bool IsProcessed { get; set; }
}

Step 2: Modify the Domain Operation to Include Outbox Message

When performing a domain operation, such as creating an order, include the creation of an outbox message within the same transaction.

public async Task CreateOrderAsync(Order order)
{
    using var transaction = await _dbContext.Database.BeginTransactionAsync();

    try
    {
        _dbContext.Orders.Add(order);

        var outboxMessage = new OutboxMessage
        {
            Id = Guid.NewGuid(),
            EventType = "OrderCreated",
            Payload = JsonConvert.SerializeObject(order),
            OccurredOn = DateTime.UtcNow,
            IsProcessed = false
        };

        _dbContext.OutboxMessages.Add(outboxMessage);

        await _dbContext.SaveChangesAsync();
        await transaction.CommitAsync();
    }
    catch
    {
        await transaction.RollbackAsync();
        throw;
    }
}

Step 3: Implement a Background Service to Process Outbox Messages

A hosted service can periodically scan the outbox table for unprocessed messages and publish them to the message broker.

public class OutboxProcessor : BackgroundService
{
    private readonly IServiceScopeFactory _serviceScopeFactory;
    private readonly IMessagePublisher _messagePublisher;

    public OutboxProcessor(IServiceScopeFactory serviceScopeFactory, IMessagePublisher messagePublisher)
    {
        _serviceScopeFactory = serviceScopeFactory;
        _messagePublisher = messagePublisher;
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            using var scope = _serviceScopeFactory.CreateScope();
            var dbContext = scope.ServiceProvider.GetRequiredService<AppDbContext>();

            var messages = await dbContext.OutboxMessages
                .Where(m => !m.IsProcessed)
                .ToListAsync(stoppingToken);

            foreach (var message in messages)
            {
                try
                {
                    await _messagePublisher.PublishAsync(message.EventType, message.Payload);
                    message.IsProcessed = true;
                }
                catch
                {
                    // Log and handle exceptions as needed
                }
            }

            await dbContext.SaveChangesAsync(stoppingToken);
            await Task.Delay(TimeSpan.FromSeconds(10), stoppingToken);
        }
    }
}

Step 4: Ensure Idempotency in Message Handling

Since messages might be delivered more than once, ensure that the message handlers are idempotent, meaning processing the same message multiple times doesn’t have adverse effects.

Advanced Considerations

Scaling the Outbox Processor

At high-throughput systems, scale the outbox processor horizontally. Use locking or database features like SKIP LOCKED to prevent multiple processors from handling the same message simultaneously.

Monitoring and Alerting

Employ monitoring to track outbox table size and processing rate. Set alerts for instances where messages are not processed within a given threshold.

Handling Failures

Include retry operations with exponential backoff for transient failures during publishing messages. In case of permanent failures, employ dead-letter queues to isolate faulty messages for examination.

Integrating Kafka with the Outbox Pattern

Kafka is a widely adopted distributed event streaming platform, making it a natural fit for publishing events stored in an outbox. However, integrating Kafka within the Outbox Pattern introduces specific architectural and operational choices.

Why Use Kafka?

Kafka provides:

  1. High-throughput, low-latency message delivery.
  2. Persistent event logs for replayability.
  3. Consumer groups for scalability.

These features align well with the event-driven goals of the Outbox Pattern.

Kafka Outbox Flow

Here’s how Kafka fits into the Outbox flow:

  1. Service Transaction
    • A domain event (e.g., OrderCreated) is saved to the OutboxMessages table as part of the same DB transaction.
  2. Outbox Publisher (Kafka Producer)
    • A background service reads unprocessed messages from the outbox.
    • Converts each message into a Kafka event.
    • Publishes it to a Kafka topic (e.g., orders).
    • Marks the message as processed once confirmed.
  3. Kafka Consumers
    • Other microservices subscribe to the topic and react accordingly.

Sample Kafka Producer in C#

To publish outbox messages to Kafka, you can use the Confluent.Kafka NuGet package.

public async Task PublishToKafkaAsync(OutboxMessage message)
{
    var config = new ProducerConfig { BootstrapServers = "localhost:9092" };

    using var producer = new ProducerBuilder<Null, string>(config).Build();
    await producer.ProduceAsync("orders", new Message<Null, string> { Value = message.Payload });

    Console.WriteLine($"Published message to Kafka: {message.EventType}");
}

Schema Management with Kafka

Kafka doesn’t enforce schema constraints, so consider:

  • Using Avro or JSON Schema for payload structure.
  • Registering schemas with Confluent Schema Registry.
  • Versioning events with a Version field in the outbox.

This ensures downstream consumers handle changes safely.

Kafka Connect and CDC Alternative

For high-scale scenarios, consider bypassing the custom outbox processor and using Kafka Connect with Debezium to stream changes from the database directly:

  • Debezium monitors the OutboxMessages table via Change Data Capture.
  • Kafka Connect publishes events automatically.
  • Ideal for cloud-native architectures with strong DevOps maturity.

Conclusion

The Outbox Pattern is a great way to accomplish reliable, atomic event publishing in microservices. It desouples message transport from your application logic, avoids inconsistencies, and is fault-tolerant. Using C# with EF Core and a background publisher, you can apply it to your system with minimal effort and maximal reliability.

<
Previous Post
Real-Time Updates with Server-Sent Events (SSE) in .NET
>
Blog Archive
Archive of all previous blog posts