Debugging a Growing Memory Problem in a .NET Service

articleOctober 4, 2026Updated October 7, 2026By Ali Haghighi

A graph that keeps climbing is enough to get everyone's attention.

Memory usage rises over a few hours. The service is still responding, but eventually a restart brings memory back down. The obvious conclusion is a memory leak.

Maybe.

But "memory is growing" is an observation, not a diagnosis. In a .NET service, the cause could be retained managed objects, a cache doing exactly what it was configured to do, allocation pressure, native memory, large buffers, workload growth, or a combination of several things.

The useful question is not "do we have a leak?"

It is: what evidence would distinguish these possibilities?

Start with the shape of the problem

Before collecting a dump, I want to understand the timeline.

Does memory grow continuously or in steps? Does it follow traffic? Does it drop after a full GC? Does the process recover when load falls? Did the behavior start after a deployment? Is one replica different from the others?

A simplified incident might look like this:

TEXT
00  traffic increases
20  allocation rate increases
10  live managed heap starts climbing across full collections
30  process memory continues to grow with it
00  latency starts drifting upward
15  container approaches its memory limit

That sequence already gives me more to work with than a single memory number.

I also want to compare healthy and unhealthy instances when possible. One strange replica next to several stable ones is a useful clue.

Process memory and managed heap are not the same thing

This distinction prevents a lot of wasted debugging time.

A process can consume memory that is not represented by the live managed objects on the GC heap. Native libraries, runtime structures, thread stacks, memory-mapped files, and other allocations can contribute to process memory.

So I avoid treating this equation as true:

TEXT
high process memory = managed memory leak

Instead, I compare several signals.

At minimum I want process memory, GC heap size, allocation rate, Gen 0/1/2 collection activity and, where relevant, the Large Object Heap.

If process memory and managed heap rise together, I investigate one path. If the managed heap stabilizes while process memory keeps climbing, I investigate another.

That difference can save hours.

Use cheap evidence before expensive evidence

My preferred diagnostic path starts with information that is safe and inexpensive to collect.

For a .NET service, runtime counters can answer useful first questions without immediately capturing a large dump. I might watch GC heap size, allocation rate, Gen 2 collections, LOH size, working set, thread count, exception rate, request rate, and latency.

The exact tool is less important than the progression. dotnet-counters, dotnet-monitor, OpenTelemetry metrics, and the existing monitoring platform can all contribute evidence.

The point is to narrow the hypothesis before reaching for heavier diagnostics.

Look for retention, not just allocation

A high allocation rate is not automatically a memory leak.

.NET applications allocate constantly. Short-lived objects can create significant allocation traffic while the GC successfully reclaims them.

Retention is different.

If objects that should have become unreachable remain referenced, the live heap can grow across collections. Long-lived collections, unbounded dictionaries, event subscriptions, queues, and caches are common places to investigate, but I do not start by guessing which one is guilty.

I want the heap to tell me.

TEXT
metrics
  -> confirm persistent growth
  -> runtime counters
  -> identify managed vs non-managed direction
  -> capture targeted diagnostic evidence
  -> inspect dominant and retained objects
  -> form a code-level hypothesis

Only then do I want to change code.

A fictional example: the cache that looked reasonable

Imagine a service that enriches requests with reference data. To reduce calls to another service, it stores results in a concurrent dictionary:

C#
private readonly ConcurrentDictionary<string, ReferenceData> _cache = new();

public async Task<ReferenceData> GetAsync(
    string key,
    CancellationToken cancellationToken)
{
    if (_cache.TryGetValue(key, out var cached))
        return cached;

    var value = await _client.GetAsync(key, cancellationToken);
    _cache[key] = value;

    return value;
}

Nothing here looks dramatic.

But if the key space is effectively unbounded, this is not really a cache. It is a collection with no eviction policy.

Suppose traffic introduces new keys continuously. Allocation rate may look normal, CPU may be fine, and the service may remain healthy for hours. The problem only becomes visible over time.

A heap inspection showing a growing number of ReferenceData instances and dictionary entries would give us evidence for the hypothesis.

The fix is not "call GC more often." The design needs a bounded lifetime or bounded size.

For example, a dedicated MemoryCache can enforce both expiration and a maximum entry count:

C#
private readonly MemoryCache _cache = new(
    new MemoryCacheOptions
    {
        SizeLimit = 10_000
    });

public void Set(string key, ReferenceData value)
{
    _cache.Set(
        key,
        value,
        new MemoryCacheEntryOptions
        {
            AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(20),
            Size = 1
        });
}

Here the cache uses one size unit per entry, so SizeLimit = 10_000 means at most roughly ten thousand entries can be retained by this policy. Without a configured SizeLimit, setting Size on an entry does not bound the cache.

The important part is not this particular API. The important part is that the storage policy now matches the intended behavior.

A restart is evidence too, but weak evidence

If restarting the process immediately releases the memory, we have learned something: the memory belongs to the process lifetime.

We have not learned why it was retained.

Restarting a container can be a valid operational mitigation when the alternative is an outage. It should not become the diagnosis.

The same applies to increasing a Kubernetes memory limit. More headroom can reduce immediate risk, but it may simply make an unbounded problem take longer to fail.

Operational mitigation and root-cause correction are different decisions.

Validate the fix under a representative workload

A memory fix is not complete when the code compiles or when one local request succeeds.

I want to reproduce the relevant workload pattern and compare before and after.

| Signal | Before | After | | --- | --- | --- | | Process memory | Continues rising | Reaches a stable range | | Live managed heap | Grows with key count | Stabilizes after eviction | | Allocation rate | Similar | Similar | | Gen 2 behavior | Increasing pressure | Stable | | Request latency | Degrades over time | Stable |

These values are deliberately qualitative. The thresholds that matter depend on the application, workload, and environment.

The shape of the graph is often more informative than one impressive-looking number.

Watch the system after the fix

Once the immediate issue is resolved, I want monitoring that makes a recurrence visible before the service approaches its limit.

Memory usage without traffic context can be misleading. GC activity without allocation rate can be misleading. Latency without dependency information can be misleading.

The useful dashboard is the one that helps answer questions during an incident:

TEXT
Is memory growing?
Is the managed heap growing?
Is allocation pressure changing?
Are full collections recovering memory?
Does the behavior correlate with traffic or a particular operation?
Is the pattern present across replicas?
What changed before the growth started?

That is a much better starting point than waiting for an out-of-memory event.

The debugging habit that matters

Memory incidents are a good example of why production engineering is mostly about reducing uncertainty.

A rising graph creates urgency. Urgency creates a temptation to jump from symptom to explanation.

I try to keep the steps separate:

TEXT
observe
-> correlate
-> narrow the hypothesis
-> collect targeted evidence
-> change the smallest justified thing
-> reproduce
-> measure again

Sometimes the answer is a classic managed memory leak.

Sometimes it is not.

The goal is not to guess correctly on the first attempt. It is to build a diagnostic path that makes the wrong explanations disappear quickly.