Designing Tool-Calling AI Without Building an Agent Too Early

articleOctober 2, 2026By Ali Haghighi

Engineering Reliable AI Systems, part 3

While working on an incident-analysis experiment, I reached a point where the model needed one thing it did not have: a current metric.

It was tempting to describe the next step as making it an agent. But the requirement was much smaller. The model needed permission to request a bounded capability, receive the result and continue.

That distinction changed the implementation. I did not need a planner, long-running memory or a general autonomous loop. I needed one explicit tool boundary.

The word agent can make a fairly small requirement sound like an architecture. I now try to make the requirement earn that architecture first.

Give the model capabilities, not infrastructure

A useful tool boundary is narrow.

C#
public interface IMetricsProvider
{
    Task<MetricSnapshot> GetSnapshotAsync(
        string service,
        CancellationToken cancellationToken);
}

The model does not need to know whether the implementation reads Prometheus, an HTTP API or a test fixture.

The application owns that detail. The model sees a capability with a clear contract.

That keeps two concerns separate: deciding that a metric would be useful, and retrieving that metric safely.

Tool arguments are untrusted input

A tool call may look structured, but it still originates from model output.

I validate it like any other external input.

Service names should be checked. Required fields should be required. Time ranges should be bounded. Cancellation and timeouts should flow through the call. A model should not gain arbitrary access to infrastructure because it generated valid JSON.

The same applies to tool results in the other direction. I prefer returning the smallest useful representation rather than dumping an entire monitoring response into the next model turn.

Smaller contracts are easier to test and easier to reason about.

Keep the loop bounded

The simplest useful flow often looks like this:

TEXT
[User]
   |
   v
[LLM] -- tool request --> [Validation] --> [MetricsProvider]
  ^                                           |
  |--------------- tool result ---------------|
  |
  v
[Final Answer]

There should be limits around that loop.

The application boundary can enforce those limits without asking the model to behave:

C#
public async Task<MetricSnapshot> ExecuteAsync(
    string service,
    CancellationToken cancellationToken)
{
    using var timeout = CancellationTokenSource.CreateLinkedTokenSource(
        cancellationToken);

    timeout.CancelAfter(TimeSpan.FromSeconds(3));

    if (!_allowedServices.Contains(service))
        throw new ArgumentException("Unknown service.", nameof(service));

    return await _metrics.GetSnapshotAsync(service, timeout.Token);
}

I would normally keep a maximum tool-call count at the orchestration layer as well. A timeout protects one execution; a call budget protects the workflow.

How many tool calls are allowed? How long may a call run? What happens when the provider is unavailable? What gets logged? Which result is safe to send back to the model?

These questions are less exciting than agent autonomy, but they are the questions that determine whether the feature behaves predictably in production.

Observability matters more once tools exist

A plain model request already needs useful telemetry. Tool calling adds another boundary where failures and cost can hide.

I want to distinguish model turns from tool executions and capture enough information to reconstruct the sequence without logging secrets.

A trace might tell me:

TEXT
llm.turn -> tool.request -> metrics.read -> tool.result -> llm.turn

From there I can answer practical questions.

Did the model call a tool unnecessarily? Did the provider timeout? How many model turns did one request consume? Was the final answer based on the tool result or did it ignore it?

Without that evidence, debugging an AI workflow quickly becomes guesswork.

When an agent becomes justified

There are cases where a more agentic design earns its complexity.

A task may require planning across several independent capabilities, adapting the plan after intermediate results, maintaining state across a longer workflow or choosing among many possible next actions.

Even then, I want the requirement to force the architecture rather than the terminology.

If the workflow is known in advance, application code can often orchestrate it more clearly. If one model turn can request one or two bounded tools and produce the result, I keep it that way.

An agent framework is not a maturity level that every AI application eventually reaches.

It is an architectural trade-off.

The progression I prefer

For a new capability, my default progression is deliberately boring:

TEXT
model call
-> structured output
-> bounded tool calling
-> measured multi-step workflow
-> agentic orchestration, if the problem actually requires it

Each step should solve a limitation I can point to.

That makes the architecture easier to explain, test and replace. It also keeps experimentation cheap: I can learn whether the feature is valuable before committing to infrastructure built for autonomy I may never need.

Reliable AI systems are not defined by how many agent concepts they contain.

They are defined by whether we can understand what they did, constrain what they are allowed to do and evaluate whether the result was useful.