Documentation

Nested Traces

Represent parent-child execution spans.

Overview

Nested tracing is the mechanism that lets a parent @trace function automatically collect steps from child functions and SDK integrations that execute within its scope. This produces a complete execution graph in a single trace document.

How It Works

Nesting is powered by a contextvars.ContextVar named _current_trace_context. When a @trace-decorated function starts, it creates a new context dict with an empty collected_steps list and stores it on the ContextVar. Child decorators and integrations read the same ContextVar and append their step records to that list.

context_mechanism.pyCopy

python

# Parent sets the context:
ctx_token = _current_trace_context.set({
    "collected_steps": [],
    "retry_count": 0,
    ...
})

# Child appends to the parent's list:
def _try_append_step(step):
    ctx = _current_trace_context.get()
    if ctx is not None:
        ctx["collected_steps"].append(step)

# When parent finalizes, it reads collected steps:
ctx = _current_trace_context.get()
if ctx and not steps:
    steps = ctx.get("collected_steps", [])

The @trace_tool Decorator

The @trace_tool decorator is designed to instrument individual tool calls within a traced function. It records each invocation as a step and attaches it to the nearest parent @trace context. It also supports automatic retries with exponential backoff.

Parameter	Type	Default	Description
name	str or None	None	Tool name (defaults to function name)
max_retries	int	0	Number of automatic retries on failure
capture_input	bool	true	Capture function arguments as step input
capture_output	bool	true	Capture return value as step output

trace_tool.pyCopy

python

from tracellm import trace, trace_tool

@trace_tool(name="vector_search", max_retries=2)
def search_vectors(query: str, top_k: int = 10) -> list[float]:
    # If this raises, @trace_tool retries up to 2 times
    # with exponential backoff: 0.5s, 1.0s
    return vector_db.query(query, top_k)

@trace(prompt="rag_query", model_name="gpt-4o")
def rag_pipeline(question: str) -> str:
    # This call to search_vectors creates a step
    # automatically attached to the rag_pipeline trace
    results = search_vectors(question, top_k=5)
    return generate_answer(question, results)

Tip

@trace_tool detects sync vs async automatically, just like @trace. Use it with async functions for tracing concurrent tool calls.

Nesting Behavior

The nesting model is single-level for trace documents: steps from all child contexts are flattened into the parent's steps array. Each step contains its own metadata (tool name, duration, status, input, output) so the execution graph can be reconstructed during replay.

Child steps are appended to the parent in execution order
Each step has a unique step_id (UUID4)
Retry count is aggregated across all child executions
Because ContextVar is used, parallel execution via asyncio.gather or threading produces correct, non-interleaved step lists per trace

Example: Nested Agent Workflow

nested_workflow.pyCopy

python

from tracellm import trace, trace_tool

# ── Tool layer (instrumented with @trace_tool) ──────────────

@trace_tool(name="retrieve", max_retries=1)
def retrieve(query: str) -> list[dict]:
    docs = vector_db.similarity_search(query, k=5)
    return docs

@trace_tool(name="rerank")
def rerank(docs: list[dict], query: str) -> list[dict]:
    return sorted(docs, key=lambda d: d["score"], reverse=True)[:3]

@trace_tool(name="generate", max_retries=2)
def generate(context: str, query: str) -> str:
    return llm.complete(prompt=context, query=query)

# ── Orchestration layer (instrumented with @trace) ──────────

@trace(
    prompt="answer_question",
    model_name="gpt-4o",
    project="rag-service",
    environment="production",
)
def answer_question(query: str) -> dict:
    docs = retrieve(query)
    ranked = rerank(docs, query)
    context = build_context(ranked)
    answer = generate(context, query)
    return {"answer": answer, "sources": len(ranked)}

When answer_question runs, the resulting trace contains three steps (retrieve, rerank, generate). Each step records its own duration, input, output, and success status. The trace is persisted once with a singletrace_id.

Context Isolation

Because contextvars.ContextVar is used, each concurrent execution chain gets its own isolated context. This means parallel traces do not interfere with each other:

context_isolation.pyCopy

python

@trace(prompt="parallel_process")
async def process_all(items: list[str]) -> list[dict]:
    # Each call to process_item gets its own context
    # Steps are NOT interleaved between items
    tasks = [process_item(item) for item in items]
    return await asyncio.gather(*tasks)

@trace_tool(name="process_item")
async def process_item(item: str) -> dict:
    step1 = await do_something(item)
    step2 = await do_something_else(step1)
    return {"item": item, "result": step2}

Info

Step collection respects async context switches. If a child tool calls awaitand another coroutine runs during that await, steps from the other coroutine are correctly routed to their own parent's context.

Common Errors

Error	Cause	Fix
Steps not appearing in trace	@trace_tool used without parent @trace	Ensure a @trace-decorated function calls the @trace_tool function
Duplicate tool names in steps	Multiple @trace_tool functions with the same name	Set explicit name= on each @trace_tool to distinguish them
Retry not happening	max_retries not set or function is not raising	Set max_retries=N and ensure the function raises on failure