Build durable agents with Temporal and Lakebase

Build durable agents with Temporal and Lakebase | Databricks Blog Skip to main content

Summary

Preserve agent progress with Temporal: recover recorded work after worker failures, retry failed operations, and wait durably for human review.

Serve live application state with Lakebase Postgres: make evidence, recommendations, review decisions, and operational metrics queryable throughout each run.

Connect execution to governed data: read Unity Catalog policy through synced tables and, with Lakebase Change Data Feed enabled, publish operational changes back to Delta history tables.

A personal-loan underwriting agent gathers evidence, applies policy, and may wait days for a reviewer. During that time, workers can restart and tool calls can fail. The application must preserve completed work, resume execution, and keep the evidence available to the reviewer. This reference implementation uses Temporal for durable execution and Lakebase Postgres for queryable operational state. A synced table makes underwriting policy from Unity Catalog available in Lakebase. Temporal Activities write evidence, decisions, and metrics to Lakebase; once enabled, Lakebase Change Data Feed can publish those changes to Unity Catalog-managed Delta history tables. This combination is especially useful when Databricks already manages the agent’s inputs and downstream analysis. The Challenge with Long-Running Cloud Agents A cloud agent may outlive the request, worker, container, or deployment that started it. A user can begin a session, return tomorrow, and continue on another worker. Deployments and process failures are routine, so the agent’s progress must survive independently of the process executing it. Recovery requires both the results of completed operations and the control-flow state needed to determine what happens next. For this underwriting agent, this creates six requirements: Recovery: A replacement worker must resume from the last completed step. Retries: Tool calls and database operations must tolerate repeated execution without duplicating side effects. Long waits: The agent must wait for people or external systems without holding a worker open. Operational visibility: Applications and operators need the current status, evidence, retry state, and failure details. Runtime governance: Policy updates must become available without a code deployment, and the application must define when an open case adopts them. Audit: The system must retain the evidence, policy, recommendation, and human decision associated with each run.

A conversation transcript covers only part of this state. Recovery also requires the control-flow history: which operations were scheduled, which results were recorded, what the agent is waiting for, and which commands it has accepted. Temporal simplifies the management of distributed systems. When building with Temporal, a Workflow is the durable control flow for one agent run. An Activity is a call to a model, tool, or database whose result is recorded in the Workflow's Event History; Activities can be retried. A Signal is an asynchronous command sent to a running Workflow, such as an underwriter’s decision. The Temporal Lakebase AgentWorkflow reference implementation is a runnable personal-loan underwriting agent. It calls several tools, reads governed policy, produces a recommendation, and waits for an underwriter. Lakebase Postgres also helps developers manage these problems, but Temporal and Lakebase store different state for different consumers. Temporal’s Event History drives replay. Lakebase stores the application-facing view: current run status, messages, evidence, review state, and metrics. Unity Catalog remains the policy source; a synced table makes that policy queryable in Postgres, and Change Data Feed provides the return path for operational history. The systems do not share a transaction. Lakebase writes run as Temporal Activities under at-least-once execution. Deterministic identifiers, constraints, guarded updates, and Postgres upserts ensure that repeated Activity attempts target the same logical record. This architecture adds two managed systems and a projection contract between them. Together, they improve the agent’s resilience and scalability while keeping operational overhead low. Temporal plus Lakebase is most useful when an agentic session must survive worker replacement, accept input after long waits, expose relational state to an application, and apply governed data while it remains open. The Underwriting Use Case I chose loan underwriting because the same run must gather evidence, apply policy, produce a recommendation, and wait for a person. A worker can fail between any of those steps. Policy can change without an application deployment, and the UI needs the current evidence before the Workflow closes. Mocked applicants replace real credit bureaus and income providers, and the tool sequence is deterministic for simplicity. Each request contains a user ID, applicant ID, amount, purpose, model choice, and turn limit. FastAPI assigns run_id , starts LoanUnderwritingWorkflow , and uses that same ID across the API, Temporal execution, and Lakebase rows. On the first turn, credit_check returns score, trade lines, delinquencies, and current debt. income_verification returns income and employment evidence. debt_to_income_calc calculates debt-to-income ratio. policy_lookup loads the policy for the loan purpose and evaluates the evidence against approval, referral, and hard-decline thresholds. The sample’s borderline applicant has a 665 credit score, $76,000 in verified annual income, $2,400 in monthly debt, and one non-material delinquency flag. The policy result records every rule, threshold, actual value, pass/fail result, source, recommendation, and rationale. The model can recommend but cannot decide. An underwriter approves, denies, or requests more information. A request for more information becomes another user message and another agent turn. The case exercises a worker crash after completed tool calls, a committed Lakebase write whose Activity completion is lost, a review left open for days, a stale browser decision, and a policy change during execution. Architecture

Figure 1. The execution, operational-state, and governance paths in the reference implementation.

To implement the underwriting agent, React and FastAPI handle HTTP and UI work: starting runs, rendering evidence, listing cases, and submitting review decisions. Temporal Cloud...