--- title: Workflows description: "Build durable background workflows with typed payloads, checkpointed steps, retries, sleeps, signals, and scale-to-zero execution." --- ## Search Page index: https://tako.sh/llms.txt. Docs and blog posts also have .md URLs. This site has a search endpoint: https://context7.com/api/v2/ask?siteKey=ask_b8918f0414955bb91bfa1a89&query=. It returns documentation that answers the question, with a source link for each part. No API key is needed. If nothing matches, it says so. Tako workflows run background work next to your app without a separate queue service. A workflow can retry after failures, remember completed steps, sleep without holding a worker open, wait for an external signal, run on a cron schedule, and let its worker process scale to zero when idle. Code tabs cover JavaScript/TypeScript and Go where their workflow APIs correspond. JavaScript uses `tako.sh`; the Go worker SDK is the separate `github.com/tako-sh/tako/sdk/go` module, not the root Go HTTP SDK. The Rust SDK does not currently provide workflow authoring or worker APIs. Shared runtime semantics apply to both supported worker SDKs; option tables below describe the JavaScript API unless stated otherwise. ## Vocabulary | Term | Meaning | | -------- | ------------------------------------------------------------------------------- | | Workflow | A named handler discovered from a JS module or registered by a Go worker. | | Run | One execution of a workflow with its own payload, run ID, attempts, and status. | | Step | A named, checkpointed operation inside a run, created with `ctx.run`. | | Worker | A separate process that claims and executes workflow runs. | Run statuses are `pending`, `running`, `succeeded`, `cancelled`, and `dead`. A run may return to `pending` while waiting for a retry, sleep, or signal. ## Define A Workflow For JavaScript, create a module under `/workflows/` and default-export `defineWorkflow`. For Go, register the handler in the worker process before `RunWorker`: ```ts JavaScript // src/workflows/fulfill-order.ts import { defineWorkflow } from "tako.sh"; type FulfillOrderPayload = { orderId: string; }; export default defineWorkflow("fulfill-order", { retries: 4, handler: async (payload, ctx) => { const order = await ctx.run("load-order", () => db.orders.find(payload.orderId)); await ctx.run("charge", () => payments.charge({ orderId: order.id, amount: order.total, idempotencyKey: `${ctx.runId}:charge`, }), ); await ctx.run("send-receipt", () => mailer.send(order.email)); }, }); ``` ```go Go // cmd/worker/main.go package main import ( "context" "encoding/json" "log" workflow "github.com/tako-sh/tako/sdk/go" ) func main() { workflow.RegisterWorkflow("fulfill-order", func(ctx *workflow.WorkflowContext, raw json.RawMessage) error { var payload struct { OrderID string `json:"orderId"` } if err := json.Unmarshal(raw, &payload); err != nil { return ctx.Fail(err) } return ctx.Step.Run("fulfill", nil, func() (any, error) { // Your helper must use the stable key to deduplicate external effects. return nil, fulfillOrder(payload.OrderID, ctx.RunID+":fulfill") }) }, workflow.WithMaxAttempts(5)) if err := workflow.RunWorker(context.Background()); err != nil { log.Fatal(err) } } ``` The Go `fulfillOrder` helper is app-owned. Install the Go worker module with `go get github.com/tako-sh/tako/sdk/go`, build a worker binary, and configure its command in `[workflows].run` when needed. `WithMaxAttempts(5)` counts all attempts; JavaScript `retries: 4` counts retries after the first attempt. The payload type flows into `.enqueue(payload)`, so callers get normal TypeScript checking without a generated workflow registry. The workflow name must be unique within the app. Using the filename stem as the name keeps discovery and logs easy to follow. Useful definition options include: | Option | Meaning | | ---------- | ---------------------------------------------------------------------------------- | | `retries` | Retries after the first run attempt. The default is `2`, for three total attempts. | | `backoff` | Run-level retry timing with optional `base` and `max` values in milliseconds. | | `worker` | JavaScript worker group; omitted means the default lane. | | `schedule` | A five-field cron expression such as `"0 9 * * 1-5"`. | | `local` | Keep the workflow on per-server local storage in a multi-server environment. | ## Enqueue A Run Import the workflow handle from server-side code and enqueue a JSON-serializable payload: ```ts JavaScript import fulfillOrder from "./workflows/fulfill-order"; const runId = await fulfillOrder.enqueue({ orderId: "ord_123" }); ``` ```go Go import ( "context" workflow "github.com/tako-sh/tako/sdk/go" ) func enqueueOrder(ctx context.Context, orderID string) (string, error) { result, err := workflow.Enqueue(ctx, "fulfill-order", map[string]string{"orderId": orderID}, workflow.EnqueueOpts{}) if err != nil { return "", err } return result.ID, nil } ``` You can enqueue from a request handler, another workflow, or a script running with Tako's server runtime. Go producers also need the Tako internal socket, and container HTTP processes do not receive it in v0; see [Container Workflows](/docs/docker-workflows/). Browser code cannot enqueue directly and throws `TakoError("TAKO_UNAVAILABLE")` if it reaches `.enqueue()`. Set options for a delayed run with a stable ID and seven total attempts: ```ts JavaScript await fulfillOrder.enqueue( { orderId: "ord_123" }, { runAt: new Date(Date.now() + 60_000), retries: 6, id: "fulfill:ord_123", }, ); ``` ```go Go import ( "context" "time" workflow "github.com/tako-sh/tako/sdk/go" ) func enqueueDelayedOrder(ctx context.Context) (*workflow.EnqueueResult, error) { runAt := time.Now().Add(time.Minute) id := "fulfill:ord_123" maxAttempts := uint32(7) return workflow.Enqueue(ctx, "fulfill-order", map[string]string{"orderId": "ord_123"}, workflow.EnqueueOpts{RunAt: &runAt, ID: &id, MaxAttempts: &maxAttempts}) } ``` | Option | Meaning | | --------- | ------------------------------------------------------------------------------------------------------- | | `runAt` | Do not make the run eligible before this time. The default is now. | | `retries` | Override the workflow's retry count for this run. | | `id` | Use this as the run ID. A retained run with the same ID, workflow, and payload is reused in any status. | `.enqueue()` returns the run ID, including the exact `id` you supply. Omit `id` to generate one and create a new run each time. IDs must contain 1–255 bytes and no NUL characters. Go's `EnqueueOpts` uses `ID`, `RunAt`, and `MaxAttempts` pointers. `MaxAttempts` counts the first attempt as well as retries. `Enqueue` returns an `EnqueueResult` with `ID` and `Deduplicated`. IDs are unique within the app and environment, across workflow names. Repeating an ID with a different workflow or payload produces a conflict. Concurrent enqueues with the same ID create one run. Existing runs keep their original schedule, retry budget, status, and completed steps; enqueue does not restart a succeeded, cancelled, or dead run. Deleting a finished run's history also frees its ID. A later enqueue with that ID creates a new run with no saved steps. Deduplication does not prevent a step's external side effect from repeating after a crash; pass a stable idempotency key to the external service as well. ## Durable Steps With `ctx.run` `ctx.run(name, fn)` saves the function's result. When the workflow is claimed again, a completed step with the same name returns its saved result instead of calling `fn` again. Keep step names stable for the lifetime of a workflow run, and keep returned values JSON serializable. The durability boundary is the saved result, not the side effect itself. If a worker stops after `fn` completes but before Tako saves the result, that step runs again on the next claim. Step bodies therefore have an at-least-once execution contract. Use provider idempotency keys, upserts, conditional writes, or another domain-specific guard for side effects. Steps can retry independently before an error reaches the run-level retry policy: ```ts JavaScript await ctx.run("call-provider", callProvider, { retries: 2, backoff: { base: 500, max: 10_000 }, }); ``` ```go Go import ( "time" workflow "github.com/tako-sh/tako/sdk/go" ) func callProviderStep(ctx *workflow.WorkflowContext, callProvider func() (any, error)) error { opts := workflow.StepRunOpts{Retries: 2} opts.Backoff.Base = 500 * time.Millisecond opts.Backoff.Max = 10 * time.Second return ctx.Step.Run("call-provider", nil, callProvider, opts) } ``` Step retries default to `0`. Set JavaScript's `retry: false` or Go's `StepRunOpts{NoRetry: true}` when an error from that step is permanent and should move the run directly to `dead`. Go's `out` argument (`nil` above) can be a pointer to receive the checkpointed result. The workflow context also provides `runId`, `workflowName`, `attempt`, and a workflow-scoped `logger`. Each step callback receives the same run metadata plus `stepName` and a step-scoped logger. ## Durable Sleep Use a stable name and a duration in milliseconds: ```ts JavaScript await ctx.sleep("wait-before-reminder", 24 * 60 * 60 * 1000); ``` ```go Go // Inside a registered Go workflow; import "time". if err := ctx.Step.Sleep("wait-before-reminder", 24*time.Hour); err != nil { return err } ``` Sleeps shorter than 30 seconds run inline. Longer sleeps park the run until its wake time so the worker can process other work or exit. Sleeping does not consume the run's retry budget, and the wake time survives worker and server restarts. ## Wait For A Signal `ctx.waitFor` (JavaScript) or `ctx.Step.WaitFor` (Go) parks a run until matching server-side code sends a signal. With a timeout, a missing signal becomes `null` in JavaScript or a nil pointer result in Go: ```ts JavaScript const decision = await ctx.waitFor<{ approved: boolean; by: string }>( `approval:order-${payload.orderId}`, { timeout: 7 * 24 * 60 * 60 * 1000 }, ); if (decision === null) { ctx.bail("approval timed out"); } ``` ```go Go import ( "time" workflow "github.com/tako-sh/tako/sdk/go" ) type ApprovalDecision struct { Approved bool `json:"approved"` By string `json:"by"` } func waitForApproval(ctx *workflow.WorkflowContext, orderID string) error { var decision *ApprovalDecision if err := ctx.Step.WaitFor("approval:order-"+orderID, &decision, 7*24*time.Hour); err != nil { return err } if decision == nil { return ctx.Bail("approval timed out") } if !decision.Approved { return ctx.Bail("approval rejected") } return nil } ``` Call the Go helper from the registered handler and return its error so the worker can park or cancel the run. Its event name is also the durable step name; keep it stable across retries. Send the signal from a request handler, webhook, script, or another workflow: ```ts JavaScript import { signal } from "tako.sh"; const woken = await signal(`approval:order-${orderId}`, { approved: true, by: userId, }); ``` ```go Go import ( "context" workflow "github.com/tako-sh/tako/sdk/go" ) func approveOrder(ctx context.Context, orderID, userID string) (uint64, error) { return workflow.Signal(ctx, "approval:order-"+orderID, map[string]any{"approved": true, "by": userID}) } ``` `signal` wakes every run waiting on that event name and returns the number of runs woken. Like enqueue, it is server-only and throws `TakoError("TAKO_UNAVAILABLE")` without an installed Tako runtime. Waiting and resuming do not consume retry attempts. Completed steps before the wait remain checkpointed. ## Finish, Bail, Fail, Or Retry | Handler outcome | Run result | | --------------------------------- | ------------------------------------------------------ | | Return normally | `succeeded`. | | Throw a regular error | Retry while attempts remain, then become `dead`. | | `ctx.bail(reason?)` | `cancelled` immediately, with no retry. | | `ctx.fail(error)` | `dead` immediately, with no retry. | | Long `ctx.sleep` or `ctx.waitFor` | Return to `pending` without spending the retry budget. | Run retries use exponential backoff with 20% jitter. The default delay starts at one second and is capped at one hour. `retries: 4` means four retries after the first attempt, for five total attempts. Go handlers return `nil` for success, an ordinary error for retry, `ctx.Bail(reason)` for cancellation, or `ctx.Fail(err)` for permanent failure. Return errors from `ctx.Step` operations so the worker can park or retry the run. Use `ctx.bail` when the work is no longer needed, such as a rejected or expired approval. Use `ctx.fail` for a permanent error that another attempt cannot fix. To finish successfully before the bottom of a handler, return normally. Workflow and step lifecycle logs appear in the normal Tako log stream. Use `ctx.logger` and the logger passed to `ctx.run` for messages that carry the run and step context. ## Scale To Zero Tako keeps no workflow worker running while the queue has no runnable work. Enqueue, cron, a signal, a due retry or sleep, and reclaimed work can wake a worker. The worker exits after its idle window and the next runnable run starts a fresh process. Production starts one scale-to-zero lane per JavaScript worker group; Go and container workers use a single lane. Workflows with `worker: "email"` run in a process that only claims that group. The config parser also accepts worker counts, concurrency, and per-server overrides; those values are not applied yet. JavaScript and Go workers honor the runtime-provided lane concurrency (currently 500). Go workers stop claiming on cancellation and wait for active handlers to finish, continuing their lease heartbeats while draining. Concurrency is a worker setting; `defineWorkflow` does not expose per-workflow concurrency or handler timeouts. The workflow process runs separately from HTTP instances. Heavy workflow dependencies do not have to occupy every request-serving process, while the worker still receives the app's runtime variables, secrets, storage bindings, and structured logs. If a worker exits with an error before it can claim any work, Tako stops the immediate respawn loop and makes the next enqueue fail with the startup error. Fixing the worker lets the next successful claim restore normal operation. ## Storage And Multiple Servers On one server, Tako stores durable workflow state locally. Runs belong to the deployed app and environment, not to one worker process or release, so a worker restart or rolling deploy does not discard progress. Local workflow transactions synchronize the SQLite write-ahead log before acknowledging changes, including enqueues and saved steps. Acknowledged state survives a process or host crash when the filesystem and storage device honor synchronization requests. This does not protect against loss of the disk or server; configure [backups](/docs/backups/) for recovery. External step side effects still have the at-least-once contract described above. Each claim receives a distinct ownership identifier. After a lease is reclaimed, a delayed earlier attempt cannot heartbeat, save a step, or change the new attempt’s run state, even when the same process claims the run again. This protects workflow state; external side effects still need application-level idempotency. Tako keeps finished runs for at least 184 days by default, covering six calendar months, then removes them and their saved steps in small batches. The age is measured from completion, so long sleeps and waits do not shorten a run's history. Cleanup is best effort while the workflow runtime is active; stopped or retired stores can keep older rows until the runtime starts again. To change the age or keep history indefinitely, set `retention` in `tako.toml`: ```toml [workflows] retention = "1d" # or "forever" ``` The setting applies to `succeeded`, `cancelled`, and `dead` runs. It does not delete `pending` or `running` runs. SQLite and Postgres can reuse freed database space even when the database file does not immediately shrink. An environment deployed to multiple servers needs one of two explicit storage models: | Project shape | Required setup | | ------------------------------------------------ | ---------------------------------------------------------------------------- | | All workflows are global | Set the environment's `postgres_url` credential for shared workflow state. | | Every JavaScript workflow is intentionally local | Set `local: true` on every workflow to keep separate state on each server. | | Local and global workflows are mixed | Set `postgres_url`; the environment needs shared workflow state. | | Go workflow worker on multiple servers | Set `postgres_url`; Go workflows do not have the source-level local opt-out. | Configure shared storage with: ```bash tako credentials set postgres_url --env production ``` `postgres_url` is a provider credential. Tako encrypts it and does not expose it to app code. The SDK API remains the same whether Tako selects local storage or Postgres. With `local: true`, each server owns its own queue and cron schedule. A scheduled workflow runs once per server, and run IDs do not deduplicate across servers. Use this only for work that is intentionally server-local, such as cleaning local files or warming a regional cache. Tako validates the multi-server storage choice before build and deploy work begins. See [Deployment](/docs/deployment/) for environment setup and the [CLI reference](/docs/cli/) for provider credential commands. ## Development And Deployment `tako dev` discovers JavaScript and TypeScript workflows under `/workflows/`. The dev daemon owns the durable state and starts a separate scale-to-zero worker when work becomes runnable. Worker output appears in the same log stream as the app. Workflow definition changes refresh the runtime, and a fresh worker picks up the new code on subsequent work. Cron schedules are synchronized from the current workflow definitions when a worker starts. Removing a schedule removes its registered cron entry on the next worker startup. `tako deploy` validates the storage model, packages workflow definitions with the app, and installs the workflow runtime for the release. A new release replaces the previous worker runtime after in-flight work drains. Removing the workflows directory in a later release retires the old workflow runtime. Durable runs remain scoped to the app and environment across releases. Use finite `tako logs --env production --days 1 --json` to inspect app and worker output with an agent; an attached terminal can follow it with `--tail`. See [Operations](/docs/operations/) for log inspection, [Development](/docs/development/) for local runtime details, and [Deployment](/docs/deployment/) for release behavior. ## Practical Rules - Keep workflow payloads and step results JSON serializable. - Give every step a stable name. Changing a name creates a different checkpoint for existing runs. - Make side-effecting step bodies idempotent. `ctx.run` is durable but still at least once at the save boundary. - Supply `id` to deduplicate enqueues for the same business operation while its run history is retained. - Use `ctx.bail` for expected cancellation and `ctx.fail` for permanent failure. - Keep `signal` and `.enqueue()` in server-side code. - Use Postgres for global workflows across multiple servers. Use `local: true` only when once per server is intentional. ## Verify Enqueue a known payload, inspect the run/step logs, and check its application-side result. Force a retry in a test environment and confirm completed steps reuse saved results and external effects deduplicate. Test duplicate enqueue IDs, a signal or durable wait when used, and the intended shared-versus-per-server execution model.