Skip to main content

Migrating from 0.12 to 0.13

Summary​

0.13 rewrites the async-batch subsystem. The three hand-written vendor batch providers are gone, replaced by the AI SDK's own batch models plus OpenRouter's Batch API as a new transport — which removes @anthropic-ai/sdk, @google/genai, and openai from your install entirely. The user-facing API barely moves: ai.batch(), AIBatch, AIBatchRequest, and AIBatchResult keep their shapes, defineAsyncBatchStage is untouched, and the kernel's suspendedState.batchId contract is unchanged, so existing async-batch stages keep working. What does change is cost: batch pricing is now read per model from OpenRouter's catalog instead of assumed to be a flat 50% discount, cost is taken from the provider when it reports one, and a registry generated by 0.12 is wrong until you re-run workflow-engine-sync. Two long-standing bugs are also fixed here — the ./client entry could not be bundled at all by anyone who honored the optional peer deps, and batch cost was being reported at roughly a quarter of its true value.

Does this affect you?​

Two of the three known consumers never call ai.batch() at all. If that is you, most of this guide does not apply.

  • You never call ai.batch() — only two required actions apply: re-run workflow-engine-sync, and remove the three vendor SDK dependencies. Everything about AIBatchProvider, validated, batchRefs, getStatus metadata and the 1024-token cap is irrelevant to you. Recorded cost numbers will still change (provider-reported cost and long-context tiers), so re-baseline any alerting on WorkflowRun.totalCost.
  • You use defineAsyncBatchStage for something that is not an AI batch (a job dispatcher, a human approval gate) — same as above. The suspend/resume contract is unchanged.
  • You call ai.batch() — read every required action. In particular the schema-validation and getStatus metadata items describe behaviour you are relying on today that does not work the way the old types said.

Then npx workflow-engine-codemod --from 0.11 (or --from 0.12) applies the mechanical renames and lists every manual item with a file and line.

Required actions​

  • Re-run workflow-engine-sync. A registry generated by 0.12 has no batch prices, and its batch-capability flag was a slug-prefix heuristic with 39 false positives. The flag is now derived from two real signals: a :batch sibling in OpenRouter's catalog (which also supplies exact batch prices), or a google/anthropic/openai model that the native AI SDK provider can batch (at the vendor's documented 50%). Models only OpenRouter can batch need the sibling; a native-vendor model without one is reachable only through its vendor transport. The CLI no longer requires OPENROUTER_API_KEY — the catalog endpoint is public — though it still sends the key when present.

    npx workflow-engine-sync

  • Add two columns and a unique index to ai_calls. Batch cost rows are now deduplicated on (batchId, requestId) at the database, which closes a race where two workers polling the same suspended stage both recorded the batch. Both columns are nullable, so existing rows are untouched.

    model AICall {
      // ...existing fields...
      batchId   String?
      requestId String?

      @@unique([batchId, requestId], map: "ai_calls_batch_request_unique")
      @@index([batchId])
    }
    ALTER TABLE "ai_calls" ADD COLUMN "batchId" TEXT, ADD COLUMN "requestId" TEXT;
    CREATE INDEX "ai_calls_batchId_idx" ON "ai_calls"("batchId");
    CREATE UNIQUE INDEX "ai_calls_batch_request_unique" ON "ai_calls"("batchId", "requestId");

    Rows written by 0.12 (batch id only inside metadata) are still recognised by isRecorded(). If you implement your own AICallLogger, the new input fields are optional and your adapter keeps compiling — but you will not get the dedupe until you honour them.

  • Remove the old vendor SDK peer dependencies and add the new ones. Only install the AI SDK providers whose batch transports you actually use; @ai-sdk/google already ships as a direct dependency.

    # Before
    npm install @anthropic-ai/sdk @google/genai openai

    # After — optional, only what you batch against
    npm install @ai-sdk/anthropic @ai-sdk/openai

  • Replace any direct use of the batch provider classes. AnthropicBatchProvider, GoogleBatchProvider, OpenAIBatchProvider, their *Config and *Request types, BatchStatus, BatchRequestText, and BatchRequestWithSchema are no longer exported. Drive batches through ai.batch() instead, which is what nearly all callers already did.

    // Before
    import { GoogleBatchProvider } from "@bratsos/workflow-engine";
    const provider = new GoogleBatchProvider({}, logger);
    const handle = await provider.submit(requests);

    // After
    const batch = ai.batch("gemini-2.5-flash", "google");
    const handle = await batch.submit(requests);

  • Add a case if you exhaustively switch on AIBatchProvider. It gained "openrouter".

    // Before
    type AIBatchProvider = "google" | "anthropic" | "openai";

    // After
    type AIBatchProvider = "google" | "anthropic" | "openai" | "openrouter";

  • Pass an explicit provider for models that are not batch-capable. ai.batch(modelKey) now throws instead of silently falling back to "google", which used to surface much later as a confusing Google-credentials error. If you relied on the fallback, name the provider.

    // Before — silently became "google", then failed deep in the Google client
    const batch = ai.batch("some-model");

    // After — either the model resolves a provider, or you name one
    const batch = ai.batch("some-model", "openrouter");

  • Expect recorded costs to change, and re-baseline any alerting on WorkflowRun.totalCost. Cost now comes from the provider when it reports one, and batch cost uses real per-model prices. Batch numbers in particular will jump, because 0.12 applied its 50% discount up to three times to the same call.

  • Budget for higher batch output if you relied on the old cap. The Anthropic and OpenAI batch paths silently capped output at 1024 tokens while Google had no cap. That cap is gone; the default is now the model's own maxCompletionTokens. Set maxTokens per request if you want a tighter bound.

    // After — maxTokens, system, and temperature are now settable per request
    await batch.submit([
      { id: "1", prompt: "Summarize…", schema: SummarySchema, maxTokens: 2048 },
    ]);

  • Stop trusting AIBatchResult.result to have been schema-validated unless validated is true. Zod schemas are not JSON-serializable, so they never survived a suspend/resume and validation silently did not run in the production lifecycle — despite what the old types claimed. Re-supply schemas at retrieval time.

    Note that checkCompletion(state, ctx) has no ctx.input — carry the request ids forward in state.metadata (or ctx.storage) at submit time and rebuild the schema map from those.

    // Before — believed to be validated; in practice never was after a resume
    const results = await batch.getResults(state.batchId);

    // After — re-supply schemas, and check the flag
    const requestIds = (state.metadata?.requestIds ?? []) as string[];
    const results = await batch.getResults(state.batchId, {
      ...state.metadata,
      schemas: Object.fromEntries(requestIds.map((id) => [id, ItemSchema])),
    });
    for (const r of results) {
      if (r.status === "succeeded" && !r.validated) {
        throw new Error(`Result ${r.id} was not validated against its schema`);
      }
    }

New features​

  • OpenRouter as a batch transport. ai.batch(model, "openrouter") reaches models from nine authors that previously had no batch code path at all. The transport is text-only, runs on a 24h window, and returns no partial results if a batch expires — so submissions are capped at 500 requests each by default.
  • Injectable credentials and transport. ai.batch(modelKey, provider?, options?) takes BatchOptions { apiKey, baseURL, fetch, endpoint, maxRequestsPerBatch, maxPartitions }. Batch no longer reads process.env when constructed, which is what previously made it unusable on edge runtimes.
  • Fan-out across multiple upstream batches. A submit() partitions by response schema and by size, so requests with different schemas no longer collide (Google rejects a batch whose requests disagree on response_format). AIBatchHandle gained refs, batchIds, requestCounts, and error. Persist refs in suspendedState.metadata.batchRefs; handle.id still holds the first batch id.
  • Provider-reported cost. AITextResult, AIObjectResult, AIEmbedResult, and the stream's getUsage() gained reportedCostUsd and costSource: "reported" | "estimated". BYOK is handled correctly — upstream inference cost is added under BYOK and not under non-BYOK, where adding it would double-count.
  • Configurable OpenRouter routing. createAIHelper takes a fifth parameter, { routing: { priceHeadroom, sort, requireParameters } }. priceHeadroom defaults to 1.25 — provider.max_price used to be pinned to the exact registry price, so any upstream price drift made calls fail outright. Pass 0 to omit the price ceiling entirely. Child helpers inherit the options.
  • Cancellable batch operations. BatchOptions.abortSignal applies to every provider call the handle makes.
  • Typed partial-failure error. submit() throws BatchSubmitError when a later partition fails after earlier batches were created; err.createdRefs lets you poll and drain them rather than orphan them (the POST is never retried — there is no idempotency key).
  • Batch inspection outside a stage. resolveAiSdkBatchModel, fromAiSdk, createOpenRouterBatchModel and the EngineBatch* types are exported, so an admin route can query a batch through the engine.

Deprecations​

  • SuspendedStateSchema.apiKey — read by nothing, and it invites persisting a raw provider key into a Json column that round-trips on every poll. Use BatchOptions.apiKey. Removal at 1.0.

Bug fixes​

  • A run whose stages had all completed could wedge in RUNNING forever, then get marked FAILED. When run.transition lost the version-claim race to a writer that itself nooped or died before acting, nothing ever resolved the run, and run.reapStuck eventually failed it despite every stage having succeeded. claimRunTransition now retries the decision from fresh state (bounded at 3 attempts, recomputed from persistence each time), and run.reapStuck heals a stuck RUNNING run whose stages are all terminal by firing that transition rather than failing it. RunReapStuckResult gained a healed count. This landed upstream as #43 after 0.12.0 shipped, so 0.13 is its first published version — if you were carrying a local patch for it, drop the patch.
  • The ./client entry could not be bundled. It transitively pulled in all three optional vendor SDKs, so anyone who honored "optional" got three unresolvable imports. The client bundle is now ~8 KB instead of ~610 KB, and a check:bundle script fails the build if it regresses.
  • Batch cost was reported at ~25% of its true value. OpenRouter publishes :batch variants with already-halved prices; those rows were emitted as their own registry keys and then discounted twice more. :batch and :free rows are no longer emitted, and the discount is applied in exactly one place.
  • The flat 50% batch discount was wrong for 19 of 69 batch models — real multipliers run from 0.25x to 4.05x, and openai/gpt-oss-120b:batch costs four times more than its base model.
  • Long-context pricing tiers were ignored, under-reporting cost by 2x on models that double above a prompt-length threshold.
  • Per-tool observability records were only written if the caller passed onStepEnd. They are now always written when tools are used.
  • A cancelled batch polled until its deadline because "cancelled" had no mapping. Cancelled and expired batches now report "failed" with an explanatory error.
  • provider.max_price was pinned to the exact registry price, so any upstream price drift made calls fail outright instead of costing slightly more.
  • Failed OpenAI batch items used to vanish; the custom-id fallback that invented result-N ids — which also silently disabled schema lookup — is gone.

Code examples​

An async-batch stage, end to end​

// Before (0.12) — provider defaulted silently, schemas were assumed validated
async execute(ctx) {
  const ai = ctx.createAIHelper(`workflow.${ctx.workflowRunId}`);
  const batch = ai.batch("gemini-2.5-flash", "google");
  const handle = await batch.submit(
    ctx.input.items.map((item) => ({
      id: item.id,
      prompt: `Analyze: ${item.feedback}`,
      schema: AnalysisSchema,
    })),
  );
  return {
    suspended: true,
    state: { batchId: handle.id, provider: handle.provider },
    pollConfig: { pollInterval: 60_000, maxWaitTime: 7_200_000 },
  };
}

// After (0.13) — persist refs and request ids so a fan-out resumes correctly
async execute(ctx) {
  const ai = createAIHelper(`workflow.${ctx.workflowRunId}`, aiLogger);
  const batch = ai.batch("gemini-2.5-flash", "google");
  const requests = ctx.input.items.map((item) => ({
    id: item.id,
    prompt: `Analyze: ${item.feedback}`,
    schema: AnalysisSchema,
    maxTokens: 2048,
  }));
  const handle = await batch.submit(requests);
  return {
    suspended: true,
    state: {
      batchId: handle.id,
      // `metadata` is the only free-form slot on SuspendedStateSchema, and it
      // must stay JSON-serializable — EngineBatchRef is five scalars by design.
      metadata: {
        batchRefs: handle.refs,
        provider: handle.provider,
        requestIds: requests.map((r) => r.id),
        // Lets getResults() detect a short result set instead of recording it
        // as complete. requestIds.length works too; this is explicit.
        totalRequests: handle.totalRequests,
      },
    },
    pollConfig: { pollInterval: 60_000, maxWaitTime: 7_200_000 },
  };
}

Checking completion and validating results​

// Before (0.12)
async checkCompletion(state, ctx) {
  const ai = createAIHelper(`workflow.${ctx.workflowRunId}`, aiLogger);
  const batch = ai.batch(ctx.config.model, "google");
  const status = await batch.getStatus(state.batchId);
  if (status.status !== "completed") return { ready: false };
  const results = await batch.getResults(state.batchId);
  return { ready: true, output: results.map((r) => r.result) };
}

// After (0.13) — pass metadata through so a fan-out fans back in, and
// re-supply schemas so `validated` is actually true.
// `checkCompletion` receives only (state, ctx); ctx has config/storage/log,
// NOT the stage input — so read what you need from state.metadata.
async checkCompletion(state, ctx) {
  const ai = createAIHelper(`workflow.${ctx.workflowRunId}`, aiLogger);
  const provider = state.metadata?.provider as AIBatchProvider | undefined;
  const batch = ai.batch(ctx.config.model, provider);

  // Pass the metadata to getStatus too, not just getResults. Without it,
  // getStatus only polls the FIRST batch and will report "completed" while
  // later partitions are still running.
  const status = await batch.getStatus(state.batchId, state.metadata);
  if (status.status === "failed") {
    return { ready: false, error: status.error ?? "Batch failed" };
  }
  if (status.status !== "completed") return { ready: false, nextCheckIn: 60_000 };

  const requestIds = (state.metadata?.requestIds ?? []) as string[];
  const results = await batch.getResults(state.batchId, {
    ...state.metadata,
    schemas: Object.fromEntries(requestIds.map((id) => [id, AnalysisSchema])),
  });
  return { ready: true, output: results.filter((r) => r.status === "succeeded") };
}

Batching on a runtime without process.env​

// After (0.13) — inject the key and fetch rather than reading the environment
const batch = ai.batch("anthropic/claude-sonnet-4.5", "openrouter", {
  apiKey: env.OPENROUTER_API_KEY,
  fetch: env.customFetch,
  maxRequestsPerBatch: 250,
});

Reading cost​

// After (0.13) — prefer the provider's number, fall back to the estimate
const { text, cost, reportedCostUsd, costSource } = await ai.generateText(
  "gemini-2.5-flash",
  prompt,
);
if (costSource === "estimated") {
  // No provider-reported cost for this call; `cost` came from the price table.
}