Chat

Stateful conversation sessions on top of a language model.

A Chat keeps Prompt history in a Ref and reuses it for text generation, streaming, and structured output. Each generation call combines the current history with the caller's new prompt, invokes the active language model, and appends the response parts back into history. Constructors create fresh sessions, seed sessions from prompts, restore exported history, or connect a chat to persistence.

14 exports Added in v4.0.0 Source

Constructors

empty

Added in v4.0.0 Source

Creates a new Chat service with empty conversation history.

When to use

Use when you need to start a fresh chat session without initial context or system prompts.

Signature

declare const empty: Effect.Effect<Chat>

Example

(Creating an empty chat)

import { Effect } from "effect"
import { Chat } from "effect/unstable/ai"

const freshChat = Effect.gen(function*() {
  const chat = yield* Chat.empty
  const history = yield* chat.export
  return (history as { content: ReadonlyArray<unknown> }).content.length
})

await Effect.runPromise(freshChat) // => 0

fromExport

Added in v4.0.0 Source

Creates a Chat service from previously exported chat data.

Details

Restores a chat session from structured data that was previously exported using the export method. Useful for persisting and restoring conversation state.

Signature

declare function fromExport(data: unknown): Effect<Chat, SchemaError>

Example

(Restoring chat data)

import { Effect, Ref } from "effect"
import { Chat } from "effect/unstable/ai"

const restoreChat = Effect.gen(function*() {
  const originalChat = yield* Chat.fromPrompt([
    {
      role: "user",
      content: "Which library are we using?"
    },
    {
      role: "assistant",
      content: "The project uses Effect."
    }
  ])

  const exported = yield* originalChat.export
  const restoredChat = yield* Chat.fromExport(exported)
  const restoredHistory = yield* Ref.get(restoredChat.history)

  const restoredResponse = restoredHistory.content[1]
  if (restoredResponse?.role === "assistant") {
    const restoredText = restoredResponse.content[0]
    if (restoredText?.type === "text") {
      return {
        roles: restoredHistory.content.map((message) => message.role),
        text: restoredText.text
      }
    }
  }
  return undefined
})

await Effect.runPromise(restoreChat) // => { roles: ["user", "assistant"], text: "The project uses Effect." }

fromJson

Added in v4.0.0 Source

Creates a Chat service from previously exported JSON chat data.

Details

Restores a chat session from JSON string that was previously exported using the exportJson method. This is the most convenient way to persist and restore chat sessions to/from storage systems.

Signature

declare function fromJson(data: string): Effect<Chat, SchemaError>

Example

(Restoring chat history from JSON)

import { Effect, Ref } from "effect"
import { Chat } from "effect/unstable/ai"

const restoreFromJson = Effect.gen(function*() {
  const original = yield* Chat.fromPrompt("Hello")
  const jsonData = yield* original.exportJson
  const restoredChat = yield* Chat.fromJson(jsonData)
  const history = yield* Ref.get(restoredChat.history)
  return history.content.length
})

await Effect.runPromise(restoreFromJson) // => 1

fromPrompt

Added in v4.0.0 Source

Creates a new Chat service from an initial prompt.

Details

This is the primary constructor for creating chat instances. It initializes a new conversation with the provided prompt as the starting context.

Signature

declare function fromPrompt(prompt: RawInput): Effect<Chat, never, never>

Example

(Creating a chat from a system prompt)

import { Effect } from "effect"
import { Chat } from "effect/unstable/ai"

const chatWithSystemPrompt = Effect.gen(function*() {
  const chat = yield* Chat.fromPrompt([{
    role: "system",
    content: "You are a helpful assistant specialized in mathematics."
  }])

  const history = yield* chat.export
  return (history as { content: ReadonlyArray<unknown> }).content.length
})

await Effect.runPromise(chatWithSystemPrompt) // => 1

Example

(Restoring chat history from a prompt)

import { Effect } from "effect"
import { Chat } from "effect/unstable/ai"

// Initialize with conversation history
const existingChat = Effect.gen(function*() {
  const chat = yield* Chat.fromPrompt([
    {
      role: "user",
      content: [{ type: "text", text: "What's the weather like?" }]
    },
    {
      role: "assistant",
      content: [{ type: "text", text: "I don't have access to weather data." }]
    },
    {
      role: "user",
      content: [{ type: "text", text: "Can you help me with coding?" }]
    }
  ])

  const history = yield* chat.export
  return (history as { content: ReadonlyArray<unknown> }).content.length
})

await Effect.runPromise(existingChat) // => 3

makePersisted

Added in v4.0.0 Source

Creates a new chat persistence service.

When to use

Use when you need programmatic persisted chat creation and retrieval backed by the current BackingPersistence.

Details

The provided store identifier will be used to indicate which "store" the backing persistence should load chats from.

See

layerPersisted for the Layer-based constructor

Signature

declare const makePersisted: (...args: [options: {
  readonly storeId: string;
}]) => Effect<Service, never, Scope | BackingPersistence>

Errors

ChatNotFoundError

Added in v4.0.0 Source

Represents an error that occurs when attempting to retrieve a persisted Chat that does not exist in the backing persistence store.

When to use

Use to represent a missing persisted conversation when lookup by id cannot find stored history.

Signature

declare class ChatNotFoundError extends {
  readonly _tag: "ChatNotFoundError";
  readonly chatId: string;
} & YieldableError<this> {
  constructor(...args: [props: {
    readonly _tag?: "ChatNotFoundError";
    readonly chatId: string;
  }, options?: MakeOptions]);
}

Layers

layerPersisted

Added in v4.0.0 Source

Creates a Layer for a new chat persistence service.

When to use

Use to provide Chat.Persistence from a configured BackingPersistence when your application needs persisted chat sessions backed by a named store.

Details

The provided store identifier will be used to indicate which "store" the backing persistence should load chats from.

See

makePersisted for the effect constructor when building the service directly instead of providing it as a layer

Signature

declare function layerPersisted(options: {
  readonly storeId: string;
}): Layer<Persistence, never, BackingPersistence>

Models

Chat interface

Added in v4.0.0 Source

Chat session with history, export, and generation operations.

See

Persisted for the persistence-backed extension

Signature

interface Chat {
  readonly "~effect/ai/Chat": "~effect/ai/Chat";
  readonly export: Effect<unknown, AiError>;
  readonly exportJson: Effect<string, AiError>;
  readonly generateObject: <ObjectEncoded extends Record<string, any>, ObjectSchema extends Encoder<ObjectEncoded, unknown>, Options extends NoExcessProperties<GenerateObjectOptions<any, ObjectSchema>, Options>>(options: Options & GenerateObjectOptions<ExtractTools<Options>, ObjectSchema>) => Effect<GenerateObjectResponse<ExtractTools<Options>, ObjectSchema["Type"], ExtractToolParametersMode<Options>>, ExtractError<Options>, LanguageModel | ExtractServices<Options> | ObjectSchema["DecodingServices"]>;
  readonly generateText: {
    <Options extends NoExcessProperties<GenerateTextOptions<{}>, Options>>(options: Options & {
      readonly toolkit?: undefined;
    } & GenerateTextOptions<{}>): Effect<GenerateTextResponse<{}, "decoded">, ExtractError<Options>, LanguageModel | ExtractServices<Options>>;
    <Tools extends Record<string, Any>, Options extends NoExcessProperties<GenerateTextOptions<Tools> & {
      readonly toolkit: ToolkitInput<Tools>;
    }, Options>>(options: Options & GenerateTextOptions<Tools> & {
      readonly toolkit: ToolkitInput<Tools>;
    }): Effect<GenerateTextResponse<Tools, ExtractToolParametersMode<Options>>, ExtractError<Options>, LanguageModel | ExtractServices<Options>>;
    <Options extends {
      readonly toolkit: WithHandler<any> | Effect<WithHandler<any>, never, any>;
    } & GenerateTextOptions<any> & Readonly<Record<Exclude<keyof Options, keyof GenerateTextOptions<any>>, never>>>(options: Options & GenerateTextOptions<ExtractTools<Options>> & {
      readonly toolkit: Options["toolkit"];
    }): Effect<GenerateTextResponse<ExtractTools<Options>, ExtractToolParametersMode<Options>>, ExtractError<Options>, LanguageModel | ExtractServices<Options>>;
  };
  readonly history: Ref<Prompt>;
  readonly streamText: {
    <Options extends NoExcessProperties<GenerateTextOptions<{}>, Options>>(options: Options & {
      readonly toolkit?: undefined;
    } & GenerateTextOptions<{}>): Stream<StreamPart<{}, "decoded">, ExtractError<Options>, LanguageModel | ExtractServices<Options>>;
    <Tools extends Record<string, Any>, Options extends NoExcessProperties<GenerateTextOptions<Tools> & {
      readonly toolkit: ToolkitInput<Tools>;
    }, Options>>(options: Options & GenerateTextOptions<Tools> & {
      readonly toolkit: ToolkitInput<Tools>;
    }): Stream<StreamPart<Tools, ExtractToolParametersMode<Options>>, ExtractError<Options>, LanguageModel | ExtractServices<Options>>;
    <Options extends {
      readonly toolkit: WithHandler<any> | Effect<WithHandler<any>, never, any>;
    } & GenerateTextOptions<any> & Readonly<Record<Exclude<keyof Options, keyof GenerateTextOptions<any>>, never>>>(options: Options & GenerateTextOptions<ExtractTools<Options>> & {
      readonly toolkit: Options["toolkit"];
    }): Stream<StreamPart<ExtractTools<Options>, ExtractToolParametersMode<Options>>, ExtractError<Options>, LanguageModel | ExtractServices<Options>>;
  };
}

Persisted interface

Added in v4.0.0 Source

Represents a Chat that is backed by persistence.

Details

When calling a text generation method (e.g. generateText), the previous chat history as well as the relevent response parts will be saved to the backing persistence store.

Signature

interface Persisted extends Chat {
  readonly id: string;
  readonly save: Effect<void, PersistenceError | AiError>;
}

Signature

declare const Chat: Context.Service<Chat, Chat>

Example

(Accessing the Chat service)

import { Effect, Layer, Stream } from "effect"
import { Chat, LanguageModel } from "effect/unstable/ai"

const FakeLanguageModel = Layer.effect(
  LanguageModel.LanguageModel,
  LanguageModel.make({
    generateText: () =>
      Effect.succeed([{
        type: "text",
        text: "Quantum computers use quantum states to process information."
      }]),
    streamText: () => Stream.empty
  })
)

const ChatLayer = Layer.effect(Chat.Chat, Chat.empty)

const program = Effect.gen(function*() {
  const chat = yield* Chat.Chat
  const response = yield* chat.generateText({
    prompt: "Explain quantum computing in simple terms"
  })
  return response.text
})

await Effect.runPromise(
  program.pipe(Effect.provide(Layer.merge(ChatLayer, FakeLanguageModel)))
) // => "Quantum computers use quantum states to process information."

Persistence

Added in v4.0.0 Source

Service tag for persistence-backed AI conversation storage.

When to use

Use to provide the storage operations needed by persisted conversation sessions.

Signature

declare class Persistence extends Shape<"effect/ai/Chat/Persisted", Service, this> {
  constructor(_: never);
}

Type IDs

TypeId

Added in v4.0.0 Source

Brand for Chat implementations.

Signature

declare const TypeId: "~effect/ai/Chat"

TypeId type

Added in v4.0.0 Source

Brand type for Chat.

Signature

type TypeId = "~effect/ai/Chat"