Docs
JSON Block

JSON Block

Allow LLMs to reply with JSON, which can be rendered as custom components in your application.

Buttons:
【{
   type: "buttons",
   buttons:[
     {text:"Star ⭐"},
     {text:"Confetti πŸŽ‰"}
   ]
}】

5x

Installation

pnpm add @llm-ui/json

Quick start

View on GitHub

Install dependencies

pnpm add @llm-ui/json @llm-ui/react @llm-ui/markdown react-markdown remark-gfm html-react-parser zod

Step 1: Create a markdown component

Create a component to render markdown using react-markdown.

import ReactMarkdown from "react-markdown";
import remarkGfm from "remark-gfm";
import { type LLMOutputComponent } from "@llm-ui/react";


// Customize this component with your own styling
const MarkdownComponent: LLMOutputComponent = ({ blockMatch }) => {
  const markdown = blockMatch.output;
  return <ReactMarkdown remarkPlugins={[remarkGfm]}>{markdown}</ReactMarkdown>;
};

Read more in the markdown block docs

Step 2: Setup your custom block’s JSON schema

Use zod to create a schema for your custom block.

We’ll set up a β€˜buttons’ block:

// buttonsSchema.ts
import z from "zod";

const buttonsSchema = z.object({
  type: z.literal("buttons"),
  buttons: z.array(z.object({ text: z.string() })),
});

Example JSON for your block:

{
  "type": "buttons",
  "buttons": [{ "text": "Button 1" }, { "text": "Button 2" }]
}

Step 3: Create a custom block component

import { parseJson5 } from "@llm-ui/json";
import { type LLMOutputComponent } from "@llm-ui/react";

// Customize this component with your own styling
const ButtonsComponent: LLMOutputComponent = ({ blockMatch }) => {
  if (!blockMatch.isVisible) {
    return null;
  }
  // use buttonsSchema from step 2
  const { data: buttons, error } = buttonsSchema.safeParse(
    parseJson5(blockMatch.output),
  );

  if (error) {
    return <div>{error.toString()}</div>;
  }
  return (
    <div>
      {buttons.buttons.map((button, index) => (
        <button key={index}>{button.text}</button>
      ))}
    </div>
  );
};

Step 4: Render custom blocks with llm-ui

Now we’ve created our components, we’re ready to use useLLMOutput to render language model output which contains markdown and buttons components.

import { markdownLookBack } from "@llm-ui/markdown";
import { useLLMOutput, useStreamExample } from "@llm-ui/react";
import { jsonBlock } from "@llm-ui/json";

const example = `Buttons:

【{type:"buttons",buttons:[{text:"Star ⭐"}, {text:"Confetti πŸŽ‰"}]}】
`;

const Example = () => {
  const { isStreamFinished, output } = useStreamExample(example);


  const { blockMatches } = useLLMOutput({
    llmOutput: output,
    blocks: [
      {
        ...jsonBlock({type: "buttons"}),
        component: ButtonsComponent, // from step 3
      },
    ],
    fallbackBlock: {
      lookBack: markdownLookBack(),
      component: MarkdownComponent, // from step 1
    },
    isStreamFinished,
  });

  return (
    <div>
      {blockMatches.map((blockMatch, index) => {
        const Component = blockMatch.block.component;
        return <Component key={index} blockMatch={blockMatch} />;
      })}
    </div>
  );
}

Step 5: Prompt LLM with your custom block

Generate the prompt for your JSON block:

import { jsonBlockPrompt } from "@llm-ui/json";

const prompt = jsonBlockPrompt({
  name: "Button",
  schema: buttonsSchema,
  examples: [
    { type: "buttons", buttons: [{ text: "Button 1" }, { text: "Button 2" }] },
  ],
  options: {
    type: "buttons",
  },
});

Generates:

You can respond with a Button component by wrapping JSON in 【】.
The JSON schema is:
{"type":"object","properties":{"type":{"type":"string","const":"buttons"},"buttons":{"type":"array","items":{"type":"object","properties":{"text":{"type":"string"}},"required":["text"],"additionalProperties":false}}},"required":["type","buttons"]}

Examples: 
【{"type":"buttons","buttons":[{"text":"Button 1"},{"text":"Button 2"}]}】

You can also hardcode the prompt into your

Options

{
  // Required
  type: "buttons",
  // Optional, defaults:
  startChar: "【",
  endChar: "】",
  typeKey: "type", // the key in the JSON object which determines the block type e.g. {"type": "buttons"}
  defaultVisible: false, // See below
  visibleKeyPaths: [], // See below
  invisibleKeyPaths: [], // See below
}

defaultVisible

defaultVisible: false

Generate no β€˜visibleText’ until the whole block is parsed.

When a partial block is parsed:

【{ type:"buttons", buttons: [{text:"Button 1"}, {text:"But `

blockMatch.visibleText;
// => ""

blockMatch.isVisible;
// => false

// llm-ui completes the partial JSON
blockMatch.output;
// => "{ type:"buttons", buttons: [{text:"Button 1"}, {text:"But"}] }"

When the whole block is parsed:

【{ type:"buttons", buttons: [{text:"Button 1"}, {text:"Button 2"}] }】

blockMatch.visibleText;
// => " "

blockMatch.isVisible;
// => true

blockMatch.output;
// => "{ type:"buttons", buttons: [{text:"Button 1"}, {text:"Button 2"}] }"

defaultVisible: true

Generates β€˜visibleText’ as the reponse is parsed:

【{ type:"buttons", buttons: [{text:"Button 1"}, {text:"But

blockMatch.visibleText;
// => "B"
// then
// => "Bu"
// then
// => "But"

blockMatch.isVisible;
// => true

blockMatch.output;
// => "{ type:"buttons", buttons: [{text:"B"}] }"
// then
// => "{ type:"buttons", buttons: [{text:"Bu"}] }"
// then
// => "{ type:"buttons", buttons: [{text:"But"}] }"

visibleKeyPaths and invisibleKeyPaths

You can use visibleKeyPaths and invisibleKeyPaths to determine which fields in the JSON object are visible or invisible.

The path syntax is jsonpath.

{
  defaultVisible: false,
  visibleKeyPaths: ["$.buttons[*].text"],
}

{
  defaultVisible: true, // typeKey e.g. "type" is always invisible
  invisibleKeyPaths: ["$.buttons[*].color"],
}

Prompts

jsonBlockPrompt

Returns a full prompt to send to the LLM.

import { jsonBlockPrompt } from "@llm-ui/json";
import z from "zod";

jsonBlockPrompt({
  name: "Button",
  schema: z.object({
    type: z.literal("buttons"),
    buttons: z.array(z.object({ text: z.string() })),
  }),
  examples: [
    { type: "buttons", buttons: [{ text: "Button 1" }, { text: "Button 2" }] },
  ],
  options: {
    type: "buttons",
    startChar: "【",
    endChar: "】",
    typeKey: "type",
    defaultVisible: false,
  },
});
// =>
"
You can respond with a Button component by wrapping JSON in 【】.
The JSON schema is:
{"type":"object","properties":{"type":{"type":"string","const":"buttons"},"buttons":{"type":"array","items":{"type":"object","properties":{"text":{"type":"string"}},"required":["text"],"additionalProperties":false}}},"required":["type","buttons"]}

Examples:
【{"type":"buttons","buttons":[{"text":"Button 1"},{"text":"Button 2"}]}】
"

jsonBlockSchema

Returns the schema as a JSON schema string.

import { jsonBlockSchema } from "@llm-ui/json";
import z from "zod";

jsonBlockSchema(z.object({
  type: z.literal("buttons"),
  buttons: z.array(z.object({ text: z.string() })),
}));
// =>
{"type":"object","properties":{"type":{"type":"string","const":"buttons"},"buttons":{"type":"array","items":{"type":"object","properties":{"text":{"type":"string"}},"required":["text"],"additionalProperties":false}}},"required":["type","buttons"]}

jsonBlockExample

Generates a single JSON block usage example.

import { jsonBlockExample } from "@llm-ui/json";
import z from "zod";

jsonBlockExample({
  schema: z.object({
    type: z.literal("buttons"),
    buttons: z.array(z.object({ text: z.string() })),
  }),
  example: { type: "buttons", buttons: [{ text: "Button 1" }, { text: "Button 2" }] },
  options: {
    type: "buttons",
    startChar: "【",
    endChar: "】",
    typeKey: "type",
    defaultVisible: false,
  },
});
// =>
【{"type":"buttons","buttons":[{"text":"Button 1"},{"text":"Button 2"}]}】

Json block functions

jsonBlock

Returns a JSON block object to be used by useLLMOutput.

import { jsonBlock } from "@llm-ui/json";

const options = {
  type: "buttons",
  startChar: "【",
  endChar: "】",
  typeKey: "type",
  defaultVisible: false,
};

jsonBlock(options);
// =>
{
  findCompleteMatch: findCompleteJsonBlock(options),
  findPartialMatch: findPartialJsonBlock(options),
  lookBack: jsonBlockLookBack(options),
  component: () => <div>Json block</div>,
}

Accepts options parameter.

findCompleteJsonBlock

Finds a complete JSON block in a string:

findCompleteJsonBlock({ type: "buttons" });

Will match:

【{"type":"buttons","buttons":[{"text":"Button 1"},{"text":"Button 2"}]}】

Accepts options parameter.

findPartialJsonBlock

Find a partial JSON block in a string.

findPartialJsonBlock({ type: "buttons" });

Will match:

【{"type":"buttons","buttons":[{"text":"Button 1"},{"text":

Accepts options parameter.

jsonBlockLookBack

Look back function for the JSON block.

jsonBlockLookBack({ type: "buttons" });

Accepts options parameter.

Parse

parseJson5

Parse a JSON5 output string.

import { parseJson5 } from "@llm-ui/json";

parseJson5(
  '{type:"buttons",buttons:[{"text":"Button 1"},{"text":"Button 2"}]}',
);
// =>
{
  type: "buttons",
  buttons: [{ text: "Button 1" }, { text: "Button 2" }],
}

Optimizing for performance

When using the JSON block you’ll want to consider the performance of your application.

JSON has a high overhead of non-content characters, this may result in a noticable pause as llm-ui waits for the full block.

You may wish to use skeleton or other loading states to indicate to the user that the block is being parsed.

In production we also recommend using a JSON format which is as concise as possible to reduce the overhead of non-content characters.

For example:

【{"type":"buttons","buttons":[{"text":"Button 1"},{"text":...

Becomes:

【{"t":"b","bs":[{"t":"Button 1"},{"t":...

You could also consider using the CSV block for a lower overhead format.

Introduction CSV block