Use Kimi API's JSON Mode

In some scenarios, we want the model to output content in a fixed format JSON document. For example, when you want to summarize an article, you might expect a structured data format like this:

{
	"title": "Article Title",
	"author": "Article Author",
	"publish_time": "Publication Time",
	"summary": "Article Summary"
}

If you directly tell the Kimi large language model in the prompt: "Please output content in JSON format," the model can understand your request and generate a JSON document as required. However, the generated content often has some flaws. For instance, in addition to the JSON document, Kimi might output extra text to explain the JSON document:

Here is the JSON document you requested

{
	"title": "Article Title",
	"author": "Article Author",
	"publish_time": "Publication Time",
	"summary": "Article Summary"
}

Or the JSON document format might be incorrect and cannot be parsed properly, such as (note the comma at the end of the summary field):

{
	"title": "Article Title",
	"author": "Article Author",
	"publish_time": "Publication Time",
	"summary": "Article Summary",
}

Such a JSON document cannot be parsed correctly. To generate a standard and valid JSON document as expected, we provide the response_format parameter. The default value of response_format is {"type": "text"}, which means ordinary text content with no formatting constraints. You can set response_format to {"type": "json_object"} to enable JSON Mode, and the Kimi large language model will output a valid, parsable JSON document as required.

When using JSON Mode, please follow these guidelines:

Inform the Kimi large language model in the system prompt or user prompt about the JSON document to be generated, including specific field names and types. It's best to provide an example for the model to refer to.
The Kimi large language model will only generate JSON Object type JSON documents. Do not prompt the model to generate JSON Array or other types of JSON documents.
If you do not correctly inform the Kimi large language model of the required JSON Object format, the model will generate unexpected results.

JSON Mode Application Example

Let's use a specific example to illustrate the application of JSON Mode:

Imagine we are building a WeChat intelligent robot customer service (referred to as intelligent customer service). The intelligent customer service uses the Kimi large language model to answer customer questions. We want the intelligent customer service to not only reply with text messages but also with images, link cards, voice messages, and other types of messages. Moreover, in a single response, we want to mix different types of messages. For example, for customer product inquiries, we provide a text reply, a product image, and finally, a purchase link (in the form of a link card).

Let's demonstrate the content of this example with code:

import json
 
from openai import OpenAI
 
client = OpenAI(
    api_key="MOONSHOT_API_KEY", # Replace MOONSHOT_API_KEY with the API Key you obtained from the Kimi Open Platform
    base_url="https://api.moonshot.ai/v1",
)
 
system_prompt = """
You are the intelligent customer service of Moonshot AI (Kimi), responsible for answering various user questions. Please refer to the document content to reply to user questions. Your reply can be text, images, links, and you can include text, images, and links in a single response.
 
Please output your reply in the following JSON format:
 
{
    "text": "Text information",
    "image": "Image URL",
    "url": "Link URL"
}
 
Note: Please place the text information in the `text` field, put the image in the `image` field in the form of a link starting with `oss://`, and place the regular link in the `url` field.
"""
 
completion = client.chat.completions.create(
    model="kimi-k2.5",
    messages=[
        {"role": "system",
         "content": "You are Kimi, an artificial intelligence assistant provided by Moonshot AI, excelling in Chinese and English conversations. You provide users with safe, helpful, and accurate answers. You will reject any questions involving terrorism, racism, pornography, and violence. Moonshot AI is a proper noun and should not be translated into other languages."},
        {"role": "system", "content": system_prompt}, # <-- Submit the system prompt with the output format to Kimi
        {"role": "user", "content": "Hello, my name is Li Lei, what is 1+1?"}
    ],
    response_format={"type": "json_object"}, # <-- Use the response_format parameter to specify the output format as json_object
)
 
# Since we have set JSON Mode, the message.content returned by the Kimi large language model is a serialized JSON Object string.
# We use json.loads to parse its content and deserialize it into a Python dictionary.
content = json.loads(completion.choices[0].message.content)
 
# Parse text content
if "text" in content:
	# For demonstration purposes, we print the content;
	# In real business logic, you may need to call the text message sending interface to send the generated text to the user.
    print("text:", content["text"])
 
# Parse image content
if "image" in content:
	# For demonstration purposes, we print the content;
	# In real business logic, you may need to first parse the image URL, download the image, and then call the image message sending
	# interface to send the image to the user.
    print("image:", content["image"])
 
# Parse link
if "url" in content:
	# For demonstration purposes, we print the content;
	# In real business logic, you may need to call the link card sending interface to send the link to the user in the form of a card.
    print("url:", content["url"])

Let's go over the steps for using JSON Mode once again:

Define the output JSON format in the system or user prompt. Our recommended best practice is to provide a specific output example and explain the meaning of each field;
Use the response_format parameter and set it to {"type": "json_object"};
Parse the content in the message returned by the Kimi large language model. message.content will be a valid JSON Object serialized as a string;

Incomplete JSON

If you encounter this situation:

You have correctly set the response_format parameter and specified the format of the JSON document in the prompt, but the JSON document you receive is incomplete or truncated, making it impossible to correctly parse the JSON document.

We suggest you check if the finish_reason field in the return value is length. Generally, a smaller max_tokens value will cause the model's output to be truncated, and this rule also applies when using JSON Mode. We recommend that you set a reasonable max_tokens value based on the estimated size of the output JSON document, so that you can correctly parse the JSON document returned by the Kimi large language model.

For a more detailed explanation of the issue of incomplete or truncated output from the Kimi large language model, please refer to: Common Issues and Solutions

Use the Web Search Tool Use Partial Mode with Kimi API