JSON Output

Enforcing structured JSON schema output is important for handling LLM outputs downstream with other systems and APIs in your applications.

For an in-depth technical deep dive of how we implemented this feature, see our blog post.

Python LibraryREST API

You can enforce JSON schema via the Lamini class is the base class for all runners. Lamini wraps our REST API endpoint.

First, return a string:

from lamini import Lamini

llm = Lamini(model_name="meta-llama/Llama-2-7b-chat-hf")
output = llm.generate(
    "How are you?",
    output_type={"my_response": "string"}
)

First, get a basic string output out:

curl --location "https://api.lamini.ai/v1/completions" \
--header "Authorization: Bearer $LAMINI_API_KEY" \
--header "Content-Type: application/json" \
--data '{
    "model_name": "meta-llama/Llama-2-7b-chat-hf",
    "prompt": "How are you?",
    "out_type": {
        "my_response": "str"
    }
}'

Expected Output

{
    'my_response': "I'm good, thanks for asking! How about you"
}

Values other than strings

You can change the output type to be a different type, e.g. int or float. This typing is strictly enforced.

Please let us know if there are specific types you'd like to see supported.

Python LibraryREST API

llm.generate(
    "How old are you?",
    output_type={"age": "int"}
)

curl --location "https://api.lamini.ai/v1/completions" \
--header "Authorization: Bearer $LAMINI_API_KEY" \
--header "Content-Type: application/json" \
--data '{
    "model_name": "meta-llama/Llama-2-7b-chat-hf",
    "prompt": "How old are you?",
    "out_type": {
        "response": "int"
    }
}'

Expected Output

{
    'age': 25
}

Multiple outputs in JSON schema

You can also add multiple output types in one call. The output is a JSON schema that is also strictly enforced.

Python LibraryREST API

llm.generate(
    "How old are you?",
    output_type={"age": "int", "units": "str"}
)

curl --location "https://api.lamini.ai/v1/completions" \
--header "Authorization: Bearer $LAMINI_API_KEY" \
--header "Content-Type: application/json" \
--data '{
    "model_name": "meta-llama/Llama-2-7b-chat-hf",
    "prompt": "How old are you?",
    "out_type": {
        "age": "int",
        "units": "str"
    }
}'

Expected Output

{
    'age': 30,
    'units': 'years'
}

Great! You've successfully run an LLM with structured JSON schema outputs.