JSON Output
Enforcing structured JSON schema output is important for handling LLM outputs downstream with other systems and APIs in your applications.
For an in-depth technical deep dive of how we implemented this feature, see our blog post.
You can enforce JSON schema via the Lamini
class is the base class for all runners. Lamini
wraps our REST API endpoint.
First, return a string:
First, get a basic string output out:
Values other than strings
You can change the output type to be a different type, e.g. int
or float
. This typing is strictly enforced.
Please let us know if there are specific types you'd like to see supported.
Multiple outputs in JSON schema
You can also add multiple output types in one call. The output is a JSON schema that is also strictly enforced.
Great! You've successfully run an LLM with structured JSON schema outputs.
Known issue: Long JSON output times out
To ensure that requests complete in a reasonable amount of time, there is a time limit on all requests including json requests. If your requests exceeds the time limit, try guiding the model to generate a shorter json object, e.g. write a description in 3 sentences or less. Timed out requests may result in failed, incomplete, or missing output.
Workaround
- Reduce the size of the output by limiting the number of fields or the prompt.
- Break down the JSON output into separate smaller requests.
- Contact us to discuss alternative solutions or workarounds for your use case.
Future support
We are evaluating the feasibility of improving our system to handle large JSON output in the future. If we decide to support this feature, we will update our documentation and notify users.
Feel free to contact us with any questions or concerns.