If you want to have more information on AI Endpoints, please read the following blog post.
You can, also, have a look at our previous blog posts on how use AI Endpoints.
You can find the full code example in the GitHub repository.
In this article, we will explore how to use structured output with OVHcloud AI Endpoints.
Introduction to Structured Output
Structured output allows you to format output data in a way that makes it easier for machines to interpret and process.
We use the langchain4j library to interact with OVHcloud AI Endpoints.
Here is an excerpt of code that shows how to define a structured output format for the responses of the language model:
ResponseFormat responseFormat = ResponseFormat.builder()
.type(ResponseFormatType.JSON)
.jsonSchema(JsonSchema.builder()
.name("Person")
.rootElement(JsonObjectSchema.builder()
.addStringProperty("name")
.addIntegerProperty("age")
.addNumberProperty("height")
.addBooleanProperty("married")
.required("name", "age", "height", "married")
.build())
.build())
.build();In this example, we define a JSON output format with a schema that specifies the name, age, height, and married properties as required.
Configure the model to use
This example uses the Mistral AI model hosted on OVHcloud AI Endpoints.
To configure the model, you need to set up the API key, base URL, and model name as environment variables.
Fell free to use another model, see AI Endpoints catalog.
You can find your access token, model URL, and model name in the OVHcloud AI Endpoints model dashboard.
ChatModel chatModel = MistralAiChatModel.builder()
.apiKey(System.getenv("OVH_AI_ENDPOINTS_ACCESS_TOKEN"))
.baseUrl(System.getenv("OVH_AI_ENDPOINTS_MODEL_URL"))
.modelName(System.getenv("OVH_AI_ENDPOINTS_MODEL_NAME"))
.logRequests(false)
.logResponses(false)
.build();
Calling the language model
Thanks to the JSON mode of the LLM, the response from the language model is received as a JSON string:
UserMessage userMessage = UserMessage.from("""
John is 42 years old.
He stands 1.75 meters tall.
Currently unmarried.
""");
ChatRequest chatRequest = ChatRequest.builder()
.responseFormat(responseFormat)
.messages(userMessage)
.build();
ChatResponse chatResponse = chatModel.chat(chatRequest);
String output = chatResponse.aiMessage().text();
System.out.println("Response: \n" + output);
Person person = new ObjectMapper().readValue(output, Person.class);
System.out.println(person);
Run the application
jbang HelloWorld.java
[jbang] Building jar for HelloWorld.java...
Prompt:
John is 42 years old.
He stands 1.75 meters tall.
Currently unmarried.
Response:
{"age": 42, "height": 1.75, "married": false, "name": "John"}
Person[name=John, age=42, height=1.75, married=false]This code example uses JBang, a Java-based tool for creating and running Java programs as scripts.
For more information on JBang, please refer to the JBang documentation.
In this article, we have seen how to use structured output with OVHcloud AI Endpoints with LangChain4J.
You have a dedicated Discord channel (#ai-endpoints) on our Discord server, see you there!
Once a developer, always a developer!
Java developer for many years, I have the joy of knowing JDK 1.1, JEE, Struts, ... and now Spring, Quarkus, (core, boot, batch), Angular, Groovy, Golang, ...
For more than ten years I was a Software Architect, a job that allowed me to face many problems inherent to the complex information systems in large groups.
I also had other lives, notably in automation and delivery with the implementation of CI/CD chains based on Jenkins pipelines.
I particularly like sharing and relationships with developers and I became a Developer Relation at OVHcloud.
This new adventure allows me to continue to use technologies that I like such as Kubernetes or AI for example but also to continue to learn and discover a lot of new things.
All the while keeping in mind one of my main motivations as a Developer Relation: making developers happy.
Always sharing, I am the co-creator of the TADx Meetup in Tours, allowing discovery and sharing around different tech topics.