There's a propertyOrdering param documented in Vertex AI that should solve this: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/control-generated-output
But it doesn't work in the Generative AI SDK. Other users have already reported this issue.
For the benchmarks, I excluded FC and used already sorted keys for JSON-Schema.
But it doesn't work in the Generative AI SDK. Other users have already reported this issue.
For the benchmarks, I excluded FC and used already sorted keys for JSON-Schema.
Comments
But JSON-Schema performed worse than NL in 5 out of 6 tasks in my tests. Plus, in Shuffled Objects, it did so with a huge delta: 97.15% for NL vs. 86.18% for JSON-Schema.
Given this result and the key sorting issue, I'd suggest avoiding using JSON-Schema, unless you really need to. JSON-Prompt seems like a better alternative.
For now, there are no clear guidelines on where each method works better.
Your best bet is testing your LLM running your own evals.
and the github code: https://github.com/dylanjcastillo/blog/tree/main/_extras/gemini-structured-outputs