Cool approach! The only thing I would change or allow customization for is the prompt you're injecting. Most users will probably want to change this:
"Output a json object or array generated with random content fitting this schema, based on the PORMPT section below." (sic)
My findings from one week of forcing chatgpt to produce structured output:
- it prefers a certain kind of format (the one it uses by default when you ask it without json/markdown), and if you deviate too far, it won't listen. E.g. I tried adding underscores to certain words, which it only did half the time.
- regex parsing the output is quite robust. Even if it adds sections you don't want like intro or conclusion, the parser will just discard these
"Output a json object or array generated with random content fitting this schema, based on the PORMPT section below." (sic)
My findings from one week of forcing chatgpt to produce structured output:
- it prefers a certain kind of format (the one it uses by default when you ask it without json/markdown), and if you deviate too far, it won't listen. E.g. I tried adding underscores to certain words, which it only did half the time.
- regex parsing the output is quite robust. Even if it adds sections you don't want like intro or conclusion, the parser will just discard these