Part of my current product uses GPT-4 with a prompt to extract information from plain text. As behaviour of large language models is inherently opaque and failure cases unpredictable, I was wondering how standard software engineering practices translate into working with generative language models such as GPT-4?

  • 7
    What exactly are you trying to test? Commented Apr 15, 2023 at 20:08
  • @PhilipKendall, I would guess, he's trying to test that the GPT part works properly, and produces outputs that would be agreeable to him if he were to review each and every output. As for how the model would know what he finds agreeable, I think that's the missing link of logic that needs to be made explicit!
    – Steve
    Commented Apr 15, 2023 at 20:54
  • Since you can't manipulate the underlying neural network or re-train the model, you can really only test how it behaves given a certain prompt, and then fine tune the prompt. The tests would have to be statistical in nature (so, you'd have to define what success means in statistical terms), and you'd need to have a large set of sample data, and randomly select a subset of samples from that. You'd probably be able to automate a lot, but not 100%. Commented Apr 16, 2023 at 12:20
  • @FilipMilovanović: I am actually wondering if only statistical approaches can be used, or if such a component might be tested using "AI" techniques for the tests themselves.
    – Doc Brown
    Commented Apr 16, 2023 at 17:30
  • 1
    That's the neat thing: you don't.
    – Alexander
    Commented Apr 16, 2023 at 21:39

3 Answers 3


Your primary goal is surely not to test GPT-4, that is the job of it's vendor OpenAI. Your goal is to test the software you write. So for your automated tests, have a stable interface around GPT-4 and use a simple mock implementation during the tests which behaves deterministicly.

Up to this point, this is pretty standard: when dealing with unreliable or indeterministic 3rd party components, they should be replaced during automated tests. From this point of view, a language like model like GPT-4 isn't different from components like a search engine API or a weather forecast API or a stock market API.

However, I can imagine scenarios where your system uses GPT-4 in a specialized way, and you have the expectation to get very specific, maybe deterministic output from certain inputs. Hence you want to have tests which assure this works as intended even when the GPT-4 model gets updated.

You wrote you want to

"extract information from plain text"

That is very vague and can mean a lot of different things to different people. Here are a few examples, it could mean

  1. to use a regex to look for certain patterns or keywords in the text, or

  2. to make a list of all nouns in some plain text, or

  3. to analyse if a certain text is relevant for a certain topic, or

  4. to analyse if a certain text conforms to some code-of-conduct, or

  5. to generate a short summary of a longer text.

For the first four types of questions it should be possible to advise the language model to produce it's output in some normalized form (for example, tell the model to omit all introductory sentences, tell it which separator to use and that you want the output in alphabetical order, or tell it to answer with yes/no exclusively). It should also be clear to start always with a clean session, with no predetermined context. Normalized output can be way easier tested against some expected output, maybe with some tolerance. And questions with only one correct answer are way more easy to address by tests than questions where multiple correct answers exists, or questions where it is not easy to decide for a human if an answer is correct.

Testing the output to questions of the fifth kind automatically against some requirements like "should still be correct in content", however, may be impossible, because content-correctness is not what a model like GPT-4 guarantees. So whether this approach is feasible or not depends a lot on the specific kind of use case you have in mind.

I can also imagine components like GPT-4 to provide some testing mode or "deterministic mode", if not now, maybe in an upcoming version. I don't know if GPT-4 offers something like that yet, but it might be worth to ask the vendor or the GPT user's community. That would enable one to write automated integration or acceptance tests without mocking.

  • "Your goal is not to test GPT-4, that is the job of it's vendor OpenAI." - how could OpenAI possibly test it without knowing specifically what it is being used for by the OP? No developer, using a component with no specification of behaviour (nor reasonable inference about the purposes the component was designed to serve), would incorporate that component without testing it himself.
    – Steve
    Commented Apr 16, 2023 at 12:27
  • @Steve: there is an integration point between a deterministic rule-based system and the output of a non-deterministic machine learning model. At some point you need to formulate rules around this unpredictable output. That is the component to test. This does not replace true integration testing. You still need to do that separate from automated test cases, just like you would with a weather forecast API. Commented Apr 16, 2023 at 16:58
  • @Steve: I admit, you have a point in case the OP has a use case where they provide specific input and expect a specific kind of output. I rewrote my answer partly.
    – Doc Brown
    Commented Apr 16, 2023 at 21:17

As you say, because GPT and similar technologies are opaque, it is impossible to analyse the logic to determine how it should be tested in a conventional way.

Typically when writing automated tests, the programmer uses insight into the target logic, the different code paths, any perceived edge behaviours, and knowledge and experience of similar logic and typical failure modes, to design tests which exercise those different paths, checks implicit constraints, and examines behaviour at the edges.

Because GPT hasn't involved the programmer specifying any constraints, and has no internals that can be usefully examined, there are effectively no tests possible (other than an exhaustive test with a data set of all possible inputs that need to be handled and specifying their correct outputs, which defeats the object of using GPT).

This is effectively why all real programmers are sitting back and laughing at the hype around "artificial intelligence". Even rubbing our hands like Fagin, at stimulated demand for our services and fixing the mess others have made!

  • I like this answer, but I think it is good to point out that automated tests are typically deterministic: given some input you expect exactly some output. We don't have that luxury anymore with large language models. Commented Apr 16, 2023 at 1:00
  • @GregBurghardt, precisely. And that inexactitude will prevent composition into more complicated systems. I was also thinking to myself whether some kind of statistically fuzzy data set might work for testing, but again, without knowing the internals, there are always specific inputs for which it can fail very badly and unexpectedly. Moreover, the work spent on generating this fuzzy data set for testing, is probably akin to the work and expertise required for doing traditional programming to a high level of quality.
    – Steve
    Commented Apr 16, 2023 at 6:49

The testing requirements for an app powered by OpenAI are relatively the same when testing a third-party dependency.

Here is a summary of techniques:

  • Your tests are associated with specific OpenAI model releases. For example; you would verify your tests against gpt-3.5-turbo-0301. Should you upgrade your usage of a model to a newer version, then you would need to update all your test data.
  • You treat the OpenAI APIs as pure-functions. Meaning, the same inputs always yield the same outputs. When you call the API with a prompt that string always gives the same completion. You increase coverage by using different prompts.
  • OpenAI implemented their API using the OpenAPI standard. They also publish those API specifications in a repo (https://github.com/openai/openai-openapi)
  • There are tools availible that let you quickly host a mock of an OpenAPI specification. I found this quickly, but there are others: https://stoplight.io/api-mocking
  • You reproduce edge cases like errors, 400 status codes or completion failures using special prompts. This is where the mock server uses conditions based on the input prompt to decide what response the test is excepting.
  • You can use pass-through where the mock forwards some calls to the real OpenAI API if that aspect of the API does not impact the testing, but this might require a custom mock server.

You maintain all the above as you would normally maintain tests. These techniques apply to both unit testing and integration testing.

There are no short cuts here. Testing against a vendor API requires setup time, on going maintenance and verifying mocking of the API is accurate.

The key points here:

  • tests should be pure. Same inputs yields same outputs.
  • re-running tests without any code changes should yield same test results as last time.
  • it's the job of the engineer to verify edge cases, errors and unknowns, in the form of tests.
  • there is no need to hit the real OpenAI APIs
  • test execution should be very fast
  • tests should be run on any code changes

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.