validating APIs

Question

We have a series of hardware and we have decided on a common API to be given to the customer for use. The customer will use this API in order to program the hardware. The API includes functions for establishing a connection with the h/w like X_Open,X_Close,X_SetConfig, X_GetConfig, no of hardware connected, error handling with the corresponding structs to be filled by the API or the customer. The hardware will ofcourse be validated independently also. What strategies can I use to validate this API ? We want to test this API i.e it works as expected for instance we are actually able to setConfig in the h/w or get no of devices on the bus or if the API function X_Open is called , the function is able to establish a connection to the hardware as expected or say API function X_GetNoDevices returns no. of present devices on the bus. We want to validate that all API functions work as described to the customer

Answer 1

Essentially what you refer to is known as compatibility test suite, a set of sample applications that demonstrate reference usage of your API, along with documentation clearly describing expected behavior of these applications, allowing to judge whether API functions as intended.

When your API is modified or extended, you change your test suite accordingly by modifying / extending your sample applications and adjusting documentation for expected behavior.

Stuff like that is widely used in the industry, one example that immediately jumps to mind is Java TCK / JCK. Another example is Khronos conformance tests for OpenGL and other API implementations.

If you plan for a long term use of such test suite, it makes much sense to treat it as a separate software product, with version control, issue tracking, releases and such.