How far can one debug a low-level API in closed-source environments?

Question

Assume a low-level API is provided without source code (e.g. DirectX).

The API provides a virtualization of hardware resources (GPU, CPU, audio card, etc.), which enables the user to call hardware-specific drivers through the OS. As a software developer, we do not care about the implementation details at the hardware level.

This is a proper implementation of encapsulation. That is, the API enables the client to work with a more abstract software layer. Therefore, the user is not required to think about lower-level contracts between the OS, the driver, and the CPU/GPU.

In the absence of encapsulation, it could become difficult (even unmanageable) for the user to work manually everything from the compiler, to the OS, to understanding all the intricate details of the CPU/GPU architecture. Furthermore, from a safety standpoint, accessing and modifying sensitive data may break the API's integrity. Usually, the contracts are specified between the manufacturers and the OS/software developers.

During the API implementation, developers specify which properties of the API should be exposed to the customer.

As a consumer of a low-level API, I have several general questions.

• Nothing's perfect and bugs can appear at the implementation phase. When this occurs, it is just impossible to backtrack those errors, as the source code is hidden. How far can we debug stuff in such situation?
• Is encapsulation, in this context, a drawback, as it does not give power to the user to backtrack private, encapsulated data, for debugging purposes? In the worst case, only the binary codes are supplied to the client, and nothing can be debugged at a lower level.
• Assume we have transparency at all the levels between the hardware, the OS, and the application (up to controlling every bits in the CPU registers). In this case, even though we have complete control upon all the resources of the computer, would we lose the abstraction provided by the API?

I have the impression that, at a certain point, we are stuck as we are not able to debug some low-level APIs due to the decisions of vendors and developers made upstream.

Answer 1

Nothing's perfect and bugs can appear at the implementation phase. When this occurs, it is just impossible to backtrack those errors, as the source code is hidden. How far can we debug stuff in such situation?

We can debug as far as our debugging tools, our time budget, our mental ability and knowledge to reverse-engineer undocumented machine code allows us.

Of course, beyond the barrier of an encapsulated, closed-source API, this becomes inherently harder than debugging the code on "our" side of the system, and for systems beyond a certain size and complexity, this may simply become unfeasible. Depending on the support the API vendor provides (which also depends on contractual obligations), it may simply cost less time and money to create a test case for a certain bug, send it to the vendor together with a bug report and wait until they fix it or provide a workaround.

Debugging on machine code levels and reverse engineering foreign APIs definitely happens for certain systems by specialists: just look at the number of security issues which have been found in the past by external people in closed-source systems.

Is encapsulation, in this context, a drawback, as it does not give power to the user to backtrack private, encapsulated data, for debugging purposes?

This is an old, opinionated debate started in the 1970s between Parnas and Brooks whether it is better for the maintenance of a system to use rigorous information hiding, or to open-source systems for debugging purposes. Brooks initially was proposing to keep everything as open as possible, but switched his point of view in 1995 in agreement to Parnas that information hiding would be crucial for designing systems beyond a certain complexity.

However, if you read the linked article above to the end, you may see that these two points of view are not necessarily opposing. Properly encapsulated APIs will ideally allow you to use them without looking at the implementation details, but if you have access to the source, it may also allow you to look behind the scenes in case you think you have to, or in case there is no better documentation at hand. On the other hand, not having access to the source may force you to contact the vendor in certain situations. Some vendors want you to do this, others not, sometimes you simply don't have the option of communicating with them. If in the end this turns out to be a benefit or a drawback is heavily case dependend.

Assume we have transparency at all the levels between the hardware, the OS, and the application (up to controlling every bits in the CPU registers). In this case, even though we have complete control upon all the resources of the computer, would we lose the abstraction provided by the API?

If we use tools which provide this transparency (and for most major platforms, there are tools which allow to switch the debugging view from a high-level view down to the machine-code level), it is still our own decision if we use them, and on which level of abstraction we prefer to stay. As long as an API provides a proper abstraction, you don't lose it in case you don't make the decision to ignore it willingly.

Issues usually arise only the other way round: if an API creator does not provide well encapsulated abstractions and forces their users to inspect an implementation for finding out how an API might be used properly, then users loose the ability to work with the API in an efficient manner.

In short: the feasibility of debugging a low level API's implementation depends on several factors: the type, size and complexity of the APIs implementation, the vendor support and your relationshop to them, the documentation, alternative debugging features provided by the system (like logging), the whole eco system around it, once own abilities and tools, and last but not least the time and money one is willing to invest.