5

Suppose I have a compiled dynamic library: .dll, .lib, .so etc. Is it (theoretically) possible to automatically create C header file for such a library? Is there an existing tool that does that?

Intuitively it looks to me like it should be possible. After all, the linker is able to find the necessary symbols inside the dynamic library and resolve those symbols at runtime. But still, some information may be missing. If so, which one? Argument types? The return type? I know that when a C++ library is compiled without the "extern" flag, with the information about the types being embedded into the name. Would this kind of library be "reverse-engineerable" ?

Update. Thanks for all the responses -- it seems like there is a consensus that it is generally NOT possible, unless one is willing to try really hard (I guess by examining the assembly and seeing how many parameters are being popped off from stack) OR the library is compiled in the debug mode.

The purpose of this question is neither to obfuscate my own library, nor to decompile an existing one. Rather, it is a theoretical question: is such action possible for a generic library? The reason for my curiosity is that I'm trying to understand the legal implications of having a library licensed under GPL while its header files licensed under LGPL.

6
  • @BasileStarynkevitch The motivation is in the first two sentences: // Is it (theoretically) possible to automatically create C header file for such a library? Is there an existing tool that does that? //
    – rwong
    Commented Oct 18, 2017 at 18:43
  • That is not enough. Is it from the point of view of finding more about a library, or on the contrary to hide it more against reverse-engineering? Commented Oct 18, 2017 at 18:45
  • 1
    "legal implications of having a library licensed under GPL while its header files licensed under LGPL." IANAL, but that does not seem to make any sense. Ask a better question on opensource.stackexchange.com Commented Oct 18, 2017 at 19:08
  • 1
    legal advice is off-topic per help center
    – gnat
    Commented Oct 18, 2017 at 21:07
  • @gnat I understand that. Which is why I'm not asking a legal advice, I'm asking a technical question. The word "legal" only came up there because I was asked for the reason for asking this question. The question however is asked generally, and other people who have other reasons might find it useful one day.
    – Pasha
    Commented Oct 19, 2017 at 15:55

4 Answers 4

8

In general, it won't be possible (at least not with ELF files on Linux). Because type and signature information is not kept (e.g. in ELF symbol files). But C++ compilers are doing name mangling to encode some type information in their ELF symbol name. However, C compilers don't do that. And C++ name mangling doesn't tell enough (e.g. it would tell that the first argument of some function is a Foo* pointer, but it won't describe the fields inside class Foo).

For example, you can't even (reliably) know how many arguments a given function (notably a C one) is expecting, and even more their type. And some functions don't have externally visible names (e.g.static functions, but read also about visibility function attribute on Linux). Read more about ABIs (e.g. here for Linux on PCs) and calling conventions.

However, if the code has been compiled (using -g) with debug information in DWARF, it could be possible. Read also about the strip command.

And if you have additional a priori information (for example, knowing that the given library is distributed by Debian) it probably should be possible. Some projects (perhaps FOSSology, but I could be wrong) simply guessed free software libraries by comparing their constant literal strings against a previously built database of them.

BTW, what you are looking at is more or less called a decompiler and the process would be decompilation. Read also about obfuscation.

With a lot of efforts and resources (e.g. what the NSA would be capable of) many things could be in practice possible, but difficult and costly.

3

No, this is not generally possible. Libraries are self-describing in that they list all available symbols (functions and variables with external linkage). But they do not contain sufficient information about the types.

This is not just a matter of providing the names of all types, e.g. that a function receives or returns a struct Foo. To properly satisfy the calling convention we need to know the layout and specifically the alignment of that type. So we would need to embed the complete type information that would also be provided by a header. (Of course incomplete types can be elided.)

The header files can be seen as an interface description language that provide all this information.

It is of course possible to design a dynamic library format that contains all relevant information, notably JVM .class files do that.

As a historic note: C did not enforce parameter types prior to standardization with C89 (“ANSI C”). A function declared without a prototype could not be checked in any way, and the programmer would have to know the correct types. In this sense, functions in dynamic libraries still behave a lot like functions in K&R C.

3

It depends on the compilation system, and especially if only C or a C & C++ compiler, as these are aspects of the development system that are not standardized behaviors.

Some systems will define a DLL's entry points but not even detect mismatched parameter count, as the raw identifier name is exported without mangling or decoration.

This highlights that insufficient metadata exists in raw DLL mechanism to automatically infer a .h header file.

Other linkage systems capture the parameter count but not the full parameter types. For example, on windows __stdcall:

An underscore _ is prefixed to the name. The name is followed by the at sign @ followed by the number of bytes (in decimal) in the argument list. Therefore, the function declared as int func( int a, double b ) is decorated as follows: _func@12.

Note that the return type is not encoded in decoration (most languages won't support overloads where only the return type differs).

Beyond that, even name mangling of C++ does ensures only matching between caller and callee of the same signature. Thus, while all parameter types are matched to ensure a full signature match of the proper overload, actually type declarations (e.g. of structs and such) that we would expect to see in a header file are absent. Thus, only the name of a struct type will be available in the mangled/decorated name, but not that struct's members.

C++ also does not always use the name export mechanism available in DLL's, and this could go to inline methods and private methods (and theoretically virtual methods also). Note here I'm distinguishing between DLL exports (post compilation and post static linking that creates the DLL) and object modules (post compilation but before static linking).

1

With debug symbols

Most of these answers seem to completely ignore debug symbols. Using tools like NSA's Ghidra, it's very easy to "Export C headers", given that the DLL (or shared objects) was built with debug symbols, or if debug symbols are available via PDB (under Windows). (Furthermore this even works with plain object files. So if you lost your headers at some point during development, you can use that to recreate them).

Ghidra usually picks up the PDB next to the DLL, but it can be manually specified, via the "Load PDB Manually" function under File.

To export the headers, right click the imported DLL, in the "Data Type Manager" pane in Ghidra in the lower left corner and select "Export C Header".

You might want to move the types you're actually interested in to a new archive, before though, as debug symbols include definitions beyond just the interface.

Furthermore there exist utilities like pdbex, that allow to directly export headers.

With name mangling

With Ghidra or any other RE tool like IDA Pro, in general, it's fairly easy to analyze and recreate structures, compatible with the original library. Given, that the name mangling of C++ exposes the function types, it's going to be like a game of Sudoku (or less spectacularly, a task of Gaussian elimination), to figure out the types.

For example, assuming we have a function, with prototype

Texture *CreateTexture(const BITMAP *pBitmap);

and another function for obtaining the size of the Texture:

size_t GetTextureWidth(Texture *pTexture);

then, finding out the structure of Texture is trivial, as it will contain code that looks like this

size_t GetTextureWidth(Texture *pTexture)
{
    return pTexture->m_nWidth;
}

This will allow the reverse engineer to simply determine, at which offset the member was located and therefore recreate the headers.

Intercepting API calls

Recreating the headers for using a library isn't really that hard, and wrapper libraries can be used to intercept and analyze function calls. (For getting a decent idea of how to use the library or patching bugs). Using LD_PRELOAD for example under Linux (similar ways exist for Windows). This will allow to preload the wrapper library so your symbols get resolved before the actual library. Then you can call the original symbols. I've prepared such a patch once, to work around a bug present in a library. It comes in very handy, when working with libraries that have memory management issue, for example because their reference counted resources are never freed internally. Or because they tried to fit the current working directory into a buffer, that they didn't allocate enough memory for.

Without name mangling

Even without name mangling, most API is easily reverse engineered, because the tools can derive the number and size of arguments as well as return type. So even without name mangling, you would get something like:

undefined8 GetTextureWidth(astruct_1 *param_1)
{
    return param_1->offset_0x4;
}

which is still pretty obvious.

Without symbol names

Another important thing to mention is, that the names of the symbols can be omitted in Windows and replaced with ordinal numbers. In this case, the ".lib" file contains the name and the ordinal number, which can be dumped using "dumpbin". Without the ".lib" file, the reverse engineer only knows, that a function is present.

undefined8 FUN_0000BEE0(astruct_1 *param_1)
{
    return param_1->offset_0x4;
}

Making it much harder, but still easy enough. I would however suggest against ever doing that, as it only complicates the build process, doesn't really stop any determined reverse-engineer and only stops legit customers from potentially fixing some bugs themselves.

Manually analyzing the assembly is no longer necessary, with the tools currently freely available.

Licenses

For your question, having LGPL headers, makes sense, as it allows people to implement their own library with their own license (like MIT) and continue using code they wrote for it in their own project, as well as the project that used the LGPL headers, assuming it's compatible with your LICENSE. (It might or might not infect your library with LGPL, though. But that is a purely legal question, I'm not qualified to answer).

I don't see however, how this is in any way influenced by the ability to obtain the function prototypes and type structures, which is very easy to do.

1
  • 1
    Feel free to leave constructive feedback :-) Commented Feb 13 at 15:38

Not the answer you're looking for? Browse other questions tagged or ask your own question.