I've always wondered how do you find undocumented / private API ?

Example the Apple undocumented / private API, Play Station, Windows Phone 7, Win32 Kernel, Windows API, hidden callback, etc...

What tools do hackers use to find out about private and undocumented functions?

Where can I read about peoples' experiences delving into private APIs and reverse engineering techniques which reveal the secrets which are normally explained in API documentation?



There are a couple of ways I can think of:

  1. Reflection. Here you get an object and ask it to reveal information about itself, like methods, properties etc. With the right options (depending on language) you can reveal information that isn't "public".

  2. Decompilation. This regenerates (a version) of the source used to create the binary. Again this might reveal non public parts of the API.

You could also add "by accident" to the list. A misspelt call/command that works, or a help screen that reveals too much information.

  • 2
    You could add to the list By accident. Yes, it's less frequent, but it happens ;) Especially when the API owners provoke the accident. – user2567 Jan 29 '11 at 15:30
  • 1
    @Pierre - good point. – ChrisF Jan 29 '11 at 15:33
  • Also, shared library signatures. This is related to decompilation, but exported functions in a .dll/.dylib/.so file are browseable with official tools (dependency explorer in Visual Studio, nm in Unixes) – Chris Dolan Jan 29 '11 at 17:28

Normal tools which you use to develop software :)

Usually undocumented API functions are just that, undocumented, and not really carefully hidden secrets.

Designing a future proof API is hard -- you can easily add stuff into API, but it's really hard to remove anything (without breaking some clients). So, you get very careful when adding anything into API. That's why there might be some extra functions (for testing, in development, a quick hack?) in API, which are not documented and with no guarantees of working or being there in next version.

These undocumented functions can be found somewhat easily, given you know how compilers, linkers, libraries and debuggers work (system programming stuff). Knowing assembly language of the target architecture won't hurt. If your IDE/compiler can build working executables, you can do that "manually" also, and keeping eyes open on that path you might discover some hidden features :)

Example in Unix environment: A scenario where we have documentation only for printf function and would like to know if there are some other printf-like functions. Train of thought might go something like:

1. Check header-files

$ grep printf /usr/include/stdio.h | head -5
extern int fprintf (FILE *__restrict __stream,
extern int printf (__const char *__restrict __format, ...);
extern int sprintf (char *__restrict __s,
extern int vfprintf (FILE *__restrict __s, __const char *__restrict __format,
extern int vprintf (__const char *__restrict __format, _G_va_list __arg);

2. Check library

$ nm /usr/lib/libc.a | grep printf | head -5
         U __asprintf
         U __fwprintf
         U __asprintf
         U __fwprintf
         U __printf_fp

3. Disassemble library function

    $ objdump -S /usr/lib/libc.a | grep -A 10 '_fwprintf' | head
00000000 <__fwprintf>:
   0:   55                      push   %ebp
   1:   89 e5                   mov    %esp,%ebp
   3:   8d 45 10                lea    0x10(%ebp),%eax
   6:   83 ec 0c                sub    $0xc,%esp
   9:   89 44 24 08             mov    %eax,0x8(%esp)
   d:   8b 45 0c                mov    0xc(%ebp),%eax
  10:   89 44 24 04             mov    %eax,0x4(%esp)
  14:   8b 45 08                mov    0x8(%ebp),%eax
  17:   89 04 24                mov    %eax,(%esp)

Or something like that...


disclaimer: I like ChrisF's answer. It does leave out a couple of approaches, I think. If it's placed in the comments to the answer how they are covered, I'll delete my answer.

It might sort of fall under decompilation:

Finding other undocumented APIs can also be through debugging a vendor supplied tool that does kind of what you want and tracking interlibrary calls. This way, you can have an idea of what type of data is sent to where.

Other "custom" tools can then be written to play with these undocumented APIs using python and CTYPES or ruby with it's version of something similar until you find out exactly what it's doing or fuzzing for crashes. This topic is covered in minimal depth by Aaron Portnoy in the: http://pentest.cryptocity.net/reverse-engineering/ and some of his other talks at conferences(I seem to remember he speaks about it directly at a talk in Brazil). It is related to RE, but I don't think it's exactly just general RE. Note: the videos on pentest.cryptocity.net are not JUST this topic. They cover other areas in more depth, this is just touched on. I think because it is often something that testers guard as "the exact steps would be giving away our secrets".

Thanks for reading any feedback appreciated.

edit: one tool that can prove useful for this on the windows side is covered minimally here: http://breakingcode.wordpress.com/2010/08/24/winappdbg-1-4-is-out/
call hijacking for thick java clients to fuzz custom network services is covered here:

that last one is only minimally relevant, but could prove pertinent to where the questioner is eventually going. using APIs you don't own to do...whatever.

  • Adding your own answer than supplies more/different information is good - it's what a Stack Exchange is all about. – ChrisF Jan 29 '11 at 20:23
  • @ChrisF thanks for the clarification. it just seemed like it might have been covered by reversing. i thought it might bear more specific notation of another possible method/more specific subset of reversing (reversing could cover searching for symbols not referenced in the MSDN as well etc). – RobotHumans Jan 30 '11 at 0:58

Hacking sounds quite romantic, industrial espionage, leaks, bribes, theft and plain luck don't. I wouldn't count them out though.

Not the answer you're looking for? Browse other questions tagged or ask your own question.