Skip to content

Some ideas about extending the IDA GUI with plugins

The first service pack of IDA 4.9 brought an important change to writing IDA plugins which I doubt many people have noticed. It's now theoretically possible to extend the IDA GUI in any way you like and in a way that's significantly easier than the low-level Win32 API methods already known. The emphasis is still on "theoretically" because even though it's looking really good, a lot of stuff needs to be figured out first. So far the method only works if you write plugins with C++ Builder but in the future it might be possible to make it work with all C++ compilers.

The important change I've mentioned is that former versions of IDA were statically linked to the Borland C++ Builder runtime modules. The most recent service pack however is linked dynamically. This is significant because of some problems in the VCL which prevents mixing controls from two binaries.

The first thing that's necessary to extend the IDA window is to get a pointer to the window object. This is spectacularly easy. If you check through the exports table of the idag.exe file you can find an entry called _IdaWindow. This is exactly the pointer you need. It can be retrieved using the following code (which goes into your plugin). Sorry about the formatting. One day I'll figure out how to post code here. The strong below is actually an asterisk. The same happens in the next code snippet.

CODE:
TForm* GetIdaWindow() { return <strong>(TForm**)GetProcAddress((void*)0x400000, "_IdaWindow"); }

You might want to clean up the image base parameter in actual plugin code but so far it worked well. There are some other exports with very promising names but I haven't yet looked into them.

The parameter returned by this function is actually of type TIdaWindow (which extends TForm), not of type TForm. Unfortunately the header file where TIdaWindow is defined is not available. Therefore it can't be used in the plugin code. The most specific known class must be used. This is TForm.

It should be possible to recreate the header file though. Using tools like DeDe or my (still non-public, I think) RTTI Viewer you can easily check out the objects that form TIdaWindow (unpack the ASPacked idag.exe first). If the Borland compiler doesn't randomly shuffle the objects while compiling the code it should be straightforward to recreate the TIdaWindow header file. Should that work it would be possible to use TIdaWindow objects in plugin code. This would simplify extending the GUI tremendously.

Should recreating the TIdaWindow object not succeed there's still plan B. Look up the components dynamically. Using a more or less sophisticated algorithm you can easily iterate over all components of the IDA window and pick the one you need from the names of the components.

This algorithm shows how to look up the tab control which is used to handle the MDI windows of IDA.

CODE:
TTabControl</strong> GetTabControl(TForm* idaw) {     for (int i=0;i<idaw->ControlCount;i++)     {         if (idaw->Controls[i]->Name == "TabControl1")         {          return (TTabControl*)idaw->Controls[i];         }     }     return 0; }

This algorithm reveals the general structure of the IDA Window. Pass the window object pointer itself to the function at the first call and you'll get a nice overview of the TWinControl tree of the window. Note that DeDe or RTTI Viewer can do this too. This example merely shows how to do it at runtime.

CODE:
void map(TWinControl* c) { for (int i=0;i<c->ControlCount;i++)     {      Application->MessageBox(c->Controls[i]->Name.c_str(), c->Name.c_str(), MB_OK);         TWinControl* pControl = dynamic_cast <TWinControl*>(c->Controls[i]);         if (dynamic_cast<TWinControl*>(c->Controls[i]) != 0)          map((TWinControl*)c->Controls[i]);     } }

This is actually trickier than it looks like. This code only works if the plugin is built using runtime packages (a C++ Builder compiler option). Unfortunately plugins built using that option tend to crash my IDA when IDA is closed. Plugins not using runtime packages seem to be stable but the code doesn't work anymore. I think I know the reason for this behaviour. If you have a plugin that uses statically linked packages you have an object hierarchy in the IDA executable as well as in the plugin executable. The code above uses RTTI (run-time type identification) to find out which control is a TWinControl. The way RTTI works for VCL objects is that the object hierarchy is traversed and it's checked whether two classes are the same. There are however two object hierarchies, one from the IDA executable and one from the plugin. The code above compares TWinControl from one hierarchy with TWinControl from the other hierarchy. This always fails.

There are at least three ways to fix this problem:
1. Figure out how to make plugins using runtime packages work in a stable manner
2. Attempt to write your own RTTI methods that work across different object hierarchies (for example by comparing class names which should work most of the time).
2a. Do this by using the undocumented functions _first_base / _next_base from typeinfo.h (seems to be tough)
2b. Do this by parsing the object hierarchy yourself (seems to be easier; layout of VCL objects is relatively well documented)

Once this is all figured out it's possible to add controls to the IDA window using standard C++ Builder code. One last problem remains. The new controls must respond to messages send to them by standard controls. Here's an example. Pressing F6 in IDA activates the next MDI window. If you add a tab to the window tab control this doesn't work out of the box though. When you select the last standard tab and hit F6 you get a message saying 'Command "NextWindow" failed'. Someone needs to figure out what the problem is. Research should begin in the TActionList "al" which is part of TIdaWindow. There's an action called "cmNext" which apparently handles this.

I'm confident that once these issues (and probably some others) have been figured out it would be possible to write a nice IDA GUI SDK for IDA plugins. Extending toolbars or menus and embedding new MDI windows with any content is just a few steps away. I don't think I'll have time to do it myself in the next few weeks but maybe someone else with more free time and some cool ideas can get further than I have.

Trackbacks

No Trackbacks

Comments

Display comments as Linear | Threaded

No comments

Add Comment

Enclosing asterisks marks text as bold (*word*), underscore are made via _word_.
Standard emoticons like :-) and ;-) are converted to images.
BBCode format allowed
Form options

Submitted comments will be subject to moderation before being displayed.