Programming stuff

After my last two book reviews were rather negative, I'm happy to say that this review is going to be positive again. Mario Herwardt's and Daniel Pravat's book Advanced Windows Debugging (Official Website / Amazon) keeps what the title promises. It's a book for people that need to find bugs in Windows programs that are for whatever reason (too) difficult to find with the "normal" developer tools like the integrated Visual C++ debugger. That's not what the authors say in the introduction of course. They say it's a book for everyone that does Windows development. And that's probably true because the book is a real eye-opener for what's possible with the debugging tools provided by Microsoft. Nevertheless I guess that most developers will probably rarely if ever leave the cushy environment of their IDE's debugger. But just in case you do, this book prepares you well. Continue reading "Book Review - Advanced Windows Debugging"

The 1.2.0 release of Hexer improves the usability of Hexer. On Windows, Hexer can now be integrated into the context menu that appears when you right-click on files in the Windows Explorer (and elsewhere). Furthermore the main window of Hexer is now scrollable. This gives the users extra space and allows him to open more windows. Another nifty thing is that the scripting dialog now supports syntax highlighting and other small features that make it easier to write scripts.

You can download Hexer 1.2.0 here.

Complete list of changes:

New Features

• Core: The main window now contains scrollbars. This gives the user more space for open MDI windows.
• Core: A tool bar that shows buttons for all open windows was added to the main window. This makes it easier to quickly switch to another window.
• Core: Progress bars are shown during long operation like plugin loading, file loading, or script execution.
• Core: The Windows menu shows a list of open windows for the user to switch to.
• Core: Added Cascade/Tile Windows feature in the main window
• Core: User is asked whether he wants to make a backup of a file before saving it.
• Core: It is now possible to stop running scripts that are stuck in infinite loops or simply take too long to execute.
• Core: Added optional Windows Explorer integration. Hexer can now be started by right-clicking on files.
• Core: Improved the scripting dialog (added syntax highlighting, tab handling, undo and redo, cut/copy/paste, icons, and other things)

Bugfixes

• Core: Problems with the split windows in the settings and plugin dialog were fixed.
• Core: Improved the layout of the hex window status bar.
• Core: Fixed a few problems during plugin initialization and loading.

Changes to the Plugin API

• All MDI windows added to the MainWindow must now extend the class HexerWindow
• Added the class HexerWindow which is the base class of all MDI windows of the main window
• Added the methods tileWindows and cascadeWindows to the IMainWindow interface
• Added the method removeChildWindow to the IMainWindow interface

Screenshots of the new scripting window and a new screenshot of the main window:

This week I managed to read Reverse Engineering Code with IDA Pro. I was pretty curious about the book because it's the first book specifically about everyone's favourite disassembler IDA Pro and it turned out to be very different from what I expected.

Co-authored by Dan Kaminsky (editor), Justin Ferguson, Jason Larsen, Luis Miras, and Walter Pearce, Reverse Engineering Code with IDA Pro sets out to give an introduction to IDA Pro and how to use it to reverse engineer software. The book is approximately 310 pages long and divided into nine chapters ("Introduction", "Assembly and Reverse Engineering Basics", "Portable Executable and Executable and Linking Formats", "Walkthroughs One and Two", "Debugging", "Anti-Reversing", "Walkthrough Four", "Advanced Walkthrough", "IDA Scripting and Plug-Ins").

Continue reading "Book Review - Reverse Engineering Code with IDA Pro"

Hexer 1.1.0 (click here to download) is primarily a bugfixing release. The following things changed since Hexer 1.0.0:

New Features

• Core: Added a status bar that displays the current offset in hex windows
• Core: Support for the ENTER key to execute a dialog's default action was added to all dialogs
• Core: When selection a result of a file comparison, the corresponding area is highlighted in the hex window
• Core: Added options for configuring the appearance of the hex windows
• Core: Structure Viewer was sped by only calculating the values of visible structure elements
• Plugins: Added hotkey management to avoid duplicate hotkeys
• Plugins: Added a sample plugin that shows how to write plugins in Java
• Plugins: Plugin CLASS and JAR files can now be placed in subdirectories of the plugins directory

Bugfixes

• Core: Improved Linux support
• Core: Fixed a problem with copy/pasting
• Core: Fixed a problem with incorrect highlighting in dialogs like Compare Files
• Core: Structure highlighted is removed when the Structure Viewer dialog is closed
• Core: Fixed some problems when searching beyond the end of files
• Core: Existing dialogs are brought to front when their menu hotkey is pressed again
• Plugins: Fixed a problem that prevented plugin JAR files from being loaded

Changes to the Plugin API

• Added the option to insert bytes into IHexFile objects
• Added the registerAccelerator method to IHexWindow which helps to make menu accelerators unique
• Added finer-grained methods for watching hex windows to IMainWindowListener
• Added several methods for configuring the appearance of hex windows to IHexWindow
• Added a clear method to IStructureHighlighter which clears previous data highlighting
• Added the IStructureViewerHelper which can be used by data viewer plugins to figure out whether they are visible in the Data Viewer dialog
• Moved the method addChildWindow from IPluginInterface to IMainWindow

I hope Hexer works on Linux now. At least it does work on my Ubuntu.

Please report bugs and request features by replying to this blog entry or by sending me an email (see the Contact information in the docs directory of the RAR file).

I finally got around to write an example plugin for my hex editor Hexer to show how simple it is to extend Hexer according to your own needs. The Java plugin I am going to present calculates the entropy of files according to the method presented on Ero Carrera's blog. The plugin adds a new tab containing a line chart and a button to the File Statistics dialog. When the user clicks the button, the entropy of the active file (that is the file in the last active hex window) is calculated and shown in the line chart. The screenshot below shows the entropy distribution of Notepad.exe.

You can download the source file of the plugin here. The archive contains the source file EntropyCalculator.java as well as two class files which were created by compiling the source file using Java 1.6. To install the plugin, simply copy the two class files to the plugins directory of your Hexer installation. Since the plugin uses the JFreeChart library to display the graph it is also necessary to get the files jcommon-1.0.12.jar and jfreechart-1.0.9.jar from the JFreeChart package. Copy those files into the jars directory of your Hexer installation.

Continue reading "Sample Hexer Plugin: Calculating the entropy of a file"

Hi everyone and welcome to another post in my favourite blog entry category: Book Reviews. I'm happy to announce that for the first time ever I have actually managed to read a book and write a review of it before its official release (unlike my other reviews where I often review three year old books). I'm talking about Adam Shostack's and Andrew Stewart's new book The New School of Information Security here which will be released tomorrow.

The New School of Information Security is a weird book. From the title of the book you'd think that this is a book about information security for people who have at least some kind of clue about information security. I mean why would people that do not have a clue about information security read a book about reforming and improving the field of information security? Unfortunately this assumption is wrong.

Continue reading "Book Review - The New School of Information Security"

Just a few days later than originally announced, I managed to finish the first release of my hexeditor Hexer. It's available for download here. Please report any problems or suggestions by replying to this blog entry.

The coolest feature of Hexer is the scripting and plugin support. Note that Hexer only supports ECMAScript out of the box. If you want Python or Groovy scripting support, you need to install and download the necessary Jython/Groovy packages. Please read the manual in the docs folder to find out exactly what to download.

There are sections in the manual that describe how to write plugins and scripts. Example scripts are included in the RAR file. There are no example plugins yet but I'll write something about them on my website before next weekend.

Update:

Known issues:

• Doesn't work on Linux

Remember Hexer? It's back. In Java form. For those who don't remember, back in 2005 I posted the alpha version of Hexer, a hex editor written in C#. Like most of my projects, Hexer was abandoned for lack of time. However, the basic idea of an easily extensible hex editor still appeals to me and so I decided to bring it back.

The new version of Hexer is not written in C# anymore. I ditched C# for Java for practical reasons. The primary reason is that I'm using Java at work which means I'm doing nearly all of my development these days in Java. The second reason is that we've had lots of code that's necessary for Hexer in our company's internal Java library already and I only had to combine our Java library with some new code. That way I managed to implement the first Java version of Hexer in a very short amount of time.

Alright, if everything works out smoothly the first version of Hexer 1.0.0 will be released next week (depending on how fast I can write the remaining unit tests, the documentation, and how quickly I can get someone to update the company website where Hexer will be available as a free download). So let's have a sneak preview of Hexer 1.0.0.

Continue reading "Hexer 1.0.0"

Two days ago I had the pleasant experience to participate in some kind of informal reverse engineering session with three other guys. Between dinner and way too long after midnight we debugged a popular piece of malware that is floating around the internet right now. The first guy already reverse-engineered an earlier version of the malware. He was the guy in charge who did most of the debugging. The second guy was the author of a program that monitors and logs the behaviour of processes, especially malware processes. The goal of the session was to find out why the malware sample worked perfectly in VMWare (after we patched out the VMWare check, at least) but crashed as soon as second guy's monitoring tool was active. The third guy was very familiar with the malware too but on a higher level (behaviour, network activity, how it spreads, its historic development and usage, ...). I was the fourth guy. Without a direct interest in the malware or the malware monitoring tool I just wanted to see what goes wrong. Furthermore I was the guy for snarky comments from the background like "see, I told you take a VMWare snapshot before stepping over that call".


Continue reading "Pair Reverse Engineering"

Over the last days some group released passwords to nearly 45000 MySpace accounts and they announced to release another 30000 passwords in the next few days. I used a few hours before Saturday's lunch to write a small program that analyzes the passwords that were released so far.

At worst the results of this are a useless time-filler, at best it's a case study of what happens if a website forces their users to choose passwords with certain minimum requirements. MySpace demands that every password contains at least one non-alphabetical character (like 0, 1, 2, or !, ?, @). How the users adhered to this requirement can be seen in the tables below.

It is my understanding that the 43713 passwords that were leaked so far come from fishing sites that trapped people to enter their password. This makes the passwords less reliable than a password list hacked straight from the MySpace servers. People could have misspelled their MySpace passwords or they could have entered fake information after they noticed that someone was trying to steal their password. A quick analysis has shown that probably less than 1% of the leaked passwords suffer from these problems. Continue reading "A brief analysis of 40,000 leaked MySpace passwords"

If you have ever wondered about things like "gee, why does that sp guy never update his site anymore" I have some good news. I was busy being a grad student. At least until 27 minutes ago when I finished the presentation of my thesis and answered the last pesky question I was asked about it by a professor.

Being a grad student turned out to be way more demanding than being an undergrad. Less time to slack off led to less site updates. A shocking concept, I know.

Anyway, unless some kind of nightmare happens like

Administration Guy: Hey sp, what was your second elective?
sp: Erm, a second elective?

I will soon receive a nifty little Master of Science (in Computer Science) diploma. It will be my second most important formal proof of qualification, topped only by my beloved Windows 95 Power User certificate I received from Brainbench like a decade ago (it looks like this but I don't have a pic of my own one here).

Well, that gave me an excuse to write a site update. Time to slack off a bit now. Between moving out of my student appartment, doing a lot of administrative stuff at college, visiting a bunch of people (potentially) for the last time, and finding a job, I probably won't make another site update in the next 2-3 weeks. But after I'm done with all of that, my goal is to write more site updates again.

The Art of Software Security Assessment - Identifying and Preventing Software Vulnerabilities (Amazon / Official Website) by Mark Dowd, John McDonald and Justin Schuh is a huge book. At more than 1100 pages it's the fourth biggest book I've ever read. It was quite a task to read it front to back and so it took me a while. Here's my review of the book.

Continue reading "Book review: The Art of Software Security Assessment"

Version 0.04 of RubLib, a high-level API for writing IDA Pro scripts in Ruby, is here. It grew from 125 methods to 163 methods since version 0.03. The most important new features are:

• Support for function chunks
• The Instruction class was restructured a bit
• The behaviour of the [ ]-operator of the RubLib classes was standardized. It behaves like the [ ]-operator of the standard String class now.

Download RubLib 0.04
Online manual, examples, API documentation

Two days ago I was sure that I would never release another version of FCEUXD SP (everybody's favourite NES emulator for hacking NES games). Today there's a new version because I was unhappy with the Cheat Console. Weird things happen sometimes.

Version 1.07 is another small update. It adds the so called RAM Filter. With this tool you set up rules to specify how you expect RAM values to change. After you set up the rules, RAM Filter eliminates all bytes in memory that changed differently. This makes finding data in NES games very easy. How is that different from the Cheat Console? The RAM Filter offers more options, it's easier to use, you get more information about value changes, and you can easily go back to earlier rules without having to restart everything.

Let me give you two examples.

In the first example we want to find out where the game Faxanadu stores the player's health points in RAM. Check this screenshot to see how I used the RAM Filter tool.

At first I started a game and used the "Any" rule. This rule tells the RAM Filter that I don't have any information about the value I want to find. Right after that I walked to another screen. There I set up two "Equal to last value" rules because the health points didn't change since the first rule. These two rules eliminated lots of potential memory locations that had nothing to do with the health points. Then I went to the next room and walked into a monster. This obviously changed the number of health points and I set up a "Not equal to last value" rule accordingly. Then I went to a store and set up two more "Equal to last value" rules to eliminate more potential memory locations. Afterwards I bought some health points from the vendor. Once again this changes the number of health points so I set up another "Not equal to last value" rule.

After all this, the RAM Filter tool found that only 2 memory locations changed according to the rules I set up: 0x399 and 0x431. A quick test reveals that the health points are stored in 0x431.

Here's another example. Suppose we want to find out where the number of player lives left are stored in Super Mario Bros. Check this screenshot to see the results

I started the game and once again I set up an "Any" rule at first because I didn't know what to look for. Immediately after I set up the first rule I added two "Equal to last value" rules because the number of lifes left didn't change. This helped to eliminate lots of memory locations already. Then I walked into the first Goomba to lose a life. I expected the number of lifes to decrease therefore I set up a "Less than last value" rule. After the game was restarted I set up another "Equal to last value" rule.

RAM Filter worked even better this time. Only one memory location was left. Now we know that the number of lives left is stored in 0x75A.

Both of these examples took less than one minute to figure out. That's pretty neat in my opinion.

Download FCEUXD SP 1.07
Download the FCEUXD SP 1.07 source code

Here comes RubLib 0.03, everybody's favourite high-level Ruby library for writing IDA scripts. RubLib grew from 88 methods to 125 methods since version 0.02. The most significant addition is probably that users have access to the sections of a file now. Lots of other smaller things were added too.

I've even begun to write an actual manual for RubLib. Here it is. Make sure to check out the examples section. The entire RubLib API documentation can now be accessed online too (check the end of the manual).

Download RubLib 0.03