Misc

Last Friday I was debugging random programs I found on my hard drive when I saw this:

Apparently µtorrent is sending lots of unrelated data back to the µtorrent servers when checking for program updates. I wanted to know what. Google was not very helpful. A thread in the official forum was all I found and that thread did not exactly have a lot of information. Apparently I had to figure out things myself.

Unfortunately it turned out that µtorrent is sending a lot more data than can be seen in the screenshot so this little project took longer than originally planned. In fact I am only 90% done but I don't want to put any more time into this. One weekend is already too much time spent on this. So, here's what I figured out.

Continue reading "Data sent home by µtorrent"

I'll be on vacation in San Francisco between November 4th and November 17th. If anybody wants to meet up there, please contact me in some way (see the side bar on the right hand side of this page). Possible reasons for hanging out with me include but are not limited to:

• You work for a cool company and feel like showing me around your office.
• You work on a cool product/technology and feel like I should see it.
• You think I work on cool stuff and want to talk about it.
• You want to talk about reverse engineering, binary code analysis, hacking NES games or anything related to that.
• You want to get a BinNavi product pitch.
• You want me to see some special place or event in San Francisco.

A few months ago my friend Rolf Rolles created the Reverse Engineering reddit. It's pretty awesome. It focuses on random reverse engineering related articles and lots and lots of academic papers, mostly for static code analysis. The submission quality is pretty high (or the moderators reject a lot of submissions, I don't really know). So just in case you are not aware of this reddit yet, I suggest you hop over there and subscribe to it.

While we are at it, for those of you stalking me already. I have a Twitter account now. That should make your job easier.

Continue reading "A flurry of Web 2.0"

In a bit less than two weeks I will be attending CanSecWest 2009. If anybody wants to meet me there to talk about reverse engineering, static code analysis, BinNavi, or why Sierpinski triangles are tools of the devil please contact me. You can find my contact information on the right side of this website.

If you do not want to talk about any of the above, you can still watch my talk and tell me your opinion about it. This would be much appreciated. Thank you.

A few days ago I complained about the incredibly awkward IT Security Girl of the Year award that will be dished out later this year at the French IT security conference FRHACK. Apparently the FRHACK organizers did not like what I wrote because they are now threatening to sue me if I do not remove the screenshot of the incredibly inappropriate photos they used to advertise the IT Sec Girl award. The following sentence is from an email I received from one of the FRHACK organizers.

We just ask you to remove this picture http://www.the-interweb.com/bdump/misc/itsec_girl.jpg because these pictures are copyrighted. Thanks to delete your screenshot or we will take legal action.

After I received this email I was incredibly annoyed. It's less about them requesting me to take down the screenshot though. I was more annoyed with their reply in general. In no way did they even bother to address the issue I complained about. This would have been the perfect time to show some character. They could have discussed their point of view. They could have apologized for their mistakes. They could have told me to stuff it (in case they think their point of view is correct; which apparently they don't because they changed their website). All of these (and more) would have been perfect ways to show some personal responsibility and the whole issue would be finished from my point of view.

But that's not what happened. Instead they sent me this incredibly limp-dicked legal threat which I believe to be more about removing evidence that documents their failures than about any legit copyright issues. The passive-aggressive vibes I am getting from this (and the fact that I am allergic to legal threats) piss me off so much that I need to complain about this publicly. I hope I will never have the displeasure to meet any of the FRHACK organizers in real life.

Now the thing is, I do not have the time or the resources to fight their legal threat. I will take down the screenshot in the next few days (probably at one point between now and next Wednesday; whenever I feel like it).

Update: Looks like the screenshot stays on my website. See the end of the original post for a detailed update.

Update 2: Please check out the comments made by the FRHACK team in response to this post. This should clear some things up.

Today I am going to break with the spirit of this website. This entry is not about Programming Stuff as advertised in the title of my blog. Today I am going to complain about something which is only very casually related to Programming Stuff. In some way it is kind of similar to when CmdrTaco used the Slashdot front page to complain about Blizzard making him change his WoW name. It's my website and I post what I want. So if you came here expecting new Programming Stuff you can hit the Back button of your browser now.

Anyway, let's start. I woke up this morning and saw that someone I know will attend the French IT Security conference FRHACK. I began to browse the FRHACK website. I checked out the Events section. I came across something called Best IT Security Girl of the Year. For a split-second I was amused about this obvious little satirical reflection about the role of women in IT and especially IT Sec. I quickly scrolled around to find the IT Security Boy of the Year. There was no such award. It dawned on me that the IT Sec Girl of the Year award is actually a serious award and I started to feel sick.

Continue reading "IT Security Girl of the Year"

I am going to attend the Chaos Communication Congress 2008 in two weeks. People who want to meet me there to talk about BinNavi, Hexer, static code analysis, reverse engineering in general, or why the movie Hackers is more realistic than most people think, please contact me using one of the options you can find on the navigation bar on the right side of this page.

Here's something amusing. I spent the first half of the day writing a short Haskell program which generates x86 instructions in MASM syntax. The program generates all variants of the non-privileged instructions from the opcodes.chm file of the MASM32 package. This means that the instruction generator is not complete at all. FPU, MMX, SSE and other newer-than-x486 instructions are not covered. Nevertheless the generator already generates nearly 150,000 different x86 instructions.

When assembled with MASM32 the resulting file is more than 600 KB big. Trying to disassemble this thing with a few standard disassemblers turns out to be a problem. IDA fails to disassemble an instruction after maybe 5% of the executable and never manages to recover afterwards. Lots of manual help is necessary to convince IDA to go on. OllyDBG manages to disassemble that instruction but has huge gaps at many, many other points of the disassembly. The created file is an interesting test file for x86 disassemblers I'd say.

The Haskell program is just about 300 lines long. 280 of those lines are the definitions of  the instructions and what operands they can take. The generation of the instructions from the instruction definitions is just 20 lines and all but 8 lines are not even strictly necessary. I love Haskell's expressiveness.

Anyway, click here to see the Haskell source or click here to download the whole package including the Haskell program (source + EXE), the generated output of the Haskell program, a MASM32 source file that can be used to assemble the test file, and the test file EXE itself.

Next week I'm going to attend this year's hack.lu conference in Luxembourg. So if anybody else who attends hack.lu wants to talk about the new version of BinNavi, Hexer, static code analysis, reverse engineering in general, or some kind of other topic contact me please (see the right side bar for contact options).

Yesterday I saw a talk given by Frank Boldewin where he mentioned the FreeIconList trick to fool code emulators. At this point I started to wonder what other Win32 API functions are basically unused. Using Ero Carrera's Python library pefile to parse PE files I wrote a small Python script that tries to find out what Win32 API are basically unused.

The modus operandi was simple. I read the exported functions of all DLL files in WindowsDir and WindowsDir/system32 and compared them to the functions imported by all EXE/DLL files in WindowsDir, WindowsDir/system32 and my entire Program Files directory.

The first result is that most exported functions are apparently basically never used. My script managed to find 127569 exported functions in 1225 DLL files. 104608 of those are never used by the 6615 EXE/DLL files which import functions ("used" is liberally defined as "imported through the import directory" here, of course). That leaves 22961 functions which are actually used.

Here are some output files which show the exported DLL functions sorted by their usage. The numeric column contains the number of PE files which import the function statically. That means that 3475 of the 6615 files I tested import GetLastError for example.

• Click here to see the Top 2000 most used API functions
• Click here to see the usage statistics of all advapi32.dll functions
• Click here to see the usage statistics of all gdi32.dll functions
• Click here to see the usage statistics of all kernel32.dll functions
• Click here to see the usage statistics of all msvcrt.dll functions
• Click here to see the usage statistics of all ole32.dll functions
• Click here to see the usage statistics of all oleaut32.dll functions
• Click here to see the usage statistics of all shell32.dll functions
• Click here to see the usage statistics of all user32.dll functions

Random notes

• kernel32.dll is surprisingly dominant while gdi32.dll is surprisingly "unused"
• pefile is extremely awesome and easy to use
• Don't be confused that API functions like lstrlen are imported 0 times, check lstrlenA and lstrlenW

Click here to download the Python script.

Unless something goes spectacularly wrong I'll be in Montreal from next Thursday to Sunday to attend RECON 2008. If you wanna do one or more of the following things, make sure to meet me at some point:

• Talk/Complain about BinNavi
• Talk/Complain about BinDiff
• Talk/Complain about VXClass
• Talk/Complain about Hexer
• Talk/Complain about Zynamics
• Talk/Complain about any other topic
• Get an awesome VXClass t-shirt
• Have some poutine with me

I'll be the guy with the yellow Sabre Security bag. Alternatively you can shoot me an email or ask around.

Two days ago I had the pleasant experience to participate in some kind of informal reverse engineering session with three other guys. Between dinner and way too long after midnight we debugged a popular piece of malware that is floating around the internet right now. The first guy already reverse-engineered an earlier version of the malware. He was the guy in charge who did most of the debugging. The second guy was the author of a program that monitors and logs the behaviour of processes, especially malware processes. The goal of the session was to find out why the malware sample worked perfectly in VMWare (after we patched out the VMWare check, at least) but crashed as soon as second guy's monitoring tool was active. The third guy was very familiar with the malware too but on a higher level (behaviour, network activity, how it spreads, its historic development and usage, ...). I was the fourth guy. Without a direct interest in the malware or the malware monitoring tool I just wanted to see what goes wrong. Furthermore I was the guy for snarky comments from the background like "see, I told you take a VMWare snapshot before stepping over that call".


Continue reading "Pair Reverse Engineering"

Over the last days some group released passwords to nearly 45000 MySpace accounts and they announced to release another 30000 passwords in the next few days. I used a few hours before Saturday's lunch to write a small program that analyzes the passwords that were released so far.

At worst the results of this are a useless time-filler, at best it's a case study of what happens if a website forces their users to choose passwords with certain minimum requirements. MySpace demands that every password contains at least one non-alphabetical character (like 0, 1, 2, or !, ?, @). How the users adhered to this requirement can be seen in the tables below.

It is my understanding that the 43713 passwords that were leaked so far come from fishing sites that trapped people to enter their password. This makes the passwords less reliable than a password list hacked straight from the MySpace servers. People could have misspelled their MySpace passwords or they could have entered fake information after they noticed that someone was trying to steal their password. A quick analysis has shown that probably less than 1% of the leaked passwords suffer from these problems. Continue reading "A brief analysis of 40,000 leaked MySpace passwords"

If you have ever wondered about things like "gee, why does that sp guy never update his site anymore" I have some good news. I was busy being a grad student. At least until 27 minutes ago when I finished the presentation of my thesis and answered the last pesky question I was asked about it by a professor.

Being a grad student turned out to be way more demanding than being an undergrad. Less time to slack off led to less site updates. A shocking concept, I know.

Anyway, unless some kind of nightmare happens like

Administration Guy: Hey sp, what was your second elective?
sp: Erm, a second elective?

I will soon receive a nifty little Master of Science (in Computer Science) diploma. It will be my second most important formal proof of qualification, topped only by my beloved Windows 95 Power User certificate I received from Brainbench like a decade ago (it looks like this but I don't have a pic of my own one here).

Well, that gave me an excuse to write a site update. Time to slack off a bit now. Between moving out of my student appartment, doing a lot of administrative stuff at college, visiting a bunch of people (potentially) for the last time, and finding a job, I probably won't make another site update in the next 2-3 weeks. But after I'm done with all of that, my goal is to write more site updates again.

Here are some more details about the program I used to create some graphs from Wikipedia two days ago. The C# source code is now available. The program takes five command-line parameters.

• The name of the Wikipedia XML input file. You can download it here. It's the 1.8 GB pages-articles file.
• A number that specifies how many nodes you want in the final graph. 25 - 50 are reasonable values.
• The keyword to search for (like Egypt for example). Note that this parameter is case-sensitive.
• A number that specifies how many sentences are considered when searching articles for the keyword.
• The name of the output file. This file is a graph definition file that can be turned into a graph using dot.exe from the GraphViz package (like dot.exe -Tsvg output.txt > graph.svg for example).

I toyed around with the parameters for a while and here are a few more things I noticed:

• Searching through the first three sentences of all articles seems to produce very nice results for most keywords.
• If the keyword is relatively rare (for example "Australian rules football") it's OK to search through the entire article (set the sentences parameter to 0). Don't do this for popular keywords though or you'll end up with a graph that shows articles that are objectively important but only tangentially related to the keyword. If you do a full-article search for "Germany" for example you end up with a graph full of nodes containing the names of other European countries that played a role in Germany's history. That's because articles about countries have a high importance and all European countries were somehow important to Germany in the last few thousand years.
• Trying to use the size of articles as an indicator of their relative importance didn't work out. Look at this 4,000 words treatise on the Goomba and compare it to the page for Niels Bohr which is only half as long. This should give you a first idea about the potential problems. There's still legacy code in the app from where I tried that idea. It's easy to enable again it but you need to recompile the app.
• Trying to use the position of the keyword compared to the size of the article didn't work either. Long articles still have too much weight. A better idea might be to give an article weight 10 if the keyword appeared in the first 10% of the article, weight 9 if it appeared in the next 10%, and so on. I didn't try that though.
• Different parameters lead to different meanings of the resulting graphs. If you do a 3-sentences search for LSD the graph shows information about the drug itself and its history. If you do a full-text search one half of the graph is dominated by rock stars.

Here are a few other graphs which turned out particularly well:

• Chile: PNG - SVG
• Nintendo: PNG - SVG
• Ferrari: PNG - SVG
• Star Trek: PNG - SVG
• Intelligent Design: PNG - SVG
• Australian Rules Football: PNG - SVG

The key to cool graphs is to choose a keyword that has lots of articles which nevertheless belong closely together. An example for a bad keyword is "Mathematics". There are thousands of math-related articles in Wikipedia but they don't belong closely together because math is a huge and fragmented field. The resulting graphs of keywords like math degenerate into trees or unconnected subgraphs.

Generating a graph takes approximately 5 minutes on my computer. In most cases nearly all the time is spent on parsing the 8 GB XML file. Generating the actual graph is nearly always a matter of seconds. Only for keywords like Germany or America which have some ten-thousand relevant articles generating the graph takes a few more minutes.