» MD5 algorithm
With the use of MD5, we can easily create an 128-bit "fingerprint" (or "message digest") of a string or file. By comparing this computed value with a 'known good' MD5 hash, we can be sure (for 99.9%) the compared file is een legitimate file.
Rootkit Hunter scans the most important files (with the greatest change of being replaced with a trojaned one). In most cases this files are also our utilities to scan for rootkits. When they are trojaned, they are immediately useless for our investigation and cannot be trusted. By comparing this files first with a MD5 check, we can eliminate this problem in most cases. However, some Linux distributions (and BSD distributions as well) have multiple legitimate files with a different MD5 hash. A possible false negative can the result of this!
» Default files
By scanning a lot of 'default' files and directories (used by rootkits) we can intercept some rootkits quickly and easily. The advantage of this scan is the little time we need to perform this action. A major disadvantage is the recognition of the rootkit, because a lot of rootkits uses parts of eachother.
» Hidden files
Although 'hidden' files can be usefull, sometimes they are an unwanted part of the system. By scanning for hidden files on places where they are not supposed to be (like in /tmp), we can track down some possible evil files.
» Operating system specific tests
Every operating system has is advantages and disadvantages like the differences between tools en disk structure. Some parts of an operating system are not available to others, so we can use not all tests every time.
Linux
- compare processes in `ps` against the available files in /proc
FreeBSD
- look for differences between the output of `netstat` and `sockstat`
- compare known (evil) strings to loaded KLD's
- scanning for promiscuous interfaces
» File permissions
All system tools at a clean system have 'normal' permissions, so a normal user can't delete this important files (imagine yourself someone delete 'ls'...). A lot of trojaned files have 'wrong' permissions, lets say full read, write and execute permissions (chmod 777). By searching for this unusual file permissions, we can sometimes easily determine a rootkit has been installed.
» Kernel modules
Operating systems like Linux and FreeBSD support the use of 'kernel modules'. These modules give an administrator the power to load and unload modules (on the fly) to extend the possibilities of the kernel. For example, creating a firewall by loading a single module and some firewallrules is one of the possibilities. Linux distros uses the abbreviation 'LKM', which stands for 'Loadable Kernel Module'. All BSD's (like FreeBSD, NetBSD and OpenBSD) use 'KLD' (Dynamic Kernel Linker)
» Listening ports
Some rootkits do listen to a static port for incoming connections. Most of the time these listening services are known as a 'backdoor shell'. By checking these known ports, we can grab some rootkits, although most rootkits are smart enough to use a dynamic port.
» String scanner
By scanning one or more directories with a 'string scanner', it's an easy job to catch some nasty files. The scanner scans plaintext and binary files on the presence of strings which are used in this evil files. The scanner will use a dictonary to compare the scanned files with known trojans, sniffers, backdoors etc.