NWA-PCUG Newsletter Article, December 2008
Rootkits - A Continuing Security Problem
by Brian K. Lewis, Ph.D.,
Member of the Sarasota Personal Computer Users Group, Inc.,

http://www.spcug.org
bwsail at yahoo.com (click to email author)

Obtained from APCUG with the author's permission for publication by APCUG member groups.

By now I suspect everyone reading this article is familiar with most malware: viruses, botnets, Trojans, etc. These are becoming less of a problem because of the efforts of the security companies to provide software solutions. More and more users are also becoming aware of the need to have some means of protecting their computer. As a result, hackers are turning to a more effective method of controlling your computer – rootkits. Although these have been around more than ten years, like other malware, their numbers seem to be increasing.

Probably the most dangerous form of the rootkit is the "kernel mode Trojan". This is a program that inserts itself into the "kernel" of the operating system. The kernel is the central component of the operating system – its heart or brain to put it in more common terms. It manages the communication between the operating system, the hardware and the software applications.

Most viruses operate as applications and can be readily found in memory or in the file system. Rootkits, however, can hide themselves in such a way that it is very difficult to find them. In order for a rootkit to alter the normal execution path of the operating system, one of the techniques it may employ is "hooking". In modern operating systems, there are many places to hook because the system was designed to be flexible, extendable, and backward compatible. For example, a rootkit can "hook" itself into the Application Programming Interface (API) which allows it to intercept the system calls that other programs use to perform basic functions, like accessing files on the computer’s hard drive. If an application tries to list the contents of a directory containing one of the root kit’s files, the rootkit will censor its filename from the list. It’ll do the same thing with the system registry and the list of running processes.

A rootkit is a collection of tools an intruder brings along to a victim computer after gaining initial access. A rootkit may contain network sniffers, log-cleaning scripts, key-loggers and trojaned replacements of core system utilities. Although the intruders still need to break into a victim system before they can install their rootkits, the ease-of-use and the amount of destruction they cause make rootkits a considerable threat. One main purpose of a rootkit is to allow the intruder to come back to the compromised system later and access it without being detected. A rootkit makes this very easy by installing a remote-access backdoor. A rootkit can also allow the intruder to use the compromised computer as part of a botnet (see Botnets, SPCUG Monitor, January, 2008).

Another mechanism for hiding a rootkit is to add it to a system driver file. Windows XP and Vista store driver files in the System32/drivers folder. Many of these system files load early in the boot process. These files have boot or system flags in the registry and load before any of the malware-prevention software. That means they are very difficult to find. Although the file size for the driver will be increased, the rootkit may report the original file size to any query, not the infected file size. All of this means that once a rootkit has been installed and activated on your computer, it is difficult to find by any of the usual malware prevention software.

Rootkits do not require large software applications to carry out their function. We are accustomed to commercial applications that are many megabytes in size. Even the anti-virus software may be 40-50 megabytes in size. In 2003 a rootkit was identified that required only 7 kilobytes for its cloaking routine and 27 kilobytes for maintaining the open backdoor.

Anti-malware programs depend on two main means of identifying malware. One is the signature method and the other is heuristics. The signature method requires that the malware be identified and reverse engineered to determine a code sequence which can be used to identify the application in the wild. This code sequence is referred to as the signature and is used by the anti-virus database. This signature is then compared to code sequences in applications to determine if they are malware. This method is of no value when dealing with new or unreported malware.

So the next option is heuristic signatures. Their primary advantage lies in their ability to identify new, previously unidentified malware. The heuristics technique assumes that malware will display certain characteristics or attributes. They also attempt to recognize deviations in "normal" system patterns or behaviors. Using these predicted patterns, the anti-malware application will attempt to determine if the target application is malware. This has been a successful approach for identifying viruses, but it is less successful for active rootkits.

The April 2008 Virus Bulletin (www.virusbtn.com) reported the results of testing a number of popular commercial A-V programs, Internet security suites, web-based scanners and specialized anti-rootkit tools. The testing involved 30 known rootkits. The testing categories were detection of: (1) inactive rootkits; (2) active rootkits; and (3) malware hidden by rootkits. Then they tested removal of (1) inactive rootkits; (2) malware hidden by rootkits; and (3) active rootkits. The results were not encouraging.

The seven Internet Security Suites used in the test were able to detect 95% of the inactive rootkits. (Remember, these were known samples that had already been identified and their signatures incorporated into the anti-malware applications.) These suites were also able to remove 95% of the inactive rootkits. However, when it came to active rootkits the story was very different. The Internet Security Suites detected only 65% of the active rootkits and were able to remove only 48%. They also were able to remove only 48% of the hidden malware. All of the versions of the Internet Security Suites were the latest available at the time of the test.

There were fourteen specialized anti-rootkit tools tested using the same thirty rootkits. They were not tested against the inactive rootkits, only the active rootkits and the hidden malware. Again, the results were anything but satisfying. These tools detected 83% of the active rootkits and 80% of the hidden malware. The anti-rootkit tools removed only 60% of the active rootkits and 67% of the hidden malware.

The web-based scanners did a far poorer job of identification of the rootkits. They also were uniformly unsuccessful in removing rootkits. The detection rate was 53% and the removal was around 32%.

In reviewing these tests it is obvious that successful detection and removal of rootkits depends on their being inactivated. This can be done by running the computer in "SAFE" mode which does not allow the rootkit to load from the hard drive. However, it would be expected that if detection/removal tools were developed for this specific purpose, then rootkits would appear that would load in "SAFE" mode. Another alternative would be to develop rootkit scanning software that would run from a CD. The computer would boot from the CD and the operating system for the scan would load from the CD. This should improve the detection and removal rates considerably. However, it then depends on the user running the CD application periodically to scan the entire computer. Considering how few users backup their hard drives on a regular basis, this CD system might be less than universally successful.

Given the current difficulty of detecting and removing rootkits from your computer, what is a user to do for protection? The only answer to this is to prevent the rootkit from getting access to your computer. That means using every tool you have available to prevent the malware from gaining access to your system. Your firewall is the first line of defense, followed by your anti-virus, then your anti-spyware. Also, when you are surfing the web, make sure you aren’t your own worst enemy. Be careful and check out links before you click on them. It just like getting spam in your e-mail. Check where the link will take you before you click on it. Social engineering techniques are also used to propagate everything from viruses to rootkits. These are techniques that encourage the user to take some action which allows the malware to be downloaded and installed on the users computer. A very interesting analysis on these techniques is contained in this article from the University of Cambridge (U.K.); http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-666.pdf. Although this is written specifically about virus propagation, similar techniques are used to gain entry for rootkits. This paper illustrates many of the "carrot & stick" methods used by malware to gain access to computer systems. Microsoft has also published a paper detailing many of the common methods used to trick users into installing malware. These can be found in the paper "Behavioral Modeling of Social Engineering-Based Malicious Software" on the Microsoft web site.

So to all of you reading this paper, I would suggest that "caution is the watchword" when it comes to using your computer. I’m afraid that the situation will only get worse when it comes to new forms of malware.

Update Note: In my article on iFrame attacks (SPCUG Monitor, May 2008), I listed a number of portals that had been affected by iFrame attacks. One of these was the eHawaii.gov portal. I have received information from the site manager that the problem has been corrected (removal of the iFrame) and actually only affected one page on their site. Thanks to Russell Castagnaro for correcting this problem and notifying me.

Dr. Lewis is a former university and medical school professor of physiology. He has been working with personal computers for over thirty years, developing software and assembling systems.

This article has been provided to APCUG by the author solely for publication by APCUG member groups. All other uses require the permission of the author (see e-mail address above).

Click here to return to top



==================================================================