[lotuseclat79]

Well it's finally happened - a friend has some sort of virus or backdoor sniffer on his Linux Redhat 8 server. As of about a month ago, he can no longer modify /bin or /usr/bin, even as root. Furthermore, "ps -fu <username>" no longer works (he gets a usage error), and an obscene message about a signal 11, when he tries to shut down. The obscene message can be found in a string contained in:

/usr/lib/nmh/include/lib/.sniffer

so presumably that's an evil packet sniffer that is getting a signal 11 when he tries to shut down.

The incident almost certainly happened on March 19 at 19:00, since find shows the following set of files modified or created at that time. Note especially the modification of /sbin/init and /etc/rc.d/init.d/functions, and the creation of a file libice.log at that time:

--------------------------------------------------------------------------------
find / -mount -print0 | xargs -0 -n 99 /bin.new/ls -ld | grep 'Mar 19 19:'
drwxr-xr-x 3 root root 32768 Mar 19 19:03 /dev/ida
drwxr-xr-x 2 root root 4096 Mar 19 19:03 /dev/ida/.hpd
-rwxr-xr-x 1 root root 10097 Mar 19 19:02
/etc/rc.d/init.d/functions
drwxr-xr-x 2 root root 61440 Mar 19 19:03 /usr/bin
-rwxr-xr-x 1 root root 15961 Mar 19 19:02 /usr/bin/(swapd)
-rw-r--r-- 1 root root 5 Mar 19 19:02 /usr/bin/x.pid
drwxr-xr-x 135 root root 65536 Mar 19 19:02 /usr/lib
drwxr-xr-x 3 root root 4096 Mar 19 19:00 /usr/lib/nmh
drwxr-xr-x 3 root root 4096 Mar 19 19:00 /usr/lib/nmh/include
drwxr-xr-x 2 root root 4096 Mar 19 19:05 /usr/lib/nmh/include/lib
-rw--w--w- 1 root root 1 Mar 19 19:00
/usr/lib/nmh/include/lib/.sniffer
-rwxr-xr-x 1 root root 28152 Mar 19 19:00
/usr/lib/nmh/include/lib/sk
-rw-r--r-- 1 root root 11 Mar 19 19:03 /usr/lib/libice.log
drwxr-xr-x 2 root root 4096 Mar 19 19:01 /usr/lib/libshtift
drwxr-xr-x 192 root root 12288 Mar 19 19:02 /usr/include
-rw-r--r-- 1 root root 5 Mar 19 19:02 /usr/include/linpid.h
-rw------- 1 root root 512 Mar 19 19:02 /usr/include/linseed.h
drwxr-xr-x 2 root root 4096 Mar 19 19:01 /bin
drwxr-xr-x 2 root root 8192 Mar 19 19:00 /sbin
-rwxr-xr-x 1 root root 28152 Mar 19 19:00 /sbin/init
-rwxr-xr-x 1 root root 33960 Mar 19 19:00 /sbin/initlib
--------------------------------------------------------------------------------

Googling shows that there is a buffer overflow exploit for nmh, but he's not sure he's ever used that to read mail (well maybe once). There are also exploits for imapdand he does run that through the firewall (ssl connection only).


More info, in case it helps anyone else recognize or avoid this. The bad guys modified /bin/ps, /bin/ls and /bin/netstat (he can tell because they have a newer date than everything else in /bin that was installed with Redhat 8).

He got a copy of an unmodified ps, ls and netstat. One thing he immediately found was that the bad copy of ps hides all occurrences of sshd, presumably so you won't know when someone logs into your system. Unfortunately, it's too stupid to hide only the unauthorized sshd's, so it even hides the ones that he knows should be there, because he's logged in from his Windows box. The bad ps also hides all instances of smbd, don't know why.

The bad netstat is currently hiding that fact that something is listening on tcp ports 1028 and 8080. He's got his Apache configured to listen on 8080, so that he can get through RCN's block; again he doesn't know why the hacker wants to hide this. Port 1028 is undoubtedly being listened to so that the hacker can connect from outside, but he doesn't have that port open in his firewall. He *does* have port 8080 open, of course, but Apache is listening on it (or maybe not).

He hasn't figured out what the evil version of ls hides, and maybe it's not working right, because he sure sees a lot of suspicious files with it, as shown above.

Still searching the web for the name of this malware and a cure.

Another friend got curious, Googled around and found these:

http://www.linuxquestions.org/questions/history/309040

http://honeynet.streetchemist.com/sc.../dvaartjes.txt

Apparently those responsible changed a lot of his utility programs. He will probably have to re-install.

Someone is behaving badly, but it is hard to tell who. One URL above suggests the hacker is in the Netherlands, the other suggests Romania. Probably it's neither; the bad guys are likely just hiding behind other machines they have compromised.

The first URL talks about how fast the hacker tried to cut out the machine's owner when the owner tried to protect his machine. Either the machine owner was just unlucky and being watched by a hacker at that precise moment or there is an organization or government doing these things which has the manpower to monitor its "private network". Also, there is obviously a LOT of effort involved on the part of the attackers to create alternate versions of so many programs. If one had to bet it's not a lone hacker....

[lotuseclat79]

Since several people have asked how how my friend thinks he got this attack (which now appears to be the "Suckit" rootkit), and what measures he took to protect the system, here's a summary:

1) The system was only protected by a Linksys firewall/router between the internet and it. He have no reason to believe that this protection wasn't working as planned, but note that he had several ports open, and forwarded to the Linux system, as described below. There was no anti-virus software running on it, and he always ignored the incomprehensible, voluminous, and completely wrong output that was mailed to him every morning by tripwire, which he could no longer remember how to turn off or configure. :-/

2) It's an unpatched Linux RedHat 8 system. By the time he thought of getting updates for it, RedHat had dropped all support for it. And the thought of installing a whole new Linux system, and transferring all the dozens of installation files for the various servers he runs (and worrying about compatibility of config files across versions) was just too exhausting. He *did* take regular backups of /etc and /var, which is partly how I could
detected the modifications made to his system by the rootkit.

3) It runs an smtp server (sendmail) open on ports 2525 and port 25 (but RCN blocks port 25 now).

4) It runs Apache, listening on port 80 and port 8080 (but RCN blocks port 80).

5) It runs the U.Washington IMAP server, listening only both the secure and insecure port (993), but only the ssl port was open through the firewall.

6) The unmodified RedHat ssh server listens on port 22, also forwarded through the firewall.

Summary, don't believe the hype your Unix-weenie friends give you about running Unix to be safe from Internet attacks and viruses. That might be true if you run a system no one cares about, but it is certainly not true if you run popular, well-known servers on a Linux system, like Apache, imapd and sshd. Apache is the most widely used Web server in the world, so naturally there are constant attempts to break into system using it, via buffer overflow attacks and so forth. (And Linux is no better than Windows at avoiding the occurrence of, or protecting against the exploitation of buffer-overlow attacks.)

Oh well, live and learn. After repairing this system as best he could, he is going to start researching a replacement Linux system. Does anyone have any suggestions as to the most trouble free (easiest to configure) and well-supported (regular security updates) version? Should he go with a commercial version, or Fedora?

[tdi_veedub]

I can't agree with you more.

I am currently running httpd, sshd, webmin, usermin, ftp, and smtp (outgoing only) on a slack 10 box.

sshd does not allow root logins
I also have it setup behind my dlink router with only the necessary ports forwarded (80, and 22). All the other servers are accessible to the LAN only (21, 10000, and 20000). ftp is tunneled through sshd.

I also have iptables setup on the linux box to automatically drop all internet requests for anything other than 80 and 22, AND to drop any forwarded and outgoing request unless explicitly implied. iptables also blocks any connections from ANY asian network, as well as any incoming connection for 10000, 21, 20000 that are not on my local network.

Right now I check logs daily, and manually add ips that I see running sshd bruteforce attempts. I want to write a script to do this automatically, but don't have time ...

Seems to be working for me right now, but my server doesn't get much traffic.

I guess the rule is, keep everything patched, firewalled and keep as little open as possible. Everytime a new stable release of slack is out, I take the server down and install it. /home is on a different drive and is backed up daily so that makes it easy.