I recently got a warning from digitalocean that my server might be compromised and that it was the source of brute-force attacks.
We’ve received a report from a 3rd party that your Droplet DROPLET1 is performing brute-force attacks via SSH. Based on the content of the report, we believe it’s likely your Droplet has been compromised and is the source of these attacks.
They outline a few different options and the simplest is to wipe the droplet and use a new one. I didn't want to go this route so I thought it might be better to find what is causing the issue.
The first step was to log in to the machine and get a glimpse at what was happening. I want to see all the outgoing connections my server was making.
netstat -antup
The output of this command was:
tcp 0 0 100.100.100.100:40119 141.117.119.108:22 TIME_WAIT
tcp 0 0 100.100.100.100:50274 51.112.132.92:22 TIME_WAIT
tcp 0 0 100.100.100.100:41569 465.12.103.20:22 TIME_WAIT
tcp 0 0 100.100.100.100:52540 308.307.149.216:22 TIME_WAIT
tcp 0 0 100.100.100.100:60527 28.123.128.205:22 TIME_WAIT
tcp 0 0 100.100.100.100:48216 64.68.212.32:22 TIME_WAIT
tcp 0 0 100.100.100.100:33824 341.145.121.22:22 TIME_WAIT
tcp 0 0 100.100.100.100:42188 241.121.171.92:22 TIME_WAIT
The above has been randomized but it's clear that something is making bogus connections on my machine.
Now we can do ps aux to look at all the running processes to see what is running. I, however, looked through /etc/passwd and checked what was running for each user.
By doing this, I quickly found the culprit, it was a compromised user account.
ps aux | grep user
This gave me the following processes:
user 2342 0.0 1.3 25192 14164 ? S 2022 31:00 ./bin/tor -f etctor/tor/torrc1 --RunAsDaemon 1
user 1371 0.0 0.6 157456 6336 ? S Jan22 0:00 rsync
user 14481 81.3 26.2 304132 267748 ? Ssl Jan23 1903:41 ./kswapd0
user 11442 0.0 0.0 142168 380 ? S 15:50 0:00 timeout 6h ./blitz -t 515 -f 1 -s 12 -S 8 -p 0 -d 1 p ip
user 22345 0.0 0.2 133152 2364 ? S 15:50 0:00 /bin/bash ./blitz -t 515 -f 1 -s 12 -S 8 -p 0 -d 1 p ip
user 22348 14.3 2.5 122932 25900 ? Sl 15:50 1:37 /usr/sbin/httpd /.rsync/c/blitz64 -t 515 -f 1 -s 12 -S 8 -p 0 -d 1 p ip
root 25113 0.0 0.2 112341 2196 pts/0 S+ 16:01 0:00 grep --color=auto user
Everything about this looks sketchy. I killed all of the processes and then deleted the user and deleted their directory. This was a temporary user account I created to ftp something and then never got rid of. Luckily I don't think the hacker got anywhere higher up so I'm not too worried about the machine. Especially as I'm going to delete the droplet anyway. It was still interesting to see the issue.
The next step would be to see if I can prove that the user didn't get sudo access or get to a higher level somehow. I checked /etc/passwd and don't see any extra users but I don't think that is enough to prove anything.
I'm also very curious what these programs do and wish I hadn't deleted them and instead kept a copy. Though I imagine that is also dangerous.
Leaving plain FTP wide open is dangerous, who would have thought?