I was looking at the logs on a couple of my servers the other day. One of them was experiencing about 600,000 attempts per month to guess passwords. Oddly enough, more than 630,000 attempts in a two month period came from four IP address within one block in China, 116.31.116.0/24. The number of attempts is somewhat imprecise because often the logs will show something like "last message repeated 2 times".

I have one computer that I'm waiting for a new hard drive (should be here Friday). While waiting, I've been trying out the latest release of OpenBSD 6.1 on the system.

Yesterday, I downloaded a couple of zone files containing the vast majority of assigned ip address blocks, one for IPv4 and one for IPv6 for the United States. I then configured my firewall to reject all attempts to connect to ssh and telnet on that machine from IP addresses outside of the IPv4 and IPv6 blocks of addresses in the two zone files. The new rule went live a little less than 21 hours ago,

Since then, using those rules, only 2 addresses have connected to try the machine for a total of about 5 times between them. More than 1800 separate attempts have been rejected by the filter.

Also, I have my computer configured to download the latest zone tables for the US every day and update the files based on that by using wget:

wget -O us4_table http://www.ipdeny.com/ipblocks/data/aggregated/usa-aggregated.zone
wget -O us6_table http://www.ipdeny.com/ipv6/ipaddresses/blocks/us.zone

The one problem with that approach, is that the zone files I'm using skip smaller blocks. It's possible that someone from a smaller block not in that zone file might need access to our computers. So what I'm planning on doing is loading all the zone files except the US zone file and rejecting attempts to connect to ssh or telnet from any of the foreign addresses.

There is a minor problem with rejecting from any of the foreign addresses because of the size of the table. On OpenBSD, the maximum size of a table used for that purpose is 100,000 entries and the IPv4 file is about 150,000 entries.

Since two adjacent blocks can be in separate countries, then they both show up in the list. I think that if I write a program that combines adjacent blocks, when possible, it might reduce the number of entries for the table sufficiently to fit. Another possibiity would b to replace all the entries in each /8 block used by APNIC, AFRINIC, RIPE NCC, and LACNIC with just the /8 for that block. Interestingly enough, there are a few blocks listed as being US IP addresses that are in non-ARIN blocks!

For example, 101.53.160.0/19 is in an APNIC (Asia Packfic NIC) block but comes back to Salesforce.com out of San Francisco.

Similarly, 103.11.64.0/22 is in an APNIC block but comes back to HostUS in Delaware.

Another approach might be to use an adaptive approach and not block anyone at the start. Instead, whenever it sees an attempt to connect, it would check to see if the IP address is in a foreign block and if it is, add the block to the table. There are undoubtedly many blocks that we would never see an attempt. Since OpenBSD lets you pass the syslog informaton to a separate program, it's not that difficult to use a separate program to detect attempts to log on, check the IP address against the foreign blocks, and if it finds them, to add the entire block to the table. Unless they make a number of connection attempts all at once, as some do, they would be blocked by the time they make a second attempt.

A couple of other things.

First, I'm not really worried about someone guessing the passwords on those machines. While the test machine is about to become a mail server with a lot of accounts on it, once the new hard drive arrives, only select accounts will be permitted to log in. All the others will refuse login attempts even if they had the passwords.

Second, I have a choice of returning a failure notice or dropping it when rejecting a connection. If I return the notice, then whoever it is knows that it was refused immediately. If I drop it, then normally it takes a while for the attempt to timeout. I like to think that maybe I'm slowing them down just a little by reducing the number of connections they can make to other computers while waiting for it the connection here to time out.

Any way to add a rule to auto block an ip block after say 5 unsuccessful attempts? You can always remove a block if someone screws up and gets themselves blocked.

FatZilla said:

---

Any way to add a rule to auto block an ip block after say 5 unsuccessful attempts? You can always remove a block if someone screws up and gets themselves blocked.

---

There is a method to rate limit connections for any port number. I've tried it before, but it really didn't help that much.

One thing I've found by going through the logs is that I'll see one or two attempts to connect from some address and then they disappear. But if I search for that address in the auth.log files, there may be hundreds or thousands of tries from the same address, but no more than one or two within half an hour of each other. So rate limiting woiuldn't help with those at all.

Hopefully you don't have a telnet server facing the internet?

Have you tried Fail2Ban? Someone makes X attempts to any configured service, they are banned for Y minutes or forever, whatever you setup. Works great.

FatZilla said:

---

Any way to add a rule to auto block an ip block after say 5 unsuccessful attempts? You can always remove a block if someone screws up and gets themselves blocked.

---

Yes, Fail2Ban

Run fail2ban and don't allow privileged logins with passwords, key only or not at all. Not sure what you are using as a gateway. Pfsense has pfblocker that you can run all the block lists fairly automatically which will become even more necessary when running a mail server. Also running ssh on a nonstandard port helps cut the volume a little, especially if you delay the Initial tcp connect on the gateway, the scanners won't wait around for the response.
Dshield.or has lots of good suggestions and good idea to submit your logs to them.

akaggie05 said:

---

Hopefully you don't have a telnet server facing the internet?

---

No, but I figure that anyone trying to connect to one is worth blocking for other things, too. So I'll end up logging telnet attempts and will probably block those who are logged when I get around to it.

aggiez03 said:

---

Have you tried Fail2Ban? Someone makes X attempts to any configured service, they are banned for Y minutes or forever, whatever you setup. Works great.

---

I've heard some good things about it, but I don't think it is ported to OpenBSD. At least, it doesn't appear to be one of the packages that are available on the OpenBSD mirrors.

It's not too big a deal to write a little program to monitor the various logs and pull out what I need to block.

UmustBKidding said:

---

Run fail2ban and don't allow privileged logins with passwords, key only or not at all. Not sure what you are using as a gateway. Pfsense has pfblocker that you can run all the block lists fairly automatically which will become even more necessary when running a mail server. Also running ssh on a nonstandard port helps cut the volume a little, especially if you delay the Initial tcp connect on the gateway, the scanners won't wait around for the response.
Dshield.or has lots of good suggestions and good idea to submit your logs to them.

---

Our gateway is a Cisco 3925 managed by AT&T.

I'm not worried about people guessing the passwords on the computer, but I do find it aggravating to look through log files with tens of thousands of entries per day.

On my older machines, I use AllowUsers to define precisely which users may use ssh. Also, some also used Match Address blocks to limit access to our own network,at least on IPv4. Of course, the Match Address blocks do not stop people trying to get in from generating tends of thousands of log entries.

Here's what I'm moving toward on the new machine and will expand to the others as I get to them:

1) Only those accounts that are members of a certain group, currently "rusers", will be allowed access via ssh. Naturally, root won't be one of them.

2) Instead of doing administration from root, I'm setting up an account for administration and use sudo to do the administration. The account name won't hint at the purpose of the account.

3) As I've done for some time, except before it was under root, the account will have a directory structure that will contain the configuration files to be edited. For example, /root/admin/etc would contain all the configuration files in /etc. I would edit them there and when done would use the make utility to copy any changed files over after making a backup copy of the current file and then restart whatever services are required. There are also directories for mail, dns, and ssh. For example, the makefile entry for pf.conf in the etc configuration directory on each machine is:

/etc/pf.conf: pf.conf
pfctl -n -f ./pf.conf
cp /etc/pf.conf /etc/old.pf.conf
cp pf.conf /etc
chmod 600 /etc/pf.conf
pfctl -f /etc/pf.conf

(using a tab in place of the spaces at the start of each line, of course). Note that the "pfctl -n -f ./pf.conf" commend checks to see that the new pf.conf file is valid. If it isn't, then the following commands aren't executed.

One major advantage of this approach is that if I'm editing something and take a lunch break, I can save my changes without having to worry about having a problem resulting in the machine being rebooted. After I finish the changes and am satisfied with them, all I need to do is type "sudo make" to update them. Of course, if I have some other changes that are unfinished, I use "make /etc/pf.conf" on the older machines and "sudo make /etc/pf.conf" from the administration account on the new machine.

Doing this also puts all the important configuration files in one location that is easy to backup at will. When building a new system, I can copy in the configuration files, modify them for the new computer, and then make the changes quickly. It really speeds things up over having to through and edit every file from scratch.

I'm undecided about one thing, though. With the size and complexity of my usual passwords, typing them in every time I want to do sudo has proven rather painful and so I've disabled entering the pasword for sudo commands. Since I'm disallowing ssh access to root, I'm tempted to go to a simple password that is relatively easy to enter without slowing me down much.

Here's a couple of examples of how I structure my most secure passwords these days (note that I haven't used the examples below as passwords before):

DAL785: MOVW #20,R7 ; elephant parade
W11: SUBL2 #99,@(R6)+ ; arugula worms

If you aren't familiar with it, those are valid VAX assembler instructions with valid labels and a comment that has absolutely nothing to do with the instruction. There are plenty of upper case, lower case, numbers, and punctuation in each. Also, they make a certain amount of sense which makes them easier to remember.

You can see how I hate having to enter something like these every time I use sudo.

One obvious concern is if I make an error in the sshd_config file and end up exposing the root account, a short, simple password such as "cowboy" could easily be breached.

Not sure anyone packages fail2ban. Its just a bunch of python scripts with a python install script included. Typically 6-7 commands to wget and "install". Lots of type this here examples for BSD variants.
You should probably use some of the more obscure assembler instructions like Branch and hang, halt and catch file, and punch disk in your passwords.
Not really a fan of passwords and prefer keys. If you need to let people login with passwords (not technical enough to deal with keys) I typically chroot jail them to something very limited to only allow them access to what they need.

UmustBKidding said:

---

Not sure anyone packages fail2ban. Its just a bunch of python scripts with a python install script included. Typically 6-7 commands to wget and "install". Lots of type this here examples for BSD variants.
You should probably use some of the more obscure assembler instructions like Branch and hang, halt and catch file, and punch disk in your passwords.
Not really a fan of passwords and prefer keys. If you need to let people login with passwords (not technical enough to deal with keys) I typically chroot jail them to something very limited to only allow them access to what they need.

---

I do use keys, but not for root access or for accounts that can use sudo.

You do use passwords to unlock your keys, don't you?

On your border router just route the specific IP addresses to null0.

JPTC08 said:

---

On your border router just route the specific IP addresses to null0.

---

It would be simpler to just use a private IP address for the computer if I didn't need access remotely. And that would free up a precious IP address for use elsewhere.

By the way, for years I had a firewall with no IP addresses on it at all. To access it, I had to sit in front of the terminal. I thought about running an RS-232 connection to it and hooking it up to a dumb terminal, but I only have the one dumb terminal and keep it connected to a computer that I can use to log onto other computers, both in the office and remotely.

Unfortunately, we ran out of routable IPv4 addresses and so I had to also use it for NAT for the extra equipment that does not require access from outside the network.

On the other hand, I have 2^72 IPv6 addresses. I'm never going to run out of those.

One thing I see a lot of is people connecting to port 22 without actually trying to log in. My assumption is that they are looking to see what version of SSH I'm running.

I typically use passwords on my keys, and use ssh-agent or pageant if connecting to remotes from whidos machine. So I only have to enter password once per session. But some machine-machine stuff has keys with no passwords and those typically connect to each other in chroot jails or within containers.
Typically the connect no attempt cases are trying to see if you support V1 (crazy to still allow it) and if ssh is present. The ssh brute force people find servers, then farm the attempts across many hosts/networks. We see this when we setup a new honeypot.
One nice thing about IPV6 is that your outgoing connections typically pick a random address out of your routers netblock and its not the address of your daemon's. This cuts down on the I connect to something (web, mail, other) and the host you connect to puts your machine on the list to be scanned.

Was looking at some stuff on dshield and they now can collect data from this
https://github.com/desaster/kippo
If you want to really see what all people poking on your machine would do if they were successful.

Fail2ban

DenyHost our Fail2Ban