Viable protections against Layer7 attacks on servers

[CORKY 242]

Hello,

 

As of recently, new methods of ddosing servers have been found out. The problem is, there isn't actually a "way to protect" for everybody.

The new attacks are based on layer 7, which the freebsd's PF and IPFW does not support unfortunately. Other then creating a reverse proxy for the server, which could filter out the ips, is there any alternative method?

After also testing on multiple servers, it seems that even the bigger servers have problems with it, if somebody decides to pay 100 euro for a stresser subscription with layer 7 methods, it can be bad.

Any ideas? 

[Shogun 4498]

Every server out there has an attack vector of some sort. Then it's all up to the patience and skill of the attacker.

The real question here is: are you able to figure out what and how is hitting you?

If the answer is yes then you can establish some sort of countermeasure. Even if it involves some discomfort for your users.

There has been layer 7 attacks always, so you need to be more specific or provide tcpdump logs.

[CORKY 242]

16 minutes ago, Shogun said:

Every server out there has an attack vector of some sort. Then it's all up to the patience and skill of the attacker.

The real question here is: are you able to figure out what and how is hitting you?

If the answer is yes then you can establish some sort of countermeasure. Even if it involves some discomfort for your users.

There has been layer 7 attacks always, so you need to be more specific or provide tcpdump logs.

I have direct access to the stresser and can always test it, I tried to block it myself but unfortunately failed.

I even tried to install nginx & deny all connections but that didn't worked LOL.

The attack is simply sending requests to https://ip:port/ where the ip is of course, the server's ip and the port being either a channel or the auth, won't matter as it will completely break the whole auth.

 

Here's a tcpdump (using tcpdump -i vtnet0 tcp) log while attacking for 5 minutes:

https://rentry.co/eystz (pastebin alternative cuz the text is way longer then 500kb)

[Shogun 4498]

1 minute ago, dumita123 said:

I have direct access to the stresser and can always test it, I tried to block it myself but unfortunately failed.

I even tried to install nginx & deny all connections but that didn't worked LOL.

The attack is simply sending requests to https://ip:port/ where the ip is of course, the server's ip and the port being either a channel or the auth, won't matter as it will completely break the whole auth.

 

Here's a tcpdump (using tcpdump -i vtnet0 tcp) log while attacking for 5 minutes:

https://rentry.co/eystz (pastebin alternative cuz the text is way longer then 500kb)

Please use the -n option and exclude the ssh port. Otherwise the dump isn't useful.

tcpdump -i vtnet0 -n -vvv port not <your ssh port number>

[CORKY 242]

7 minutes ago, Shogun said:

Please use the -n option and exclude the ssh port. Otherwise the dump isn't useful.

tcpdump -i vtnet0 -n -vvv port not <your ssh port number>

Sorry, here's the tcpdump using your arguments & the port of the attacked channel. (tcpdump -i vtnet0 -n -vvv port 20085 in my case)

This is the hidden content, please

• Sign In
• or
• Sign Up

And here is the tcpdump for every port besides the one of my ssh (in my case is 22 since it's a test server)

This is the hidden content, please

• Sign In
• or
• Sign Up

Edited July 4, 2022 by dumita123

[Shogun 4498]

What sort of attack are you running and what is your pf.conf?

[CORKY 242]

Just now, Shogun said:

What sort of attack are you running and what is your pf.conf?

Running a layer7 attack via a public stresser, using a method called "HTTP-SOCKET", which is spamming GET requests to https://myserverip:channel1port/ with 30 requests per ip, using around 2k ips with random geolocation.

 

Here's my pf conf:

# This is a minimalistic configuration with basic rate limiting

# Change the value to reflect your public interface. You can see this with ifconfig.
ext_if="vtnet0"

# Ports used for services (ssh, http, https)
service_ports="{ 22, 3306 }"

# Ports
game_ports="{20076, 20083, 20081, 20079, 20077 }"
auth_ports="{20085}"

icmp_types = "{ echoreq, unreach }"

# table for abusive_hosts = Block IP´s.
table <abusive_hosts> persist file "/usr/home/game/Firewall/abusive_hosts"

# IP addresses that should override the firewall rules, such as your web server.
table <whitelist> const { 127.0.0.1 } 

martians = "{ 127.0.0.0/8, 192.168.0.0/16, 172.16.0.0/12, \
              10.0.0.0/8, 169.254.0.0/16, 192.0.2.0/24, \
              0.0.0.0/8, 240.0.0.0/4  255.255.255.255/32 \
              ::/128 ::1/128 ::ffff:0:0/96 ::/96 100::/64 \
              2001:10::/28 2001:db8::/32 fc00::/7 fe80::/10 \
              fec0::/10 ff00::/8 }"

set block-policy drop

set limit { states 100000, frags 20000, src-nodes 100000, table-entries 200000 }

set loginterface $ext_if

set skip on lo

scrub on $ext_if reassemble tcp no-df random-id

antispoof quick for { lo0 $ext_if }

block in
block in quick from <abusive_hosts>

# Drop all Non-Routable Addresses
block drop in quick on $ext_if from $martians to any
block drop out quick on $ext_if from any to $martians

pass out all keep state
pass out on $ext_if all modulate state

# Allow ping
pass inet proto icmp all icmp-type $icmp_types keep state

# Rate limits
pass in on $ext_if proto tcp to any port $service_ports flags S/SA synproxy state \
(max-src-conn 30, max-src-conn-rate 15/5, overload <abusive_hosts> flush global)

pass in on $ext_if proto tcp to any port $game_ports flags S/SA synproxy state \
(max-src-conn 30, max-src-conn-rate 15/5, overload <abusive_hosts> flush global)

pass in on $ext_if proto tcp to any port $auth_ports flags S/SA synproxy state \
(max-src-conn 30, max-src-conn-rate 15/3, overload <abusive_hosts> flush global)

pass in quick from <whitelist>

 

[Shogun 4498]

Such attack is normally too weak to bring any server down. The problem most likely lies elsewhere.

Please provide the output of ifconfig and specifically, there's something to consider since you are running off a virtual machine:

https://docs.netgate.com/pfsense/en/latest/virtualization/virtio.html

You should also disable pf and see what happens then. Does the attack still bring the channel down? If not, then your problem is with VirtIO. If yes, I suggest you follow this tutorial by Papix which is actually a pretty good idea:

 

Otherwise you can contact me on Discord if you want me to have a look at it myself for a small fee.

[CORKY 242]

3 minutes ago, Shogun said:

Such attack is normally too weak to bring any server down. The problem most likely lies elsewhere.

Please provide the output of ifconfig and specifically, there's something to consider since you are running off a virtual machine:

https://docs.netgate.com/pfsense/en/latest/virtualization/virtio.html

You should also disable pf and see what happens then. Does the attack still bring the channel down? If not, then your problem is with VirtIO. If yes, I suggest you follow this tutorial by Papix which is actually a pretty good idea:

 

Otherwise you can contact me on Discord if you want me to have a look at it myself for a small fee.

Actually, it is not in fact a small attack. And idk if i'm running on a virtual machine, for now it's a test vps bought from a hosting company which offers iso installation directly, so I just installed a 12.1 FreeBSD with that feature for it.

 

Here's the output from ifconfig:

vtnet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 00:ce:b4:37:72:cd
        inet 160.20.145.45 netmask 0xffffff00 broadcast 160.20.145.255
        inet6 fe80::2ce:b4ff:fe37:72cd%vtnet0 prefixlen 64 scopeid 0x1
        media: Ethernet 10Gbase-T <full-duplex>
        status: active
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

 

I disabled pf, I can login and play normally, however, once I start the attack, the whole server will start lagging and, after a little bit will eventually kick my character and the whole login will break down, not even getting past "Connecting to server..."

[Shogun 4498]

Run this command:

ifconfig vtnet0 -txcsum -rxcsum -rxcsum6 -txcsum6

See if anything changes (with pf on)

 

[CORKY 242]

7 minutes ago, Shogun said:

Run this command:

ifconfig vtnet0 -txcsum -rxcsum -rxcsum6 -txcsum6

See if anything changes (with pf on)

 

Server is running, I can connect and do all the things I should be able to, however after starting the attack, only the auth will go down (Not crashing, only being stuck at "Connecting to server...").

[Shogun 4498]

You can contact me on Discord tomorrow and I can have a look

[DrTurk 131]

new? that type of attacks in this scene arent new at all, just not many have known about it.

the attack is just flooding the sockets, so that no one can connect to the server, there are methods to fix this, but its not easy.

Edited July 5, 2022 by DrTurk

[masodikbela 1333]

I have limited knowledge of freebsd and network security, but most of the time what ppl do is create a whitelist for the gameserver who can connect to it. By this I mean that on clientside they silently connect or send a request to a different server and they solve some kind of quiz to validate that it is actually a client on this ip, and then they update the whitelist on the firewall side. Now there are lots of room to play, like this validator server could be on a different machine or just listen on a different ip, there could be many other validators, etc.

This would solve many problem, like you mentioned normally the bottleneck is the application itself like auth which gets overrun by connection requests and then ends up in a spiral of death rendering the application totally irresponsive. Now I believe that the freebsd kernel (and probably most of the os kernels) is very effective dropping unwanted packets so this type of attacks would have little to no noticeable affects on your network/machine. I've seen relatively big botnets fail against this method. Obviously if the attack is so large that the kernel (more specifically the firewall) becomes the bottleneck and your hardware cannot keep up, or your network bandwidth becomes limited because of the amount of data you receive this wont work. In these cases probably nothing you can do anyway and you would require to have an industrial grade hardware firewall instance before your machine (or if you do have it like for example at ovh they do have it afaik you just need control over it).

Now we can argue how effective is this extra validation method. To be able to manipulate the quiz you have to reverse engineer the client which is already a relatively difficult task and you can make it harder with virtualisations. After that you have to implement this to the botnet itself which is already a problem for some of the botnets, like you dont have that fine grade of control over it. Now you can say that you will just target the validators, this indeed could work but you can anytime add more, change validation method, etc. Also this would only mean that new ips wouldnt be able to connect to your server which would affect relatively few players, because lets say you allow the ips for 48 hours to be whitelisted. The majority of the players log in every day or at least every few days, so they would be fine.

Like Shogun said its about counter measures, counter counter measures and so on, its just about how patient the attacker is. In general this method is more than enough for a metin server.

Now about how to implement it: I don't know, never configured any firewalls, never wrote any rules and don't know what tools you have for example for pf on freebsd. Like I said, this is not my field of expertise so feel free to correct me if I'm just talking bullshit now.

Edited July 5, 2022 by masodikbela