Jump to content

Game/Auth handshake vulnerability


Sherer

Recommended Posts

  • Bronze

M2 Download Center

This is the hidden content, please
( Internal )

Hello,

Didn't know whether to place this subject either here or on guides but since I'm gonna not only share the mitigation but also express my conclusion about this issue and further after-effects.
Only part of codes are attached below.
For methods definitions + headers jump here:

This is the hidden content, please

1. Vulnerability overview.

Any kind of tcp application is required to have a server (anda  client). Once a server is launched it's binded to appropriate socket and set on listening for further connections (in a cutoff, navigate here for more precise info)
On the other side client is the one which is suppose to connect to server. It does it by connecting to server's socket and start to process a handshake.
This is how it works in a big shortcut - or I should rather say, it does work if we deal with normal peer.
In case of more-like modern apps the traffic is held by efficient algorithms that can carry even heavy traffic or eventual feeble attack.
But since Metin2's core is not a state-of-art app and simply runs on old C style code (libthecore) without a real concurrency support that might dive us into some tricky issues.
So let's imagine what happens if someone tries to pull off an attack and flood the server with enormous amount of packets.
Since each of connection needs to be validated first, it goes through the handshake.

Server catches it through the fdwatch function (jump to io_loop, main.cpp), then if no desc is presented moves it to AcceptDesc where the connection is validated.
It is  usually allowed unless the peer's ip is not in a banlist. Then desc is created and connection is saved for further handshake response.
Sounds reasonable, right?
Now imagine when thousands connection are accepted, validated, and expected to send handshake response.
Does it still sound so optimistic?
Additionally, when a desc is created it does allocated buffer for I/O traffic. Each connection. Each time. Each desc.

Now do the math and try to conceive how much of memory is simply wasted.
So that's the point we've been heading.

That's our vulnerability.
And btw, if so many agents are in the queue, do you think anybody will be able to connect your server?

2. Four major problems.

Let's start from handshake itself.
Imagine that someone approaches you and gives you his hand to shake it.
And than second.
And then third.
Doesn't make sense, does it?
Same applies to handshake process. Simply, only one handshake shall be ongoing for the host unless it completes it.
So, let's jump to AcceptDesc in desc_manager.cpp and add this little code above:
 

newd = M2_NEW DESC;
	// Let's check if a handshake from this host is already ongoing
	if (GetHostHandshake(peer))
	{
		sys_log(0, "Handshake from %s is not permitted!", host);
		socket_close(desc);
		return NULL;
	}

So let's consider this as solved for now.

Then the second issue.
Let's imagine yet another event.
Someone shakes your hand but this time completes the handshake. But he does it again. And again. And again.
Sounds exhausting?
Let's add this 2 conditions below our recent code from above:

	static const int HOST_CONNECTION_LIMIT = 3;
	// In case if host completed handshake process let's check if it doesn't reach the limit
	if (GetHostConnectionCount(peer) >= HOST_CONNECTION_LIMIT)
	{
		sys_log(0, "Host %s connection limit has been reached!", host);
		socket_close(desc);
		return NULL;
	}

	// And block intrusive connections as well
	if (IsIntrusiveConnection(host))
	{
		sys_log(0, "Host %s is intrusive!", host);
		socket_close(desc);
		return NULL;
	}

First if checks if host doesn't reach the handshake limit and if it does - the host is dropped.
Second if seeks for intrusive peers.
That simply means if one tries to connect again, and again, and again in defined time lapse it probably turns out to be an attacking IP.
Let's jump for a moment to desc.cpp, Initialize and add this variable initialization there:

tt_creation_time = get_global_time();

Yet another problem solved.


Still tho, all this code is considered to work more for authentication than for game's core.
Why is so?
Imagine a person how is not suppose to have any attention at all - a movie star for example.
Usually when one walks on red carpet there are bunch of body guards sealing him/her out from the crowd around.
Same should happen to game cores because why one would try to perform a handshake with game if hadn't even logged in?
So that's we are going to do, simply whitelist players who were succeeded to perform through the login process and obtained login key.
Firstly let's jump back to the desc_manager.cpp and add this little code above our previous alterations:

	// If it's not an auth server - check for validation first
	if (!g_bAuthServer)
	{
		if (!IsOnHandshakeWhitelist(peer))
		{
			// sys_log(0, "Host %s has not validated through login!", host);
			socket_close(desc);
			return NULL;
		}
	}

Now open input_db.cpp, move to AuthLogin function and add at the end:

	// Validating handshake
	TPacketGGHandshakeValidate pack;
	pack.header = HEADER_GG_HANDSHAKE_VALIDATION;
	strlcpy(pack.sUserIP, d->GetHostName(), sizeof(pack.sUserIP));
	P2P_MANAGER::instance().Send(&pack, sizeof(pack));

And so on repeat it for AuthLoginOpenID if you use it.
Now let's jump to input_p2p.cpp, move to Analyze function and after the initial syslog:

	// Auth server is not allowed for p2p
	if (g_bAuthServer)
	{
		// Clearing buffers for dynamic packets
		switch (bHeader)
		{
			case HEADER_GG_RELAY:
			{
				TPacketGGRelay * p = (TPacketGGRelay *) c_pData;
				if (m_iBufferLeft < sizeof(TPacketGGRelay) + p->lSize)
					iExtraLen = -1;
				else
					iExtraLen = p->lSize;
			}
			break;
			case HEADER_GG_NOTICE:
			{
				TPacketGGNotice * p = (TPacketGGNotice *) c_pData;
				if (m_iBufferLeft < sizeof(TPacketGGNotice) + p->lSize)
					iExtraLen = -1;
				else
					iExtraLen = p->lSize;
			}
			break;
			case HEADER_GG_GUILD:
			{
				iExtraLen = m_iBufferLeft - sizeof(TPacketGGGuild);
			}
			break;
			case HEADER_GG_MONARCH_NOTICE:
			{
				TPacketGGMonarchNotice * p = (TPacketGGMonarchNotice *) c_pData;
				if (m_iBufferLeft < p->lSize + sizeof(TPacketGGMonarchNotice))
					iExtraLen = -1;
				else
					iExtraLen = p->lSize;
			}
			break;
		}

		return iExtraLen;
	}

Since some of the packets might be dynamic, we need to ensure that data they hold is cleared properly.
If you have more dynamic packets binded - add them as above.

Move to db.cpp, find function SendLoginPing and replace with following:

void DBManager::SendLoginPing(const char * c_pszLogin)
{
	/*
	TPacketGGLoginPing ptog;

	ptog.bHeader = HEADER_GG_LOGIN_PING;
	strlcpy(ptog.szLogin, c_pszLogin, sizeof(ptog.szLogin));

	if (!g_pkAuthMasterDesc)  // If I am master, broadcast to others
	{
		P2P_MANAGER::instance().Send(&ptog, sizeof(TPacketGGLoginPing));
	}
	else // If I am slave send login ping to master
	{
		g_pkAuthMasterDesc->Packet(&ptog, sizeof(TPacketGGLoginPing));
	}
	*/
}

Avoiding clearing billing like that (wtf is that btw, shouldn't be executed at all).
Now move to packet_info.cpp and add this code in constructor:

Set(HEADER_GG_HANDSHAKE_VALIDATION,		sizeof(TPacketGGHandshakeValidate),	"HandShakeValidation",		false);

Jump back to input_p2p.cpp and add this case in Analyze function:
 

		case HEADER_GG_HANDSHAKE_VALIDATION:
			DESC_MANAGER::instance().AddToHandshakeWhiteList((const TPacketGGHandshakeValidate *) c_pData);
			break;

Finally jump to ClientManager.cpp in DB. Find function QUERY_SETUP and if condition with bAuthServer and add there following code:

peer->SetChannel(1);

Sine P2P communication is allowed only for peers possessing any channel number greater than zero, we set it.
Usually this practice should be forbidden but since we restrain the traffic for auth server (with code above) it should be safe.

Beware that this might cause first login failed because of packet propagation that can reach the cores after player connects.
Voilà, were mostly done with coding!

Last but no least, we need to take a brief introduction into kqueue and tedious tour between sockets and kernel vars.
Starting with kqueue. I would try to explain this but you better jump to this link. Freebsd documentation always appreciated.
Since Metin2 implementation of kqueue wrapper has its size limit you may try to increase it a bit and seek for a feedback.
If'd like to do so jump to main.cpp, start function and edit this variable:
 

main_fdw = fdwatch_new(VALUE);

Yet keep in mind! Do not try to over-optimize it! Try to experiment, put the different values. If you somehow screw it up it might drag you into the checkpoint issues and eventually crash the whole app.
So now a few words about sockets and how the listening process works.
When each connection aiming to appropriate port is detected it is dropped into the queue where it's waiting for application to pick it up.
So simply we can consider this as a waiting queue like in a grocery store.
The point is that this queue has it's limit and once the limit is reached, any new connection is declined at sight.
The listening limit for Metin2 core is wrapped into variable called SOMAXCONN.
If you dive into C socket documentation you can find something like this:

/*
 *Maximum queue length specifiable by listen
/*
#define SOMAXCONN        10

As for me it was 128.
Since it's a define the value is simply embedded into the app and you cannot manipulate it once a binary is built.
So let's change it to let more connection be scheduled.
You may ask, why?
If player tries to log in it does connect the channel port.
If the channel is unavailable you see fadeout and connection is terminated.
It happens because there is no place in the queue thus connection is scheduled at all.
But be careful! Do not set this value into some high-peak numbers!
Be aware that our io_loop function need to iterate through these all events and manage to handle this during the heartbeat session.

If you try to over optimize this value you can end up causing lags on the server, internal packets delays and more.
In case you'd ask me, value around 1024 is acceptable but still it's better if you take some lecture and experiment a bit.
And one more thing, don't forget to set this kernel option on the machine where your server runs:

sysctl kern.ipc.soacceptqueue=1024
sysctl kern.ipc.somaxconn=1024

So we are done! Don't forget to add the code from my github repo!

Epilogue
Metin2's quite an old app and we should not forget about that.
The netcode is old, rubbish and cumbersome thus this issue might be only one of many we haven't found just yet.
Keep in mind tho that even that mitigation won't protect your server.
Actually I doubt that even rewriting the code into more modern shape would do that if you don't get yourselves a good protection.
Protection, filters, external firewalls are actually the key especially now when stressers and all this stuff are back and harmful again.
Hope that this little thread will help you in your future development.

Extra

I manage to write a little collector for getting rid of handshakes that never completed this process (outdated).
If you'd like to switch it on jump to desc_manager.cpp constructor and add there:

	CEventFunctionHandler::instance().AddEvent([this](SArgumentSupportImpl *) {
		desc_manager_garbage_collector_info* info = AllocEventInfo<desc_manager_garbage_collector_info>();
		m_pkDescManagerGarbageCollector = event_create(desc_manager_garbage_collector_event, info, PASSES_PER_SEC(1));
	}, "DESC_MANAGER_COLLECTOR", 1);

Beware that you need this feature:

And don't forget to add this to destructor:

event_cancel(&m_pkDescManagerGarbageCollector);


Regards

Btw, credits for @Flourine for flooding my dev server with 20k packets per sec (asked for 2 btw). That helped me to analyze the problem.

  • Metin2 Dev 87
  • Eyes 1
  • Angry 1
  • Not Good 1
  • Sad 1
  • Think 1
  • Scream 1
  • Good 43
  • Love 6
  • Love 105
Link to comment
Share on other sites

  • Premium

One of those very rare moments when you can actually read a well explained non-bullshit release on a metin2 dev forum that follows a logical thread written with decent language skills. Can't believe my eyes.

  • Sad 1
  • Good 1
  • Love 6

The one and only UI programming guideline

Link to comment
Share on other sites

First of all, thank you for informing me. I've had a pest attack before. So I had to solve a similar attack with "Pf Sense".

However, if you proceed with a logical way, you seem to have overlooked the slightest roughness.

1 - DB Core crashes at a request of 600 - 700 instant.
2 - In case of any malicious attack, you can receive 3,000 requests in 5 minutes and this attack will suffice for DB core to crash.
3 - Why not check the contents of the package?

My opinion; I think it would be more efficient if you do the operations on the Game core instead of DB core.

  • Love 1
Link to comment
Share on other sites

else if (!m_pPacketInfo->Get(bHeader, &iPacketLen, &c_pszName))
{
	if (g_bAuthServer)
		LOG_IP("%s",inet_ntoa(lpDesc->GetAddr().sin_addr));

	sys_err("Input: Unknown header:%d, Last header:%d(%d), Remain bytes:%d, fd:%d", bHeader, bLastHeader, iLastPacketLen, m_iBufferLeft, lpDesc->GetSocket());
	lpDesc->SetPhase(PHASE_CLOSE);
	return true;
}

Or you can save all the ip that send unknown packets to auth/game. After about 1 - 5 minutes you will have all the ips of the attacker and you can block them in the firewall. Or if you use OVH you can use OVH_API to filter IPs through the mitigation system.

This is the hotfix I used. ?

  • Love 1
Link to comment
Share on other sites

  • Bronze
6 godzin temu, cBaraN napisał:

First of all, thank you for informing me. I've had a pest attack before. So I had to solve a similar attack with "Pf Sense".

However, if you proceed with a logical way, you seem to have overlooked the slightest roughness.

1 - DB Core crashes at a request of 600 - 700 instant.
2 - In case of any malicious attack, you can receive 3,000 requests in 5 minutes and this attack will suffice for DB core to crash.
3 - Why not check the contents of the package?

My opinion; I think it would be more efficient if you do the operations on the Game core instead of DB core.

Thanks for the hint.
Due to your feedback I refactored handshake authentication to be done through P2P rather than by CORE->DB->CORE.
If someone follows this tutorial you need to reimplement that part (with github stuff).
According to your third question I don't really think it matters since some of methods don't even bother sending any packets at all.
They just spoof the queue with fake connection and stuck in handshake stage.

 

1 godzinę temu, ridetpro napisał:

else if (!m_pPacketInfo->Get(bHeader, &iPacketLen, &c_pszName))
{
	if (g_bAuthServer)
		LOG_IP("%s",inet_ntoa(lpDesc->GetAddr().sin_addr));

	sys_err("Input: Unknown header:%d, Last header:%d(%d), Remain bytes:%d, fd:%d", bHeader, bLastHeader, iLastPacketLen, m_iBufferLeft, lpDesc->GetSocket());
	lpDesc->SetPhase(PHASE_CLOSE);
	return true;
}

Or you can save all the ip that send unknown packets to auth/game. After about 1 - 5 minutes you will have all the ips of the attacker and you can block them in the firewall. Or if you use OVH you can use OVH_API to filter IPs through the mitigation system.

This is the hotfix I used. ?

Nice idea with OVH_API - haven't really heard about that (you can make some tutorial if you find it helpful for us).
In reference to the code you attached all methods bother sending packets at all.

Link to comment
Share on other sites

Acum 1 oră, Sherer a spus:

Thanks for the hint.
Due to your feedback I refactored handshake authentication to be done through P2P rather than by CORE->DB->CORE.
If someone follows this tutorial you need to reimplement that part (with github stuff).
According to your third question I don't really think it matters since some of methods don't even bother sending any packets at all.
They just spoof the queue with fake connection and stuck in handshake stage.

 

Nice idea with OVH_API - haven't really heard about that (you can make some tutorial if you find it helpful for us).
In reference to the code you attached all methods bother sending packets at all.

First you need to download

This is the hidden content, please
, which is available in PHP or Python.


AntiDDOS option. Add new IP on permanent mitigation

Spoiler

<?php
/**
 * First, download the latest release of PHP wrapper on github
 * And include this script into the folder with extracted files
 */
require __DIR__ . '/vendor/autoload.php';
use \Ovh\Api;

/**
 * Instanciate an OVH Client.
 * You can generate new credentials with full access to your account on
 * the token creation page
 */
$ovh = new Api( 'xxxxxxxxxx',  // Application Key
                'xxxxxxxxxx',  // Application Secret
                'ovh-eu',      // Endpoint of API OVH Europe (List of available endpoints)
                'xxxxxxxxxx'); // Consumer Key

$result = $ovh->post('/ip/xxx.xxx.xxx.xxx/mitigation', array(
    'ipOnMitigation' => 'Metin1', //  (type: ipv4)
));

print_r( $result );
?>

 

I was surprised too, to see how great that API from  OVH is. If you ever want to use this API, I suggest you send the data from the source to the website with the help of the web API embedded in the Metin2 source, then from website call the OVH api.

 

And yes, all DDOS methods send packets. I was referring strictly to the topic, the method by which this attacker killed your server was through packets sent to auth/channels. Anonymous packets, without a name..

With the code posted below, all incoming packets to auth will be wrote intro a text file then u can block em in few minutes.. Or you can blacklist them to ovh mitigation. Anyway, that's not really necessary right now.. As i said that's just a HOTFIX, it was a hotfix that worked very well.

  • Metin2 Dev 18
  • Think 1
  • Good 7
  • Love 4
Link to comment
Share on other sites

  • Bronze
25 minut temu, ridetpro napisał:

First you need to download

This is the hidden content, please
, which is available in PHP or Python.


AntiDDOS option. Add new IP on permanent mitigation

  Odkryj ukrytą treść


<?php
/**
 * First, download the latest release of PHP wrapper on github
 * And include this script into the folder with extracted files
 */
require __DIR__ . '/vendor/autoload.php';
use \Ovh\Api;

/**
 * Instanciate an OVH Client.
 * You can generate new credentials with full access to your account on
 * the token creation page
 */
$ovh = new Api( 'xxxxxxxxxx',  // Application Key
                'xxxxxxxxxx',  // Application Secret
                'ovh-eu',      // Endpoint of API OVH Europe (List of available endpoints)
                'xxxxxxxxxx'); // Consumer Key

$result = $ovh->post('/ip/xxx.xxx.xxx.xxx/mitigation', array(
    'ipOnMitigation' => 'Metin1', //  (type: ipv4)
));

print_r( $result );
?>

 

I was surprised too, to see how great that API from  OVH is. If you ever want to use this API, I suggest you send the data from the source to the website with the help of the web API embedded in the Metin2 source, then from website call the OVH api.

 

And yes, all DDOS methods send packets. I was referring strictly to the topic, the method by which this attacker killed your server was through packets sent to auth/channels. Anonymous packets, without a name..

With the code posted below, all incoming packets to auth will be wrote intro a text file then u can block em in few minutes.. Or you can blacklist them to ovh mitigation. Anyway, that's not really necessary right now.. As i said that's just a HOTFIX, it was a hotfix that worked very well.

Looks really interesting. How did it work for you? Once you added the ip into mitigation one wasn't able to connect to the server no more?
Did the user get dropped immediately?

  • Metin2 Dev 1
Link to comment
Share on other sites

  • Premium
6 hours ago, santa12 said:

the worst guide I've ever seen. Are you fucked up ?

all the time errors and mistakes what is this tutorial? When you were little, did you drown in the ocean?

I add it 24 hours and probably never add. Massacre 

xdddddddddddddddddddddd

  • Lmao 1


 

Link to comment
Share on other sites

6 godzin temu, santa12 napisał:

the worst guide I've ever seen. Are you fucked up ?

all the time errors and mistakes what is this tutorial? When you were little, did you drown in the ocean?

I add it 24 hours and probably never add. Massacre

xddddddddddddddddddddddddddddddddddddddddd

that is reason for which he shouldn't publish it

Link to comment
Share on other sites

Acum 46 minute, santa12 a spus:

but it's not all the code many things are missing here

Yes like your gratitude and your humbleness. I suggest you go and find santa and leave the grown ups to talk here. 

From my perspective I like the tutorial and I like the fact that you made it so that everyone would understand (well almost everyone).

 

  • Love 1
Link to comment
Share on other sites

1 godzinę temu, Catii napisał:

Yes like your gratitude and your humbleness. I suggest you go and find santa and leave the grown ups to talk here. 

From my perspective I like the tutorial and I like the fact that you made it so that everyone would understand (well almost everyone).

 

In file included from input_db.cpp:5:
desc_manager.h:116:38: error: 'TPacketGGHandshakeValidate' does not name a type
   void AddToHandshakeWhiteList(const TPacketGGHandshakeValidate * pack);

 

foreman fix this mistake for me because I've tried everything

Link to comment
Share on other sites

  • Bronze
8 godzin temu, santa12 napisał:

In file included from input_db.cpp:5:
desc_manager.h:116:38: error: 'TPacketGGHandshakeValidate' does not name a type
   void AddToHandshakeWhiteList(const TPacketGGHandshakeValidate * pack);

 

foreman fix this mistake for me because I've tried everything

Updated github repo with code line that's gonna solve this issue.

  • Love 2
Link to comment
Share on other sites

  • 1 month later...
  • 2 months later...
  • 4 weeks later...
  • 11 months later...
  • 3 weeks later...
  • Forum Moderator
2 hours ago, LordZiege said:

Seems not to work. Login wont work anymore.

 

I've been using it for quite a bit of time, the only "problem" (which is not one) is when I connect right after starting the server.

Gurgarath
coming soon

Link to comment
Share on other sites

  • 4 weeks later...

we tested this now, with a small penetration tool and this fix does literally nothing. It wont handle the handshakes better in anyway. 1 second after i start my pen tool (the source code for this is public on a microsoft website for testing, its not even a "hard" program, it only sends 14 bytes per second) the auth server will not take any other handshake. nothing will happen. The false handshakes will not getting rejected, they wont get cleared, it only handles one by one so as long as the tool sends simple handshake requests, no one else can connect. This "fix" or "improvement" does literally nothing to your server. 

just install an pf on your freebsd and the false handshakes are getting blocked after maybe 3 seconds and than you are absolutly fine. But do not install this here and think you have a much better auth server now. The "fix" / "improvement" just didnt do anything.

Link to comment
Share on other sites

Announcements



×
×
  • Create New...

Important Information

Terms of Use / Privacy Policy / Guidelines / We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.