[SRC] Metin2 on LINUX - The Old Metin2 Project

[Exynox 1538]

Hi there,

Your favourite mad-lad is back! As I hinted over a year ago, I had a port of the Kraizy serverfiles in the works, with the goal of having the Metin2 Server run on Linux. Progress was slow, as 10 years passed since the Kraizy days, and now we all have education and jobs to deal with, and any free time I had was promptly allocated to sleep with the same passionate love Google Chrome eats any free RAM on a 1km radius.

Jokes aside, I still had this project on the back of my mind and still managed to bring it to a state where I feel I can release to you, guys. So I raise you The Old Metin2 Project, a more sensible attempt at continuing the work of others before us and having modern-ish serverfiles which run on modern technology, while still preserving the 2014 (or earlier) gameplay.

What needs to be said right away:

• This project is freely available for everyone to use, modify, and pretty much everything you'd like.
• You are invited to contribute any ideas, changes and improvements to the project on the git server. I don't have too much time but I'm willing to work on it as much as I can.
• The project is currently VERY experimental and should not be used in order to host private servers for now.
• Not many known bugfixes were applied to the project, and it is vulnerable to a lot of known attacks, another reason for it being unfit for production.

Improvements

• The binaries run on 64-bit x86-64/arm64 Linux with the network stack being partially rewritten in Libevent.
• CMake build system mainly based on vcpkg. Docker-friendly architectural approach.
• HackShield and other proprietary binaries were successfully yeeted, the project only has open-source dependencies.
• Included gamefiles from TMP4's server files.
• Modern logging system.
• Other updates and small refactorings.

The project's home

The project is currently hosted on a private Gitea instance. More instructions are available in their respective README.md files.

• Server:
  This is the hidden content, please

  • Sign In
  • or
  • Sign Up
• Client:
  This is the hidden content, please

  • Sign In
  • or
  • Sign Up
• Deployment files:
  This is the hidden content, please

  • Sign In
  • or
  • Sign Up
• AIO: (Last update by ASIKOO: 2024-03-03 19:29:00) 
  This is the hidden content, please

  • Sign In
  • or
  • Sign Up

I might require people to sign up in order to access the source in order to avoid trouble with Gameforge and other IP holders.

Future plans

• A Laravel website + item-shop + patch server based on my older work is also in the cards, yet I need to translate everything to English so that y'all don't jump on me.
  
• Refactoring the configuration code so that we could easily configure the gamecores from Docker-friendly environment variables (and .env files!).
• General refactoring of the old C++ code: memory safety, string formatting, UTF-8
• Adding all the known bugfixes
• Maybe releasing an alternate version with some quality-of-life systems

Ending notes

I am looking for any help in organizing, contributing and growing the project! I'll try to help you guys as much as I can, but time is tight, so my dream is for this to become a community effort. 

 This project wouldn't have been possible without the help and work of these amazing people:

• My good friend @ Sonitex, who morally helped me and also provided me with some special sauce.
• @ TMP4's serverfiles - it's a great resource and he deserves all the kürtőskalács we can get him:
• @ arves100's great guide on using vcpkg for Metin2 library management:
• @ Vanilla, who I'd argue was a pioneer in improving the gamecore and inspired me to start this.
• A lot of other members of this community whose posts provided me with a lot of crucial information.

 

Yours,

Exynox

[Gurgarath 2447]

Thank you for your release! I have seen a few libevent based binaries and sources ever since its first release in 2014 by iMer and I admit I was thrilled to expect yet another network poller like asio or libev or some others. Frankly well done for your project, this is exactly what makes us go forward as a community! Such project would give access to WSL or Docker to most people and I admit the vcpkg approach is a good idea!

Thank you for your project and I am looking forward to more people to commit on this!

[otaku 2]

Holy moly, thank you so much. <3

[wolfhelm 2]

If you want a better native version of the network stack for Linux, I can send it to you. I have a tested stack on epoll with 1k+ players.
 

https://discordapp.com/users/699723160467144777

Epoll example version: 

#define __LIBTHECORE__

#include "stdafx.h"

fdwatch *fdwatch_new() {
    fdwatch *fdw;
    CREATE(fdw, fdwatch, 1);

    fdw->fd = epoll_create1(0);

    if (fdw->fd == -1) {
        sys_err("%s", strerror(errno));
        return nullptr;
    }

    CREATE(fdw->events, epoll_event, 256);

    return fdw;
}

void fdwatch_delete(fdwatch *fdw) {
    close(fdw->fd);
    free(fdw->events);
    free(fdw);
}

int32_t fdwatch_count(fdwatch *fdw) {
    int32_t num = epoll_wait(fdw->fd, fdw->events, 256, 0);
    if (num == -1) {
        fprintf(stderr, "%s\n", strerror(errno));
    }
    return num;
}

void fdwatch_add_fd(fdwatch *fdw, socket_t fd, void *client_data) {
    TFDWatchData *data;
    CREATE(data, TFDWatchData, 1);

    data->client_data = client_data;
    data->fd = fd;

    epoll_event event{};
    event.events = EPOLLIN | EPOLLOUT;
    event.data.ptr = data;

    if (epoll_ctl(fdw->fd, EPOLL_CTL_ADD, fd, &event)) {
        fprintf(stderr, "%s\n", strerror(errno));
    }
}

void fdwatch_del_fd(fdwatch *fdw, socket_t fd) {
    epoll_ctl(fdw->fd, EPOLL_CTL_DEL, fd, nullptr);
}

int32_t fdwatch_check_event(fdwatch *fdw, socket_t fd, int32_t event_idx) {
    auto *data = (TFDWatchData *) fdw->events[event_idx].data.ptr;

    if (data->fd == fd) {
        return fdw->events[event_idx].events;
    } else {
        return 0;
    }
}

void *fdwatch_get_client_data(fdwatch *fdw, int32_t event_idx) {
    auto *data = (TFDWatchData *) fdw->events[event_idx].data.ptr;
    return data->client_data;
}

#ifndef __INC_LIBTHECORE_FDWATCH_H__
#define __INC_LIBTHECORE_FDWATCH_H__

#include <sys/epoll.h>

struct fdwatch {
    socket_t fd;
    epoll_event *events;
};

typedef struct FDWatchData {
    socket_t fd;
    void *client_data;
} TFDWatchData;

extern fdwatch *fdwatch_new();

extern void fdwatch_delete(fdwatch *fdw);

extern int32_t fdwatch_check_event(fdwatch *fdw, socket_t fd, int32_t event_idx);

extern void fdwatch_add_fd(fdwatch *fdw, socket_t fd, void *client_data);

extern int32_t fdwatch_count(fdwatch *fdw);

extern void *fdwatch_get_client_data(fdwatch *fdw, int32_t event_idx);

extern void fdwatch_del_fd(fdwatch *fdw, socket_t fd);


#endif

 

 

Edited December 14, 2023 by wolfhelm

[.NyanCat 22]

What a wonderful project! 
Thanks for the effort!

The source compiles and runs natively on my m1 mac as well! (Without any vm)
Next is tryin to connect with the client.

Gonna keep you updated...

[Exynox 1538]

10 minutes ago, .NyanCat said:

What a wonderful project! 
Thanks for the effort!

The source compiles and runs natively on my m1 mac as well! (Without any vm)
Next is tryin to connect with the client.

Gonna keep you updated...

I'm surprised to learn that it actually compiles on macOS -- I guess that's one point for having the network code implemented in libevent and not epoll, as @wolfhelm kindly suggested. (But in order to give Caesar the things that are Caesar's, the original kqueue code should have also worked on macOS)

 

Also, expect really weird behaviour on ARM, I tried using armour on my character and when I'd right click on it, it would just vanish. It surely needs some polish.

Edited December 14, 2023 by Exynox

[wolfhelm 2]

In my opinion, the linux implementation of epoll is better than the abstraction layer like libevent. When it comes to the server, we care about performance. If there is no macos client, I don't see any point in running the macos server. But of course, you can run it on macos for fun. As you mentioned kqueue should work on macos. 

In my tests, this epoll implementation is really fast. Logging into the server or changing the map is several times faster than in the bsd version. My version was also tested "in production", all with several hundred players on one ch.

As for the strange behavior, I suspect the problem is the port to 64bit. I haven't checked your code yet, but in my case similar problems occurred when the 32bit source version uses long which is treated as 32bit. Where when transferred to 64 bit it is treated as a 64 bit variable. This occurs in many places, but you probably know it because without changing the protocol from long to int, you would not be able to connect to the server.

Edited December 15, 2023 by wolfhelm

[.NyanCat 22]

The strange problems are different from arm to x86. On arm the items disappear and some other things are broken.
Running the deployment on windows, those bugs are not available but you are not able to connect after several minutes and the same happens on arm as well.(Wrong password message)
Starting the auth over helps but the problem continues.

[Exynox 1538]

On 12/15/2023 at 2:09 AM, wolfhelm said:

In my opinion, the linux implementation of epoll is better than the abstraction layer like libevent. When it comes to the server, we care about performance. If there is no macos client, I don't see any point in running the macos server. But of course, you can run it on macos for fun. As you mentioned kqueue should work on macos. 

In my tests, this epoll implementation is really fast. Logging into the server or changing the map is several times faster than in the bsd version. My version was also tested "in production", all with several hundred players on one ch.

As for the strange behavior, I suspect the problem is the port to 64bit. I haven't checked your code yet, but in my case similar problems occurred when the 32bit source version uses long which is treated as 32bit. Where when transferred to 64 bit it is treated as a 64 bit variable. This occurs in many places, but you probably know it because without changing the protocol from long to int, you would not be able to connect to the server.

I honestly can't really disagree with you. Of course any abstraction layer comes with a performance penalty. I even have, in the past, ported the network code to Boost.Asio and declared it a lost cause due to its really weird design patterns. Let's say the least, I'm a bit tired of rewriting the network stuff, and settled on libevent which has a decent balance of portability, ease of writing and performance (Chromium, Tor, Transmission, and other projects successfully rely on it). Not to mention that it would be really easy to add TLS to the server (as I've removed the original encryption schemes and communication is now plain).

I really, really don't want to discredit you and I truly appreciate a battle-tested solution, especially in my field of work. Metin2 is not that - it's just a hobby of mine, and hobby projects only get hobby amounts of time allocated to them. Still, you are more than welcome to publish your work and I encourage you to do that, as it's healthy for any project. Maybe we could even give people the choice to choose their network stacks one day.

Regarding the strange behaviour, I'm aware that certain C/C++ types have variable length based on the word size of the CPU architecture. Therefore, I did replace relevant variables back to their correct int32_t / uint32_t types. It works pretty fine on x86_64, but armv8 is still buggy. More time is needed to squash those bugs (I spent a grand total of 5 minutes on that issue).

8 hours ago, .NyanCat said:

The strange problems are different from arm to x86. On arm the items disappear and some other things are broken.
Running the deployment on windows, those bugs are not available but you are not able to connect after several minutes and the same happens on arm as well.(Wrong password message)
Starting the auth over helps but the problem continues.

Please be aware that the db and game cores need to advertise an IP address that clients can use in order to reach (directly or via NAT) the servers. Automatic IP address detection based on RFC1918 addresses does not always work (and IIRC I broke compatibility with Windows as I plan to rewrite config.cpp to use environment variables anyway).

Therefore, I think you have to manually set the IP addresses in CONFIG; given your machine's IP of 192.168.1.123, try setting:

INTERNAL_IP: 192.168.1.123
PUBLIC_IP: 192.168.1.123

 

Just keep in mind that my goal was to make this run on Linux, I didn't really have cross-platform and cross-architecture compatibility in mind, even though it would be amazing. And based on your comments, it actually seems in the realm of possibility, so I might have a crack at it.

[wolfhelm 2]

I can't sleep and I'm just looking through your source code. As for the variable types, I see that you have changed the long dword etc. correctly. See that there are variables left such as time_t size_t which still depend on the architecture.

I like the project, I wonder if I could help you with it. Do you have a discord server for this project where the community could gather?

[Exynox 1538]

22 hours ago, wolfhelm said:

I can't sleep and I'm just looking through your source code. As for the variable types, I see that you have changed the long dword etc. correctly. See that there are variables left such as time_t size_t which still depend on the architecture.

I like the project, I wonder if I could help you with it. Do you have a discord server for this project where the community could gather?

I'm aware about time_t, size_t and their friends, as I've had to change them to make the packet sizes match. I suppose there might be some sections of code where I've overlooked them.

There was no need for a discord server, but I've created one just now, please find an invitation in your PMs.

[Gurgarath 2447]

On 12/14/2023 at 2:19 AM, wolfhelm said:

If you want a better native version of the network stack for Linux, I can send it to you. I have a tested stack on epoll with 1k+ players.
 

https://discordapp.com/users/699723160467144777

Epoll example version: 

#define __LIBTHECORE__

#include "stdafx.h"

fdwatch *fdwatch_new() {
    fdwatch *fdw;
    CREATE(fdw, fdwatch, 1);

    fdw->fd = epoll_create1(0);

    if (fdw->fd == -1) {
        sys_err("%s", strerror(errno));
        return nullptr;
    }

    CREATE(fdw->events, epoll_event, 256);

    return fdw;
}

void fdwatch_delete(fdwatch *fdw) {
    close(fdw->fd);
    free(fdw->events);
    free(fdw);
}

int32_t fdwatch_count(fdwatch *fdw) {
    int32_t num = epoll_wait(fdw->fd, fdw->events, 256, 0);
    if (num == -1) {
        fprintf(stderr, "%s\n", strerror(errno));
    }
    return num;
}

void fdwatch_add_fd(fdwatch *fdw, socket_t fd, void *client_data) {
    TFDWatchData *data;
    CREATE(data, TFDWatchData, 1);

    data->client_data = client_data;
    data->fd = fd;

    epoll_event event{};
    event.events = EPOLLIN | EPOLLOUT;
    event.data.ptr = data;

    if (epoll_ctl(fdw->fd, EPOLL_CTL_ADD, fd, &event)) {
        fprintf(stderr, "%s\n", strerror(errno));
    }
}

void fdwatch_del_fd(fdwatch *fdw, socket_t fd) {
    epoll_ctl(fdw->fd, EPOLL_CTL_DEL, fd, nullptr);
}

int32_t fdwatch_check_event(fdwatch *fdw, socket_t fd, int32_t event_idx) {
    auto *data = (TFDWatchData *) fdw->events[event_idx].data.ptr;

    if (data->fd == fd) {
        return fdw->events[event_idx].events;
    } else {
        return 0;
    }
}

void *fdwatch_get_client_data(fdwatch *fdw, int32_t event_idx) {
    auto *data = (TFDWatchData *) fdw->events[event_idx].data.ptr;
    return data->client_data;
}

#ifndef __INC_LIBTHECORE_FDWATCH_H__
#define __INC_LIBTHECORE_FDWATCH_H__

#include <sys/epoll.h>

struct fdwatch {
    socket_t fd;
    epoll_event *events;
};

typedef struct FDWatchData {
    socket_t fd;
    void *client_data;
} TFDWatchData;

extern fdwatch *fdwatch_new();

extern void fdwatch_delete(fdwatch *fdw);

extern int32_t fdwatch_check_event(fdwatch *fdw, socket_t fd, int32_t event_idx);

extern void fdwatch_add_fd(fdwatch *fdw, socket_t fd, void *client_data);

extern int32_t fdwatch_count(fdwatch *fdw);

extern void *fdwatch_get_client_data(fdwatch *fdw, int32_t event_idx);

extern void fdwatch_del_fd(fdwatch *fdw, socket_t fd);


#endif

 

 

It would frankly be excellent to create a branch with epoll! There are high chances it would be more performant than libevent and eventually libev too as this is the base for both, despite losing a few feature (that in my opinion are not needed in our case anyway). I am glad to see this project gaining traction.

--

About size_t and time_t, as long as those are not sent directly to the client (aka not in packet.h / input_main.cpp ...), it should be fairly fine. I haven't checked the usage in those sources so far but in my case I simply converted them to signed and unsigned 32bit integer (unless one doesn't use 32bit time_t on the client which is highly, highly, highly unlikely).

 

[shh 30]

You guys should also consider raknet, or maybe enet.. it needs to be a game network lib..

Edited December 17, 2023 by shh

[wolfhelm 2]

3 hours ago, shh said:

You guys should also consider raknet, or maybe enet.. it needs to be a game network lib..

Can you show the advantages of raknet over epoll/libevent vversion of protocol in this case?

[Sokus 0]

I really like this idea, I host a server just for me and my friends so the "Imagine being able to take a small Raspberry Pi and host a Metin2 server on it" part really speaks to me

I have experience with C, networking and old school gamedev with my own projects but never contributed to anything bigger so I would like to learn some more - if that is okay I would also like the discord server invitation

[Helia01 2066]

awesome!

[Ridleypr 1]

Thanks for your post. That was exactly what I was looking for.

[.NyanCat 22]

I just learned that on most modern ARM cpus char is unsigned.
X86 cpus handle the char as signed in most cases.
I guess thats the problem with those strange behaviors on my M1-Mac.

I will test the -fsigned-char compiler flag and let you know if that helps.

[Sbonkers 0]

@ Exynox do you have a Discord channel where we can all gather to discuss about this project? I would really like to help!

[Hyu 2]

Just wondering, does this have all the latest fixes from TPM like the item duplication fix?

[.NyanCat 22]

On 2/17/2024 at 9:04 PM, Hyu said:

Just wondering, does this have all the latest fixes from TPM like the item duplication fix?

No currently there are no fixes inclued.

[Exynox 1538]

I'm happy to announce a small update!

All the old logging functions, printfs, etc. were removed in favour of spdlog. A small performance improvement might be felt, as in release mode, trace and debug messages are not compiled into the final binary, therefore avoiding function calls and expensive string parsing.

Also, thanks to @.NyanCat, the server now successfully runs on arm64! The main advantage to this I can think for now is lower hosting costs due to the increased efficiency of the ARM architecture. For example, Hetzner rents an 8-core (shared) ARM VM with 16GB of RAM for 12,49€, whereas an equivalent x86 server goes for 25,20€ (source). You don't need an Ampere CPU; an M-series Apple processor or even something like a Rock 5B could make for a very energy efficient server for a few players.

Now I'm wondering if it's possible to have this run decently on a Raspberry Pi...

 

 

[Helia01 2066]

It remains to make a cross-platform client

[Exynox 1538]

21 hours ago, Helia01 said:

It remains to make a cross-platform client

That would be really nice to do - the issue with that, though, is that there are dependencies that only work on Windows (DirectX & Miles), and these would need to be replaced. There are others that we do have public Linux SDKs for (Speedtree, Granny), so at least that would alleviate some work, even though it would be nice to have a fully "open-source" client. It's nonetheless a massive undertaking.

[Gurgarath 2447]

Having an open source client would be excellent but the amount of work is crazy. Using OpenGL or Vulkan would be an alternative to DX, and a good one at that. But the amount of work is crazy for only a few % of the players (I am not taking into account the benefits it would bring to the game who are increasingly interesting other than cross-platform). Those players will usually just run Wine or a VM to make the game run, which shifts the compatibility work to the player