M2 Docker, Redis, haproxy? (permanent instance)

[Deso 992]

Hey,
i want to start a new project and was thinking about a to use a new technology.

i want to try to open the server and when i want to make updates (core replace) or a core crashes, to swap all user datas /ping/pong and so on to an other instance - the player shouldnt be disconnected (the player should be still stay on the server while we're swaping the core or when a crash happen that the core swap.

firstly i was thinking about to use kubernetes and other technolgies. but i think its not easy because of the ping/pong and the userdatas.
now i was thinking about to use an external cache (redis) to put all datas in the external cache.
also im thinking about to use HAproxy as load balancing and notice when an instance went down to swap to an other core
-> 1 server (active) 2 server (active standby)

Someone have experience if it could work? or maybe someone made it already done?
would be great if you can share your expierence.

Thanks

[b6d4a82c15 296]

Overall its possible and it was done by a few different people... and the tech stack doesnt really matter. You don't even need all of these, especially k8s. This can be achieved by editing solely server code

[Deso 992]

1 hour ago, b6d4a82c15 said:

Overall its possible and it was done by a few different people... and the tech stack doesnt really matter. You don't even need all of these, especially k8s. This can be achieved by editing solely server code

glad to hear!
you know especially which methods needs to make it work? i can lean them myself, but i usually need only informations which way it should go, because im thinking about some weeks about this topic - i just found out this method, but im not clear if its 100% possible, because of the 1ms downtime and redis cache to swap to dont disconnect the player (just this case is not sure)

thanks for your input

[Metin2 Dev 12217]

Redis, I understand. I moved to redis long time ago after a db crash. But why haproxy and moving the player on another instance ? Why do you want to work a lot to do this change instead of fixing the crashes. From my experience I can tell you crash cores are disaprearing completly after several months of online server if you solve them one by one. I recomend you start your game/db in test mode with ASAN for 10-15 players and after each exception just solve it.

[Deso 992]

11 hours ago, Abel(Tiger) said:

Redis, I understand. I moved to redis long time ago after a db crash. But why haproxy and moving the player on another instance ? Why do you want to work a lot to do this change instead of fixing the crashes. From my experience I can tell you crash cores are disaprearing completly after several months of online server if you solve them one by one. I recomend you start your game/db in test mode with ASAN for 10-15 players and after each exception just solve it.

i run already my server around 1 year with asan and in last half year with some testers, i still fixed all upcoming issues i got - thats not the main reason.
temporary i want to try to run a server with 'zero' downtime, it should be a very special service to the clients - i want to make updates without a disconnect of an player.
in same point if somehow a crash should also happen, it should save the player datas to avoid some last made things (before cashing happen).

HAProxy because i was thinking firstly about nginx but the respond time is ~ 2 seconds and i've heared haproxy respond in ~ 0,5 seconds.
it should make health checks to notice if an docker instance went down to swap to the "active standby" instance and catch the redis cache datas before the core crashed.

this is just a thinking. im not 100% sure if it would work, because im not sure if the method with haproxy could work.
In that case i opened this topic to get maybe some informations/new ideas to make that project work like i said.

thanks for your time guys.
Hope some other ideas will come too 

[Deso 992]

no one have ideas? :D

[Koray 3316]

Kubernetes feels like overkill for what you are trying to do. It adds a layer of complexity that might not be worth it, especially if you are building from scratch. so I never recommend it but... if you are set on using it, take a look at Agones - it's specifically designed for game servers and could help make your system more scalable.

HAProxy isn't really the best fit for current M2 game server arch either. It's better suited for different types of scalable services. What I did suggest instead is separating your authentication process from the game server entirely. Make it a standalone app using something web-friendly like JS/TS and then you can consider HAProxy if needed. Though honestly, I did probably just go serverless with something like CF Workers/Spectrum rather than building out all that infrastructure.

Now about that "zero downtime" goal, also it's just not feasible with current M2 game server arch. You did need to rebuild your entire server arch. Here is a simplified version of what you might want to consider:
- At the first, you need a master server that oversees all connections and sub-servers.
- For your database needs, ditch the "stateful" approach. Go with something that can scale alongside your authentication system, works well with multiple processes, and keeps the actual data in a cache based database like Redis. This makes your system "stateless" which is a good thing for scalable infrastructure.
- For game channels, each one needs its own alt clone aka. live shadow instances. These should regularly save snapshots of the server state, especially during crashes. Using RPC, they can transfer this data to sub applications which then become the new "main" servers, spinning up their own backups and continuing the game. The tricky part is maintaining those user connections during the switch aka. "traffic shadowing", that's where a load balancer like Envoy or HAProxy comes in handy.

Since current channel structure wasn't designed for scalability, a simple fix would be running each channel as its own game app or maybe adopting a shard system similar to what WoW uses.
For updates, consider a blue/green deployment approach, set up new channels for the updated version and direct new authentication requests there, keeping them separate from active channels.

Once you have got all this in place, you can manage everything through Docker without needing k8s at all...