What better performance for queries and packets?

[valdirk2 81]

Hello guys, I would like opinions from everyone.

I'm setting up a global store for metin, a kind of auction, to retire offline stores.

My idea is for players to advertise items they want to sell for a certain price, and these items are stored in a database.

But my doubt is the following, what would be the most performative way to make this system, in relation to storing the items that are for sale. When the player opens the global store, simply the "game" sends several packets with all items currently available for sale (can be 1 to 1000kk of items in the database, but all summarized in just one query) and in pyhton i filter the items that will be displayed and paged to the player.

Or send these items as needed, that is, every time the player searches for an item name or changes the page, the client asks for a request for 10 new items at a time, but this would cause a good amount of query to be processed. Each request would be necessary to always make a query to the database and return the items to players.

I'm really in doubt which way would be more performatica, let the client get by with a large amount of data, or the "game" will send requests as needed.

Edited August 18, 2022 by Metin2 Dev
Core X - External 2 Internal

[Sonitex 1406]

Sending a request for every 10 items to the game core does not sound very practical. I cannot imagine how it would even look like as you would meet unnecessary issues like which items have been sent, which items have been removed/changed but not sent yet etc. and with monkeys on the other side of the client it would result in numerous for loops.

I would recommend sending a large packet containing every item that meets the search criteria and then categorize them as you want on the client-side.

[valdirk2 81]

3 minutes ago, Sonitex said:

Sending a request for every 10 items to the game core does not sound very practical. I cannot imagine how it would even look like as you would meet unnecessary issues like which items have been sent, which items have been removed/changed but not sent yet etc. and with monkeys on the other side of the client it would result in numerous for loops.

I would recommend sending a large packet containing every item that meets the search criteria and then categorize them as you want on the client-side.

I really thought about prioritizing performance in the "game". Whenever someone opens the item shop they send all the items currently in the shop available. Then the customer is concerned with filtering the items.
And when any player adds an item it sends to all players connected the packet with the new item, without the need for any other query. Just the initial query when someone requests the items.

[Sonitex 1406]

5 minutes ago, valdirk2 said:

I really thought about prioritizing performance in the "game". Whenever someone opens the item shop they send all the items currently in the shop available. Then the customer is concerned with filtering the items.
And when any player adds an item it sends to all players connected the packet with the new item, without the need for any other query. Just the initial query when someone requests the items.

If the game can handle such large data go for it I guess. Much better than sending a fixed amount of items to the user.

[valdirk2 81]

14 minutes ago, Sonitex said:

If the game can handle such large data go for it I guess. Much better than sending a fixed amount of items to the user.

I thought to store all this in a variable in python. Could python handle working with a variable with 100k of items?

It would make a lot of difference in memory.

[Sonitex 1406]

4 minutes ago, valdirk2 said:

I thought to store all this in a variable in python. Could python handle working with a variable with 100k of items?

It would make a lot of difference in memory.

Python is really slow compared to C++ so you should do the filtering part inside the binary and forward the data to Python and do the visual parts there. 

[valdirk2 81]

3 minutes ago, Sonitex said:

Python is really slow compared to C++ so you should do the filtering part inside the binary and forward the data to Python and do the visual parts there. 

So there's something I'll have to research, because I don't know how to store this variable inside the binary.

But thanks for the tips, now I know what path to take to create this system, I'm not very good with c++, but I'm learning.

I also want to see how the cache system works inside the game, maybe I can do something there.

[valdirk2 81]

On 6/10/2021 at 5:53 PM, Sonitex said:

Python is really slow compared to C++ so you should do the filtering part inside the binary and forward the data to Python and do the visual parts there. 

So, I believe I got a good result, but I'll still need to test it in production. To avoid the customer receiving too many packages at once, when the player opens the global store, I create an EVENTFUNC, which every 1 second sends a package containing 100 items that the global store has.
I did some tests and apparently it's working fine.
I want to see how it will be in production hahaha

[Cunoo 209]

 

[valdirk2 81]

On 6/25/2021 at 10:33 AM, Cunoo said:

 

Unfortunately in this case he used quest as a system mediator. I already made the system using packet communications and 70% of the system in the server and client's own source.
So far I haven't tested in production, but the tests I did were good.