fix Initial Entity Meeting Freeze Fix

[Amun 3224]

The client is freezing for a while(anywhere from a few ms to multiple seconds for slower PCs) when meeting a new NPC/enemy for the first time because that's when it registers, creates and loads the entity.

Note: This is only a partial fix.

Why is it partial? Because it only loads the static mobs and NPCs available in the map(npc, regen, boss, group and group_group), so it won't fix the problem for dynamic entities, like pets and mounts.

It can be extended to load those as well, of course, but you'll have to take care of that on your own.

Variant:

A way of completely fixing this would be to create the instances and load their files after registering the path(in root->playerSettingModule->LoadGameNPC), BUT that would slow down the initial loading quite a bit, since it'll load ALL mobs/npcs/pets/mounts, even if most of them will never be used.

There's advantages and drawbacks to both methods. Choose for yourself.

 

Here's the link:

• [or] 
  This is the hidden content, please

  • Sign In
  • or
  • Sign Up
• [or] 
  This is the hidden content, please

  • Sign In
  • or
  • Sign Up

 

Good luck!

- Amun

Edit:

Spoiler

PS: Fuck off - NO SUPPORT.

PSS: Another reason why the client is freezing is because of granular loading of maps. Implement the first step of @ masodikbela experimental full terrain render and you'll get rid of that as well.

Here's the link(Show some love while you're there):

 

 

 UPDATE 08/12/2022:

1. Refactored to use unordered_set from the beginning instead of creating a vector, then filtering unique entities by creating a set, and then moving them back to a vector.

2. Extended to load the full spectrum of entities(both static and dynamic), except for PC(which are loaded in loading phase), WARP, GOTO, and DOOR. They can be included by altering this block in char_manager.cpp:

#ifdef ENABLE_ENTITY_PRELOADING
	//@Amun: or be specific, like if(ch->ispet, mount, whatever)
	if (!ch->IsPC() && !ch->IsGoto() && !ch->IsWarp() && !ch->IsDoor())
		SECTREE_MANAGER::Instance().ExtendPreloadedEntitiesMap(lMapIndex, pkMob->m_table.dwVnum);
#endif

 

Edited December 8, 2022 by Amun
Updated the code/amended some details.

[Artvr3 20]

Thank you @ Amun for sharing with the community every optimization option is useful

[Helia01 3409]

ty 

[Tatsumaru 7368]

Edited October 18, 2022 by Metin2 Dev
Core X - External 2 Internal

[Amun 3224]

Topic/code updated. Apologies for the bump.

[Euzebiusz 0]

Can you make a recording of what the situation looks like before adding the code and what the situation looks like after adding the code?

[Amun 3224]

19 minutes ago, Euzebiusz said:

Can you make a recording of what the situation looks like before adding the code and what the situation looks like after adding the code?

no

[Gurgarath 3698]

1 hour ago, Euzebiusz said:

Can you make a recording of what the situation looks like before adding the code and what the situation looks like after adding the code?

Before: Consistent stuttering upon meeting an unloaded entity or walking through the map

After: Minimal to no stuttering upon loading a chunk of the map

[Dr3Ame3r 36]

Tested, it works, but, with a slight little problem, after the loading phase ends (the loadingscreen photos disappear) and it should popup your character, it freezes for about 0.7-1s until it shows entities near you(that happens only 1 time for each client)

[Amun 3224]

On 2/26/2023 at 10:08 AM, Dr3Ame3r said:

Tested, it works, but, with a slight little problem, after the loading phase ends (the loadingscreen photos disappear) and it should popup your character, it freezes for about 0.7-1s until it shows entities near you(that happens only 1 time for each client)

Yes, and I said that in the readme. If you want to avoid that, load npclist ahead of time and send the entities before the game world is shown. I didn't bother to make that as well because I wanted to keep it simple, so that everyone(or most people) can understand what's going on.

 

If you consider that to be a big problem and have the knowledge to do what I said earlier, you can do it and post it here, so that everyone can use. Unfortunately, I'm extremely busy at the moment, so I won't be able to work on community stuff for a while.

 

Cheers!

- Amun

[Viello 135]

I had some teleport issues, sometimes client crashes, no syserr maybe this code is not perfect for my files, btw thanks for sharing with community.

[Vaynz 2687]

1 hour ago, Viello said:

I had some teleport issues, sometimes client crashes, no syserr maybe this code is not perfect for my files, btw thanks for sharing with community.

I assure you the code is great coded, i tested it over 2 weeks and i dont had any problems with my testers, be sure you didn't messed up something or cause is from another system or bad code. As an advice try to have permanently debugging maybe you will find the problem. Good luck !

[gleskpem1 5]

Spoiler

0505 21:47:51889 :: CPythonCharacterManager::CreateInstance: VID[0] - ALREADY EXIST

0505 21:47:51910 :: Failed to preload race 192
0505 21:47:52341 :: CPythonCharacterManager::CreateInstance: VID[0] - ALREADY EXIST

0505 21:47:52341 :: Failed to preload race 191
0505 21:47:52489 :: CPythonCharacterManager::CreateInstance: VID[0] - ALREADY EXIST

0505 21:47:52489 :: Failed to preload race 111
0505 21:47:52646 :: CRaceManager::GetRaceDataPointer: cannot load data by dwRaceIndex 20025
0505 21:47:52646 :: CPythonCharacterManager::CreateInstance VID[20025] Race[20025]
0505 21:47:52646 :: Failed to preload race 20025
0505 21:47:53121 :: CPythonCharacterManager::CreateInstance: VID[0] - ALREADY EXIST

0505 21:47:53121 :: Failed to preload race 144
0505 21:47:53556 :: CPythonCharacterManager::CreateInstance: VID[0] - ALREADY EXIST

0505 21:47:53556 :: Failed to preload race 151
0505 21:47:53556 :: CPythonCharacterManager::CreateInstance: VID[0] - ALREADY EXIST

0505 21:47:53556 :: Failed to preload race 154
0505 21:47:53775 :: CPythonCharacterManager::CreateInstance: VID[0] - ALREADY EXIST

0505 21:47:53775 :: Failed to preload race 153
0505 21:47:53777 :: CPythonCharacterManager::CreateInstance: VID[0] - ALREADY EXIST

0505 21:47:53777 :: Failed to preload race 184
0505 21:47:54086 :: CPythonCharacterManager::CreateInstance: VID[0] - ALREADY EXIST

0505 21:47:54086 :: Failed to preload race 152
0505 21:47:54099 :: CPythonCharacterManager::CreateInstance: VID[0] - ALREADY EXIST

0505 21:47:54099 :: Failed to preload race 194
0505 21:47:54134 :: CPythonCharacterManager::CreateInstance: VID[0] - ALREADY EXIST

0505 21:47:54134 :: Failed to preload race 176
0505 21:47:54261 :: CPythonCharacterManager::CreateInstance: VID[0] - ALREADY EXIST

0505 21:47:54261 :: Failed to preload race 182
0505 21:47:54366 :: CPythonCharacterManager::CreateInstance: VID[0] - ALREADY EXIST

0505 21:47:54366 :: Failed to preload race 155
0505 21:47:54435 :: CPythonCharacterManager::CreateInstance: VID[0] - ALREADY EXIST

0505 21:47:54435 :: Failed to preload race 193
0505 21:47:54435 :: CPythonCharacterManager::CreateInstance: VID[0] - ALREADY EXIST

0505 21:47:54435 :: Failed to preload race 142
0505 21:47:54435 :: CPythonCharacterManager::CreateInstance: VID[0] - ALREADY EXIST

0505 21:47:54435 :: Failed to preload race 183
0505 21:47:54702 :: CPythonCharacterManager::CreateInstance: VID[0] - ALREADY EXIST

0505 21:47:54702 :: Failed to preload race 179
0505 21:47:54703 :: CPythonCharacterManager::CreateInstance: VID[0] - ALREADY EXIST

0505 21:47:54703 :: Failed to preload race 180
0505 21:47:54926 :: CPythonCharacterManager::CreateInstance: VID[0] - ALREADY EXIST

0505 21:47:54926 :: Failed to preload race 110
0505 21:47:55025 :: CPythonCharacterManager::CreateInstance: VID[0] - ALREADY EXIST

0505 21:47:55025 :: Failed to preload race 112
0505 21:47:55025 :: CPythonCharacterManager::CreateInstance: VID[0] - ALREADY EXIST

i see this when i move randomly is it a problem or what does it mean?

Edited May 5, 2023 by gleskpem1
d

[Koray 3328]

I would like to provide feedback based on my experience from a few months ago. On some maps with many entities (like metin farm maps), there is an issue where the screen stays black for 3-4 seconds at the map loading phase.

The recorded instances are transferred from the server as race numbers, but the client uses VID within the

This is the hidden content, please

• Sign In
• or
• Sign Up

. If that VID has been used before or will be used later to save an entity, it throws the error "CPythonCharacterManager::CreateInstance: VID[%d] - ALREADY EXIST," and most of the new entitys becomes invisible on the client.

Additionally, on maps with a high number of entities, during the startup phase, there is sometimes an "unknown packet header" error. I haven't fully understood the cause of this, but it is likely that "pack.count = s.size();" just probably returns a value +1 or is not properly cast, causing it to reset when reaching the uint16_t limit.

Edited July 12, 2024 by Koray

[Amun 3224]

4 minutes ago, Koray said:

I would like to provide feedback based on my experience from a few months ago. On some maps with many entities (like metin farm maps), there is an issue where the screen stays black for 3-4 seconds at the map loading phase.

The recorded instances are transferred from the server as race numbers, but the client uses VID within the

This is the hidden content, please

• Sign In
• or
• Sign Up

. If that VID has been used before or will be used later to save an entity, it throws the error "CPythonCharacterManager::CreateInstance: VID[%d] - ALREADY EXIST," and most of the new entitys becomes invisible on the client.

Additionally, on maps with a high number of entities, during the startup phase, there is sometimes an "unknown packet header" error. I haven't fully understood the cause of this, but it is likely that "pack.count = s.size();" just probably returns a value +1 or is not properly cast, causing it to reset when reaching the uint16_t limit.

Thanks for sharing, but I have a few things to point out on this

 

The black screen:

You weren't supposed to load them in the game phase. I've only done it this way to keep it simple and for everyone to know where to find the code and do the initial changes. You were supposed to load them in the loading phase, along with all the other things.

 

VID Already exist: Don't know what to say about this one right now, but I'll have a look at it when I got some free time(which probably won't be very soon..)

 

As for the last problem, it's unlikely that you'll ever hit the upper limit of uint16_t. There's space for 65k races, you wouldn't hit it even if you sent the entire mob list * 20, so I tend to think your problem is coming from some other place.

[msmajda9 35]

[CLIENT SIDE]
don't forget add in packet.h:
#ifdef ENABLE_ENTITY_PRELOADING
#include <stdint.h>
#endif

if you have problem with this:
typedef struct packet_preload_entities
{
    uint8_t header;
    uint16_t size;
    uint16_t count;
} TPacketGCPreloadEntities;
#endif

[SERVER SIDE]
And if you use old server src you need upgrade gcc to 4.8 or bigger than 4.8
after it you need upgrade source code for C++11 compatible.
Upgrade:
https://ftp.gwdg.de/pub/misc/gcc/releases/ select version and download with browser.
upload to root folder with SFTP.
putty commands:
tar -xzvf gcc-4.8.0.tar.gz
cd gcc-4.8.0
./configure --prefix=/usr/local/gcc48 --enable-languages=c,c++

(how many cores you have in Server src compiler machine? if you have 4 cpu core(mag) modify -j3 to -j5
i have 2 core in my virtual machine so i set -j3)
gmake -j3
gmake install

When you finish total:
setenv PATH /usr/local/gcc48/bin:$PATH

ln -sf /usr/local/gcc48/bin/gcc /usr/local/bin/gcc
ln -sf /usr/local/gcc48/bin/g++ /usr/local/bin/g++

Check your gcc version:
gcc --version

Open makefile and add on new line after "CFLAGS =" this:
CFLAGS += -std=c++11  -fpermissive

Now your compiler using C++11 so you need rewrite your source code.

 

Edited August 30, 2025 by msmajda9
pretty print

[HAZEJ 251]

PythonNetworkStreamPhaseGame.cpp

add:
#ifdef ENABLE_ENTITY_PRELOADING
extern bool IsInvisibleRace(WORD raceNum);
#endif

change:
 

#ifdef ENABLE_ENTITY_PRELOADING
bool CPythonNetworkStream::RecvPreloadEntitiesPacket()
{
    TPacketGCPreloadEntities pack{};
    if (!Recv(sizeof(pack), &pack))
        return false;

    assert(int32_t(pack.size) - sizeof(pack) == pack.count * sizeof(uint32_t) && "HEADER_GC_PRELOAD_ENTITIES");

    uint32_t dwEntityRace = 0;
    CInstanceBase::SCreateData d{};
    static uint32_t s_dwPreloadVID = 0xC0000000;
    CPythonCharacterManager& rkChrMgr = CPythonCharacterManager::Instance();

    for (uint16_t i = 0; i < pack.count; i++)
    {
        if (!Recv(sizeof(uint32_t), &dwEntityRace))
            return false;

#ifdef _DEBUG
        TraceError("Preloading %d", dwEntityRace);
#endif
        if (IsInvisibleRace(static_cast<WORD>(dwEntityRace)))
            continue;

        d.m_dwRace = dwEntityRace;

        const uint32_t dwPreloadVID = s_dwPreloadVID++;
        if (s_dwPreloadVID < 0xC0000000)
            s_dwPreloadVID = 0xC0000000;

        d.m_dwVID = dwPreloadVID;

        if (rkChrMgr.CreateInstance(d))
        {
            rkChrMgr.DeleteInstance(dwPreloadVID);
        }
        else
        {
            TraceError("Failed to preload race %u", dwEntityRace);
        }
    }

    return true;
        }
#endif

sectree_manager.cpp

change:

namespace
{
    bool IsPreloadableRace(uint32_t raceVnum)
    {
        if (!raceVnum)
            return false;

        const CMob* pkMob = CMobManager::instance().Get(raceVnum);
        if (!pkMob)
            return false;

        return pkMob->m_table.szFolder[0] != '\0';
    }

    bool ShouldCollectForPreload(LPENTITY ent)
    {
        if (!ent || !ent->IsType(ENTITY_CHARACTER))
            return false;

        LPCHARACTER pkChr = static_cast<LPCHARACTER>(ent);

        if (pkChr->IsPC() || pkChr->IsPet())
            return false;

#ifdef ENABLE_MOUNT_COSTUME_SYSTEM
        if (pkChr->IsMount())
            return false;
#endif

        if (pkChr->IsWarp() || pkChr->IsGoto())
            return false;

        return IsPreloadableRace(pkChr->GetRaceNum());
    }
}

void SECTREE_MANAGER::GeneratePreloadedEntitiesMap(int32_t mapIndex, LPSECTREE_MAP lpMapSectree)
{
    if (m_preloadedEntities.find(mapIndex) == m_preloadedEntities.end())
    {
        m_preloadedEntities.insert({ mapIndex, {} });

        auto lmbd = [&mapIndex, this](LPENTITY ent)
            {
                if (!ShouldCollectForPreload(ent))
                    return;

                auto* pkChr = static_cast<LPCHARACTER>(ent);
                this->m_preloadedEntities.at(mapIndex).emplace(pkChr->GetRaceNum());
            };

        lpMapSectree->for_each(lmbd);
        for (int32_t i : m_preloadedEntities.at(mapIndex))
            sys_log(0, "ENTITY_PRELOADING: [map: %d], [entity race: %d]", mapIndex, i);
    }
}

void SECTREE_MANAGER::ExtendPreloadedEntitiesMap(int32_t mapIndex, uint32_t mob_race)
{
    auto it = m_preloadedEntities.find(mapIndex);
    if (it == m_preloadedEntities.end())
        return;

    if (!IsPreloadableRace(mob_race))
        return;

    if ((it->second.emplace(mob_race)).second)
        sys_log(0, "ENTITY_PRELOADING: extended [map: %d], with [entity race: %d]", mapIndex, mob_race);
}
 

[RobinHoodye 3]

some fix for freezing at the beginning when connecting or teleporting to another map?

[CONTROL 922]

The idea here is amazing.

However, I made a few improvements to make the system cleaner and solve some edge cases.

 

char_manager.cpp

Spoiler

#ifdef ENABLE_ENTITY_PRELOADING
	//@Amun: or be specific, like if(ch->ispet, mount, whatever)
	if (!ch->IsPC() && !ch->IsGoto() && !ch->IsWarp() && !ch->IsDoor() && !ch->IsPet()
	)
	{
		SECTREE_MANAGER::Instance().ExtendPreloadedEntitiesMap(lMapIndex, pkMob->m_table.dwVnum);
#ifdef ENABLE_ENTITY_PRELOADING_EXTRA // add this
		if (ch->IsStone())
			PreloadStoneGroup(lMapIndex, pkMob->m_table.sAttackSpeed, pkMob->m_table.sMovingSpeed);
		else
			PreloadBossGroup(ch, lMapIndex);
#endif
	}
#endif

 

 

By the way, IsPet() is actually useless here if it's implemented like this:

bool IsPet() const { return m_bIsPet; }

because m_bIsPet gets assigned after:

CHARACTER_MANAGER::instance().SpawnMob(...)

has already executed, meaning it will always return false at this point anyway

Unless you have

bool IsPet() const { return m_bCharType == CHAR_TYPE_PET; }

Either way, it’s harmless—just a single boolean check.

 

 

add to char_manager.cpp

Spoiler

#ifdef ENABLE_ENTITY_PRELOADING_EXTRA
void CHARACTER_MANAGER::PreloadStoneGroup(uint32_t lMapIndex, uint32_t StartVnum, uint32_t EndVnum)
{
	CMobGroup * pkGroup;
	const std::vector<DWORD> * pkMembers;
	for (int s = StartVnum; s < EndVnum; s++)
	{
		pkGroup = CMobManager::Instance().GetGroup(s);
		if (pkGroup) 
		{
			pkMembers = &pkGroup->GetMemberVector();
			for (int i = 0; i < pkMembers->size(); ++i)
				SECTREE_MANAGER::Instance().ExtendPreloadedEntitiesMap(lMapIndex, (*pkMembers)[i]);
		}
	}
}

void CHARACTER_MANAGER::PreloadBossGroup(LPCHARACTER ch, uint32_t lMapIndex)
{
	if (!ch) {return;};
	uint32_t SummonVnum = ch->GetSummonVnum();
	if (SummonVnum <= 0)
		return;
	SECTREE_MANAGER::Instance().ExtendPreloadedEntitiesMap(lMapIndex, SummonVnum);
}
#endif

 

 

add to char_manager.h

Spoiler

#ifdef ENABLE_ENTITY_PRELOADING_EXTRA
	protected:
		void 	PreloadStoneGroup(uint32_t lMapIndex, uint32_t StartVnum, uint32_t EndVnum);
		void 	PreloadBossGroup(LPCHARACTER ch, uint32_t lMapIndex);
#endif

 

 

One of the biggest issues was where the preload packet gets sent.

Originally it was sent inside:

input_login.cpp → EnterGame()

That’s an awkward place because the client is already entering the world.

Instead, I moved it into:

input_db.cpp → PlayerLoad()

 

move this from input_login.cpp to input_db.cpp inside PlayerLoad

Spoiler

long lPublicMapIndex = lMapIndex >= 10000 ? lMapIndex / 10000 : lMapIndex;

	if (!map_allow_find(lPublicMapIndex))
	{
		sys_err("InputDB::PlayerLoad : entering %d map is not allowed here (name: %s, empire %u)",
				lMapIndex, pTab->name, d->GetEmpire());

		ch->SetWarpLocation(EMPIRE_START_MAP(d->GetEmpire()),
				EMPIRE_START_X(d->GetEmpire()) / 100,
				EMPIRE_START_Y(d->GetEmpire()) / 100);

		d->SetPhase(PHASE_CLOSE);
		return;
	}

#ifdef ENABLE_ENTITY_PRELOADING
    SECTREE_MANAGER::Instance().SendPreloadEntitiesPacket(ch); // right here
#endif

 

 

 

 

sectree_manager.cpp

Spoiler

#ifdef ENABLE_ENTITY_PRELOADING_EXTRA // add this
#include "mob_manager.h"
#endif



#ifdef ENABLE_ENTITY_PRELOADING // after this
			GeneratePreloadedEntitiesMap(iIndex, pkMapSectree);
#endif
#ifdef ENABLE_ENTITY_PRELOADING_EXTRA // add this
			GeneratePreloadedGroupsMap(iIndex, pkMapSectree);
#endif







// replace completely
#ifdef ENABLE_ENTITY_PRELOADING
void SECTREE_MANAGER::GeneratePreloadedEntitiesMap(int32_t mapIndex, LPSECTREE_MAP lpMapSectree)
{
    if (m_preloadedEntities.find(mapIndex) == m_preloadedEntities.end())
    {
        m_preloadedEntities.emplace(mapIndex, std::unordered_set<uint32_t>());
        
        struct EntityProcessor 
		{
            SECTREE_MANAGER* manager;
            int32_t mapIndex;
            
            void operator()(LPENTITY ent) 
			{
                LPCHARACTER character = static_cast<LPCHARACTER>(ent);
                manager->m_preloadedEntities[mapIndex].emplace(character->GetRaceNum());
            }
        };
        
        EntityProcessor processor = {this, mapIndex};
        lpMapSectree->for_each(processor);
    }
}

void SECTREE_MANAGER::ExtendPreloadedEntitiesMap(int32_t mapIndex, uint32_t mob_race)
{
    auto it = m_preloadedEntities.find(mapIndex);
    if (it == m_preloadedEntities.end())
        return;
    
    if (it->second.emplace(mob_race).second)
        sys_log(0, "ENTITY_PRELOADING: extended [map: %d], with [entity race: %d]", mapIndex, mob_race);
}

void SECTREE_MANAGER::SendPreloadEntitiesPacket(LPCHARACTER ch)
{
    LPDESC d = ch->GetDesc();
    if (!d)
        return;
    
    long lMapIndex = ch->GetMapIndex();
    auto it = m_preloadedEntities.find(lMapIndex);
    if (it == m_preloadedEntities.end())
        return;
    
    const std::unordered_set<uint32_t>& s = it->second;
    if (s.empty())
        return;
    
    TEMP_BUFFER buf;
    
    TPacketGCPreloadEntities pack{};
    pack.header = HEADER_GC_PRELOAD_ENTITIES;
    pack.count = s.size();
    
    for (const auto& race : s)
        buf.write(&race, sizeof(uint32_t));
    
    pack.size = sizeof(pack) + buf.size();
    d->BufferedPacket(&pack, sizeof(TPacketGCPreloadEntities));
    d->Packet(buf.read_peek(), buf.size());
}

#ifdef ENABLE_ENTITY_PRELOADING_EXTRA
void SECTREE_MANAGER::GeneratePreloadedGroupsMap(int32_t mapIndex, LPSECTREE_MAP lpMapSectree)
{
    if (m_preloadedEntities.find(mapIndex) == m_preloadedEntities.end())
    {
        m_preloadedEntities.emplace(mapIndex, std::unordered_set<uint32_t>());
        
        struct GroupProcessor 
		{
            SECTREE_MANAGER* manager;
            int32_t mapIndex;
            
            void operator()(LPENTITY ent) 
			{
                LPCHARACTER character = static_cast<LPCHARACTER>(ent);
                const CMob* pkMob = CMobManager::instance().Get(character->GetRaceNum());
                
                if (!pkMob)
                    return;
                    
                manager->m_preloadedEntities[mapIndex].emplace(character->GetRaceNum());
                
                if (pkMob->m_table.bType == CHAR_TYPE_STONE) 
				{
                    uint16_t StartVnum = pkMob->m_table.sAttackSpeed;
                    uint16_t EndVnum = pkMob->m_table.sMovingSpeed;
                    
                    for (int s = StartVnum; s < EndVnum; s++) 
					{
                        CMobGroup* pkGroup = CMobManager::Instance().GetGroup(s);
                        if (!pkGroup)
                            continue;
                            
                        const std::vector<DWORD>& members = pkGroup->GetMemberVector();
                        auto& targetSet = manager->m_preloadedEntities[mapIndex];
                        
                        for (const auto& memberId : members) {
                            targetSet.emplace(memberId);
                        }
                    }
                }
                else 
				{
                    uint32_t SummonVnum = character->GetSummonVnum();
                    if (SummonVnum > 0) {
                        manager->m_preloadedEntities[mapIndex].emplace(SummonVnum);
                    }
                }
            }
        };
        
        GroupProcessor processor = {this, mapIndex};
        lpMapSectree->for_each(processor);
    }
}
#endif
#endif

 

 

 

sectree_manager.h

Spoiler

#ifdef ENABLE_ENTITY_PRELOADING
		void 		GeneratePreloadedEntitiesMap(int32_t mapIndex, LPSECTREE_MAP lpMapSectree);
		void 		ExtendPreloadedEntitiesMap(int32_t mapIndex, uint32_t mob_race);
		void 		SendPreloadEntitiesPacket(LPCHARACTER ch);
#ifdef ENABLE_ENTITY_PRELOADING_EXTRA
		void 		GeneratePreloadedGroupsMap(int32_t mapIndex, LPSECTREE_MAP lpMapSectree);
#endif
#endif



#ifdef ENABLE_ENTITY_PRELOADING
		std::unordered_map<int32_t, std::unordered_set<uint32_t>> m_preloadedEntities;
#endif

 

 

Result

These changes provide two major improvements:

ENABLE_ENTITY_PRELOADING

Preloads all regular entities on the map exactly as before—just without the awkward black screen, since loading now happens during the actual loading screen instead of after entering the game.

 

ENABLE_ENTITY_PRELOADING_EXTRA

Additionally preloads:

Stone monsters

Boss summons

etc..

 

 

 

It’s also worth mentioning that the original CreateInstance() / DeleteInstance() approach isn’t really the right solution here.

The actual goal of entity preloading is not to fully create and destroy game instances—it’s simply to force the client to load the .gr2 resources into memory beforehand.

So instead of creating full instances just to immediately delete them, I ended up implementing a much lighter approach that only preloads the required Granny resources directly.

 

In other words:

We only preload what actually matters—the model resources themselves.

#ifdef ENABLE_ENTITY_PRELOADING
bool CPythonNetworkStream::RecvPreloadEntitiesPacket()
{
	TPacketGCPreloadEntities pack{};
	if (!Recv(sizeof(pack), &pack))
		return false;

	#ifdef _DEBUG
	const size_t expectedSize = sizeof(pack) + (size_t(pack.count) * sizeof(uint32_t));
	assert(pack.size == expectedSize && "HEADER_GC_PRELOAD_ENTITIES");
	#endif

	if (pack.count == 0)
		return true;

	std::vector<uint32_t> entityRaces(pack.count);
	if (!Recv(entityRaces.size() * sizeof(uint32_t), entityRaces.data()))
		return false;

	auto& rkRaceMgr = CRaceManager::Instance();
	for (const uint32_t dwEntityRace : entityRaces)
	{
#ifdef ENABLE_BUFFI_SYSTEM
		if (dwEntityRace == 10 || dwEntityRace == 11)
			continue;
#endif

		rkRaceMgr.PreloadRace(dwEntityRace);
	}

	return true;
}
#endif

 

unfortunately i don't have the time to share it,

but it's very simple it goes directly to the

 

CRaceData::TMotionModeDataIterator itor;
	if (!CreateMotionModeIterator(itor))
		return;

	do
	{
		CRaceData::TMotionModeData* const pMotionModeData = itor->second;
		for (const auto& rkMotionPair : pMotionModeData->MotionVectorMap)
		{
			const CRaceData::TMotionVector& rkMotionVector = rkMotionPair.second;
			for (const auto& rkMotion : rkMotionVector)
			{
				if (!rkMotion.pMotion)
					continue;

				PreloadStaticResource(rkMotion.pMotion->GetFileNameString());
			}
		}
	}
	while (NextMotionModeIterator(itor));

no fake vid no nothing

 

 

and ofc don't forget to add the

#define ENABLE_ENTITY_PRELOADING_EXTRA

 

 

thx @ Amun this is golden !

Edited May 11 by CONTROL

[Amun 3224]

Idk why I always forget to reply to topics

@msmajda9 they can just change unordered_set to set and change the types to byte/word

@ HAZEJ good catch with invisible races

@RobinHoodye yeah, move the loading to loading phase, I left it in game phase because it's simpler for people to follow

@ CONTROL interesting idea with the groups. I think(didn't check) they all use Spawn so they'll register for preloading as soon as they appear for the first time, but thanks for sharing.

This was never supposed to run in the game phase(which is why you see that delay between end of loading phase and first render), but adding it in the loading phase would've complicated the tutorial and most people just want it to work - they don't care how.

 

Funnily enough, this could've been simplified to just calling GetRaceDataPointer instead of creating an entity and discarding it, since it calls __LoadRaceData on its own when it can't find it(and it will fix the "vid already exists" shit as well).

Here's the code for copy/pasta:

#ifdef ENABLE_ENTITY_PRELOADING
bool CPythonNetworkStream::RecvPreloadEntitiesPacket()
{
	extern bool IsInvisibleRace(uint16_t raceNum);

	TPacketGCPreloadEntities pack{};
	if (!Recv(sizeof(pack), &pack))
		return false;

	assert(int32_t(pack.size) - sizeof(pack) == pack.count * sizeof(uint32_t) && "HEADER_GC_PRELOAD_ENTITIES");

	CRaceManager& rkRaceManager = CRaceManager::Instance();
	CRaceData* pkRaceData = nullptr;

	for (uint16_t i = 0; i < pack.count; i++)
	{
		uint32_t race = 0;
		if (!Recv(sizeof(uint32_t), &race))
			return false;

		if (IsInvisibleRace(race))
			continue;

		rkRaceManager.GetRaceDataPointer(race, &pkRaceData);
	}

	return true;
}
#endif

 

[CONTROL 922]

7 hours ago, Amun said:

Idk why I always forget to reply to topics

@msmajda9 they can just change unordered_set to set and change the types to byte/word

@ HAZEJ good catch with invisible races

@RobinHoodye yeah, move the loading to loading phase, I left it in game phase because it's simpler for people to follow

@ CONTROL interesting idea with the groups. I think(didn't check) they all use Spawn so they'll register for preloading as soon as they appear for the first time, but thanks for sharing.

This was never supposed to run in the game phase(which is why you see that delay between end of loading phase and first render), but adding it in the loading phase would've complicated the tutorial and most people just want it to work - they don't care how.

 

Funnily enough, this could've been simplified to just calling GetRaceDataPointer instead of creating an entity and discarding it, since it calls __LoadRaceData on its own when it can't find it(and it will fix the "vid already exists" shit as well).

Here's the code for copy/pasta:

#ifdef ENABLE_ENTITY_PRELOADING
bool CPythonNetworkStream::RecvPreloadEntitiesPacket()
{
	extern bool IsInvisibleRace(uint16_t raceNum);

	TPacketGCPreloadEntities pack{};
	if (!Recv(sizeof(pack), &pack))
		return false;

	assert(int32_t(pack.size) - sizeof(pack) == pack.count * sizeof(uint32_t) && "HEADER_GC_PRELOAD_ENTITIES");

	CRaceManager& rkRaceManager = CRaceManager::Instance();
	CRaceData* pkRaceData = nullptr;

	for (uint16_t i = 0; i < pack.count; i++)
	{
		uint32_t race = 0;
		if (!Recv(sizeof(uint32_t), &race))
			return false;

		if (IsInvisibleRace(race))
			continue;

		rkRaceManager.GetRaceDataPointer(race, &pkRaceData);
	}

	return true;
}
#endif

 

 

You're right that GetRaceDataPointer is much cleaner and already avoids the "vid already exists" problem.

The only reason I went further is because, at least in my tests, it doesn't fully preload the heavy Granny side of the resources yet.

GetRaceDataPointer -> __LoadRaceData -> RegisterMotionData mainly registers the motion resources through GetResourcePointer(), which is still relatively lazy depending on the client/resource system implementation.

The heavier initialization usually starts happening later through RegisterMotionThing(), where the motion data actually gets bound/processed.

That's why I ended up forcing the motion iteration directly.

CGraphicThing* CRaceData::RegisterMotionData(...) { // Only gets a resource POINTER - no actual loading 
CGraphicThing* pMotionThing = (CGraphicThing*)CResourceManager::Instance() .GetResourcePointer(c_szFileName); ... }

GetResourcePointer is lazy — it registers the resource but doesn't guarantee it's actually loaded into memory.

The actual loading happens in

CRaceData::TMotionModeDataIterator itor;

	if (CreateMotionModeIterator(itor))
	{
		do
		{
			WORD wMotionMode = itor->first;
			CRaceData::TMotionModeData * pMotionModeData = itor->second;

			CRaceData::TMotionVectorMap::iterator itorMotion = pMotionModeData->MotionVectorMap.begin();
			for (; itorMotion != pMotionModeData->MotionVectorMap.end(); ++itorMotion)
			{
				WORD wMotionIndex = itorMotion->first;
				const CRaceData::TMotionVector & c_rMotionVector = itorMotion->second;
				CRaceData::TMotionVector::const_iterator it;
				DWORD i;
				for (i = 0, it = c_rMotionVector.begin(); it != c_rMotionVector.end(); ++i, ++it)
				{
					DWORD dwMotionKey = MAKE_RANDOM_MOTION_KEY(wMotionMode, wMotionIndex, i);
					CGraphicThingInstance::RegisterMotionThing(dwMotionKey, it->pMotion);
				}
			}
		}
		while (NextMotionModeIterator(itor));
	}

 

via RegisterMotionThing, which is where the heavy Granny initialization usually happens

 

Also regarding stones/boss summons:

Some of these entities only exist inside groups or bosses and never appear through normal map spawns during loading, so their race data may never get touched until the first summon happens in-game (when attacked) — which is exactly where the freeze usually occurs.

That's mainly why I added ENABLE_ENTITY_PRELOADING_EXTRA.