Instant cache flush after than shutdown

[Koray 3322]

I have made a modification to immediately complete the cache flush processes in the db app without waiting after the /shutdown command is used. Following this modification the flush operation which is triggered every 5-6 minutes will start immediately and will close active game applications once it is finished. So, in this way you can perform quickly maintenance or restart operations after all applications have closed by monitoring the process list.

Spoiler

* common/tables.h

Search:

    HEADER_GD_SETUP            = 0xff,

Add above:

    HEADER_GD_FORCE_FLUSH_CACHE = 250,

--

Search:

    HEADER_DG_MAP_LOCATIONS            = 0xfe,

Add above:

    HEADER_DG_SHUTDOWN_CORE = 250,

* db/clientmanager.cpp

Search:

CClientManager::CClientManager() :

Add above:

static constexpr uint32_t sc_kShutdownMagic = 0x12345678;

---

Search:

    m_bShutdowned(false),

Add below:

    m_bForceFlushHandled(false),

---

Search:

void CClientManager::QUERY_ITEM_DESTROY(CPeer* pkPeer, const char* c_pData)
{
    ...
}

Add below:

void CClientManager::ForceFlushCache(CPeer* pkPeer, const char* c_pData)
{
    sys_err("FORCE_FLUSH_CACHE requested by %s (%s:%u) CH %u", pkPeer->GetHost(), pkPeer->GetPublicIP(), pkPeer->GetListenPort(), pkPeer->GetChannel());

    m_bForceFlushHandled = true;

    if (pkPeer)
    {
        pkPeer->EncodeHeader(HEADER_DG_SHUTDOWN_CORE, 0, 0);
        pkPeer->Encode(&sc_kShutdownMagic, sizeof(sc_kShutdownMagic));
    }
}

---

Search:

        case HEADER_GD_FLUSH_CACHE:
            QUERY_FLUSH_CACHE(peer, data);
            break;

Add below:

        case HEADER_GD_FORCE_FLUSH_CACHE:
            ForceFlushCache(peer, data);
            break;

---

Search:

int CClientManager::Process()
{
    int pulses = thecore_idle();

    if (!pulses)
        return 0;

    while (pulses--)
    {
        ++thecore_heart->pulse;

        if (!(thecore_heart->pulse % thecore_heart->passes_per_sec))
        {
            if (g_test_server)
            {
                ...
            }
            else
            {
                pt_log("[%9d] return %d/%d/%d/%d async %d/%d/%d%/%d",
                    thecore_heart->pulse,
                    CDBManager::Instance().CountReturnQuery(SQL_PLAYER),
                    CDBManager::Instance().CountReturnResult(SQL_PLAYER),
                    CDBManager::Instance().CountReturnQueryFinished(SQL_PLAYER),
                    CDBManager::Instance().CountReturnCopiedQuery(SQL_PLAYER),
                    CDBManager::Instance().CountAsyncQuery(SQL_PLAYER),
                    CDBManager::Instance().CountAsyncResult(SQL_PLAYER),
                    CDBManager::Instance().CountAsyncQueryFinished(SQL_PLAYER),
                    CDBManager::Instance().CountAsyncCopiedQuery(SQL_PLAYER));

                //if ((thecore_heart->pulse % 50) == 0)
                    /*sys_log(0, "[%9d] return %d/%d/%d async %d/%d/%d",
                        thecore_heart->pulse,
                        CDBManager::Instance().CountReturnQuery(SQL_PLAYER),
                        CDBManager::Instance().CountReturnResult(SQL_PLAYER),
                        CDBManager::Instance().CountReturnQueryFinished(SQL_PLAYER),
                        CDBManager::Instance().CountAsyncQuery(SQL_PLAYER),
                        CDBManager::Instance().CountAsyncResult(SQL_PLAYER),
                        CDBManager::Instance().CountAsyncQueryFinished(SQL_PLAYER));*/
            }

Add below:

            if (m_bForceFlushHandled)
            {
                auto bHasGameChannel = false;
                for (const auto& it : m_peerList)
                {
                    if (it->GetChannel() > 0)
                    {
                        bHasGameChannel = true;
                        break;
                    }
                }

                if (!bHasGameChannel)
                {
                    sys_err("No peer and force flush handled. Shutdown.");

                    // Send shutdown packet to auth servers
                    for (const auto& it : m_peerList)
                    {
                        if (it->GetChannel() == 0)
                        {
                            sys_log(0, "Send shutdown packet to auth server. %s:%u", it->GetHost(), it->GetListenPort());
                            it->EncodeHeader(HEADER_DG_SHUTDOWN_CORE, 0, 0);
                            it->Encode(&sc_kShutdownMagic, sizeof(sc_kShutdownMagic));
                            it->Send();
                        }
                    }
                    if (m_pkAuthPeer)
                    {
                        m_pkAuthPeer->EncodeHeader(HEADER_DG_SHUTDOWN_CORE, 0, 0);
                        m_pkAuthPeer->Encode(&sc_kShutdownMagic, sizeof(sc_kShutdownMagic));
                        m_pkAuthPeer->Send();
                    }

                    m_bShutdowned = true;
                }
                else
                {
                    sys_log(0, "Waiting for all peer to be disconnected. %d", m_peerList.size());
                }
            }

 

---

Search:

            if (fdwatch_check_event(m_fdWatcher, m_fdAccept, idx) == FDW_READ)

Add above:

            if (m_bForceFlushHandled)
            {
                sys_log(0, "Ignoring new peer while force flush is handled");
                continue;
            }

* db/clientmanager.h

Search:

    void        SendPartyOnSetup(CPeer* peer);

Add below:

    void        ForceFlushCache(CPeer* pkPeer, const char* c_pData);

---

Search:

    bool                    m_bShutdowned;

Add below:

    bool                    m_bForceFlushHandled;

* db/main.cpp

Search:

    CClientManager::Instance().MainLoop();

Add below:

    sys_err("DBCacheServer End");

* game/cmd_general.cpp

Search:
 

        else if (*pSec < -10)
            return 0;

Change:

        else if (*pSec < -10)
        {
            CHARACTER_MANAGER::Instance().ProcessDelayedSave();
            ITEM_MANAGER::Instance().FlushDelayedSave();

            if (db_clientdesc)
                db_clientdesc->DBPacketHeader(HEADER_GD_FORCE_FLUSH_CACHE, 0, 0);
            
            return 0;
        }

---

Search:
 

    if (g_bNoMoreClient)
    {
        thecore_shutdown();
        return;
    }

Change:    

    if (g_bNoMoreClient)
    {
        CHARACTER_MANAGER::Instance().ProcessDelayedSave();
        ITEM_MANAGER::Instance().FlushDelayedSave();

        if (db_clientdesc)
            db_clientdesc->DBPacketHeader(HEADER_GD_FORCE_FLUSH_CACHE, 0, 0);

        thecore_shutdown();
        return;
    }

* game/input.h

Search:

    void        GuildLoad(const char* c_pData);

Add below:

    void        ShutdownCore(const char* c_pData);

* game/input_db.cpp

Search:
 

void CInputDB::GuildLoad(const char* c_pData)
{
    ...
}

Add below:

void CInputDB::ShutdownCore(const char* c_pData)
{
    sys_err("InputDB::ShutdownCore");

    static constexpr uint32_t sc_kExpectedMagic = 0x12345678;
    uint32_t magic = *(uint32_t*)c_pData;

    if (magic != sc_kExpectedMagic)
    {
        sys_err("InputDB::ShutdownCore: magic error %u != %u", magic, sc_kExpectedMagic);
        return;
    }

    thecore_shutdown();
}

---

Search:

    case HEADER_DG_GUILD_SKILL_UPDATE:
        GuildSkillUpdate(c_pData);
        break;

Add below:

    case HEADER_DG_SHUTDOWN_CORE:
        ShutdownCore(c_pData);
        break;

---

Search:

int CInputDB::Analyze(LPDESC d, uint8_t bHeader, const char* c_pData)
{

Add below:

    if (thecore_is_shutdowned())
        return 0;

* game/item_manager.cpp

Search:
 

void ITEM_MANAGER::FlushDelayedSave(LPITEM item)
{
    TR1_NS::unordered_set<LPITEM>::iterator it = m_set_pkItemForDelayedSave.find(item);

    if (it == m_set_pkItemForDelayedSave.end())
    {
        return;
    }

    m_set_pkItemForDelayedSave.erase(it);
    SaveSingleItem(item);
}

Add below:

void ITEM_MANAGER::FlushDelayedSave()
{
    TR1_NS::unordered_set<LPITEM>::iterator it = m_set_pkItemForDelayedSave.begin();
    TR1_NS::unordered_set<LPITEM>::iterator this_it;

    while (it != m_set_pkItemForDelayedSave.end())
    {
        this_it = it++;
        SaveSingleItem(*this_it);
        m_set_pkItemForDelayedSave.erase(this_it);
    }

    if (db_clientdesc)
        db_clientdesc->FlushOutput();
}

* game/item_manager.h

Search:

        void            FlushDelayedSave(LPITEM item);

Add below:

        void            FlushDelayedSave();

* game/main.cpp

Search:
 

        else if (thecore_pulse() > g_shutdown_disconnect_force_pulse + PASSES_PER_SEC(5))
        {
            thecore_shutdown();
        }

Change:

        else if (thecore_pulse() > g_shutdown_disconnect_force_pulse + PASSES_PER_SEC(5))
        {
            if (db_clientdesc)
                db_clientdesc->DBPacketHeader(HEADER_GD_FORCE_FLUSH_CACHE, 0, 0);

            thecore_shutdown();
        }

* game/input.cpp

Search:

    if (!m_pPacketInfo)
    {
        sys_err("No packet info has been binded to");
        return true;
    }

Add above:

    if (thecore_is_shutdowned())
    {
        lpDesc->SetPhase(PHASE_CLOSE);
        return true;
    }

 

[HFWhite 976]

Awesome man! Thanks, I will test it tonight

[Kaptan Yosun 863]

Excellent work

[Amun 3221]

Cool idea and thanks for sharing, but since all the cores(including db) flush the cache on shutdown anyway, why not just make the DB flush the cache(or kill itself) when it loses all the peers? That way you could just shut down the cores and let the db cut its wrists in the bath tub when it's left sad and alone in the world

[Koray 3322]

9 hours ago, Amun said:

Cool idea and thanks for sharing, but since all the cores(including db) flush the cache on shutdown anyway, why not just make the DB flush the cache(or kill itself) when it loses all the peers? That way you could just shut down the cores and let the db cut its wrists in the bath tub when it's left sad and alone in the world

If you don't trigger kill cores by yourself, the auth cores stay alive so the database won't shutdown by itself and therefore won't switch to the shutdown phase unless you forcibly kill all auth processes. Anyway, I was already tried as you described, but when you shutdown without flushing the processes in the cores and the database socket, I encountered some errors like "multiple item id" in db process's save item query function and issues with save to the database. I don't remember the exact query error because it happened several months ago, but you will likely encounter it if you test it on a live server. It worked most reliably in this way. How you want to use it is up to you, consider this just an example.

Edited July 12, 2024 by Koray

[Amun 3221]

42 minutes ago, Koray said:

If you don't trigger kill cores by yourself, the auth cores stay alive so the database won't shutdown by itself and therefore won't switch to the shutdown phase unless you forcibly kill all auth processes. Anyway, I was already tried as you described, but when you shutdown without flushing the processes in the cores and the database socket, I encountered some errors like "multiple item id" in db process's save item query function and issues with save to the database. I don't remember the exact query error because it happened several months ago, but you will likely encounter it if you test it on a live server. It worked most reliably in this way. How you want to use it is up to you, consider this just an example.

Because the master was supposed to forward the shutdown packet to them. Anyhow, it was just a question I had, no bad intentions.

That item ID thing is pretty strange and don't really see why it would happen, but thanks for the heads up

[Gurgarath 3693]

On 7/12/2024 at 7:27 AM, Koray said:

If you don't trigger kill cores by yourself, the auth cores stay alive so the database won't shutdown by itself and therefore won't switch to the shutdown phase unless you forcibly kill all auth processes. Anyway, I was already tried as you described, but when you shutdown without flushing the processes in the cores and the database socket, I encountered some errors like "multiple item id" in db process's save item query function and issues with save to the database. I don't remember the exact query error because it happened several months ago, but you will likely encounter it if you test it on a live server. It worked most reliably in this way. How you want to use it is up to you, consider this just an example.

Duplicate item ID is a symptom of something bigger, not a cause of the shutdown. It can happen if you do not lock the item when you manipulate it (which can make it available for other functions) or it most likely happen when you do not flush the item / leave it available in a core and then you switch core. This is the most likely explanation I can have, even though it is weird as it creates a new one more often than not. At least I never had the issue in any live server when I shutdowned. The worst that I had was recent (last cache flush) items or changes being lost.

Regarding the the shutdown, I personally called the equivalent of what you call in the do_flush command for every logged-in player. It's not as good but somewhat worked fine. The best is to call a graceful kill of the process (killall game for example), wait for them to exit, then gracefully kill the db the same way and wait until it properly flushes. Any -9 kill is to entirely avoid.

Anyway, this is a good release, thank you very much!