Jump to content
  • We need you!

    You must register to discover all the features of our community!

Asset loading from network at run-time


Recommended Posts

Hey guys,

 

I just programmed this feature for my server but I thought it can be really useful for everyone so now I release it.

This stuff is about how you can load images, etc. in the game without directly packing it into the client but uploading them to a web-server. In this tutorial we will make it work for images, but you can extend it to any type of file you want.

There are not much requirements we only use up to C++11 features and you have to have libcurl library.

 

Open up EterLib/ResourceManager.h and add add the following to the end of the class (don't forget to include <future> and <utility>):

private:
	std::list<std::future<CResource*>>		ongoingDownloads;
public:
	void AddDownload(std::future<CResource*>&& f)
	{
		ongoingDownloads.emplace_back(std::forward<std::future<CResource*>>(f));
	}

Go to EterLib/ResourceManager.cpp and find the CResourceManager::Update function, add the following to the end of it:

	for (auto it = ongoingDownloads.begin(); it != ongoingDownloads.end();)
	{
		if (it->wait_for(std::chrono::seconds(0)) == std::future_status::ready)
		{
			it->get()->LoadDownloadedData();
			it = ongoingDownloads.erase(it);
		}
		else
		{
			++it;
		}
	}

Next, open EterLib/Resource.h, find the constructor and modify it like:

CResource(const char* c_szFileName, bool _loadFromNetwork = false);

After add the following to the end of the class:

protected:
	bool loadFromNetwork;
private:
	std::vector<BYTE> downloadedData;
public:
	void LoadDownloadedData();

 

Then go to EterLib/Resource.cpp, find the constructor and also modify it like:

CResource::CResource(const char* c_szFileName, bool _loadFromNetwork) 
	: me_state(STATE_EMPTY)
	, loadFromNetwork(_loadFromNetwork)
{
	SetFileName(c_szFileName);
}

In the same file, add this to the beginning right after the includes:

#include <curl/curl.h>
#include "ResourceManager.h"
  
#define ASSET_SERVER "http://assets.my-website.com/"
  
static size_t CurlWriteCallback(void* contents, size_t size, size_t nmemb, void* userp)
{
	if (nullptr != userp)
	{
		std::vector<BYTE>& vec = *((std::vector<BYTE>*)userp);
		vec.reserve(vec.size() + (size * nmemb));
		for (size_t i = 0; i < size * nmemb; ++i)
		{
			vec.push_back(((BYTE*)contents)[i]);
		}
	}
	return size * nmemb;
}

void CResource::LoadDownloadedData()
{
	if (downloadedData.empty())
		return;

	Clear();

	if (OnLoad(downloadedData.size(), downloadedData.data()))
	{
		me_state = STATE_EXIST;
	}
	else
	{
		Tracef("CResource::Load Error %s\n", GetFileName());
		me_state = STATE_ERROR;
	}

	downloadedData.clear();
}

 

Now, - still in the same file - find CResource::Load function and modify it like this:

void CResource::Load()
{
	if (me_state != STATE_EMPTY)
		return;

	std::string fileName = GetFileName();

	if (loadFromNetwork && downloadedData.empty())
	{
		CResourceManager::instance().AddDownload(std::move(std::async(std::launch::async, [this, fileName]()
			{
				std::string url = ASSET_SERVER;
				url += fileName;

				CURL* curl = curl_easy_init();
				if (curl)
				{
					curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
					curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, CurlWriteCallback);
					curl_easy_setopt(curl, CURLOPT_WRITEDATA, &downloadedData);
					curl_easy_perform(curl);
					curl_easy_cleanup(curl);
				}

				return this;
			})));

		fileName = "d:/ymir work/ui/placeholder.tga";
	}

	DWORD		dwStart = ELTimer_GetMSec();
	CMappedFile	file;
	LPCVOID		fileData;

	//Tracenf("Load %s", c_szFileName);

	if (CEterPackManager::Instance().Get(file, fileName.c_str(), &fileData))
	{
		m_dwLoadCostMiliiSecond = ELTimer_GetMSec() - dwStart;
		//Tracef("CResource::Load %s (%d bytes) in %d ms\n", c_szFileName, file.Size(), m_dwLoadCostMiliiSecond);

		if (OnLoad(file.Size(), fileData))
		{
			me_state = STATE_EXIST;
		}
		else
		{
			Tracef("CResource::Load Error %s\n", fileName.c_str());
			me_state = STATE_ERROR;
			return;
		}
	}
	else
	{
		if (OnLoad(0, NULL))
			me_state = STATE_EXIST;
		else
		{
			Tracef("CResource::Load file not exist %s\n", fileName.c_str());
			me_state = STATE_ERROR;
		}
	}
}

 

Still in Resource.cpp, find the CResource::Reload function and modify like:

void CResource::Reload()
{
	Tracef("CResource::Reload %s\n", GetFileName());

	std::string fileName = GetFileName();

	if (loadFromNetwork && downloadedData.empty())
	{
		CResourceManager::instance().AddDownload(std::move(std::async(std::launch::async, [this, fileName]()
			{
				std::string url = ASSET_SERVER;
				url += fileName;

				CURL* curl = curl_easy_init();
				if (curl)
				{
					curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
					curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, CurlWriteCallback);
					curl_easy_setopt(curl, CURLOPT_WRITEDATA, &downloadedData);
					curl_easy_perform(curl);
					curl_easy_cleanup(curl);
				}

				return this;
			})));
	}
}

 

Open EterLib/GrpImage.h and modify the constructor:

CGraphicImage(const char* c_szFileName, DWORD dwFilter = D3DX_FILTER_LINEAR, bool loadFromNetwork = false);

In EterLib/GrpImage.cpp also modify it:

CGraphicImage::CGraphicImage(const char * c_szFileName, DWORD dwFilter, bool loadFromNetwork) 
	: CResource(c_szFileName, loadFromNetwork)
	, m_dwFilter(dwFilter)
{
	m_rect.bottom = m_rect.right = m_rect.top = m_rect.left = 0;
}

 

Finally open ScriptLib/Resource.cpp and add this somewhere the beginning:

CResource* NewOnlineImage(const char* c_szFileName)
{
	return new CGraphicImage(c_szFileName, D3DX_FILTER_LINEAR, true);
}

Go down where you see m_resManager.RegisterResourceNewFunctionPointer("jpg", NewImage); and add after:

m_resManager.RegisterResourceNewFunctionPointer("oimg", NewOnlineImage);

 

We're done! Now if you use *.oimg extension anywhere it will load them from what you define as ASSET_SERVER in EterLib/Resource.cpp (http://assets.my-website.com/filename.oimg). You have to rename the image you upload from the original extension to oimg!

 

Put a placeholder image at "d:\ymir work\ui\placeholder.tga" that it will load while waiting for bigger images.

 

GIF in action (normal size image, the pic in the right bottom corner):

online-asset-normal-load.gif

 

GIF of loading a big (10MB image) that takes more time:

online-asset-big-load.gif

 

Hope you like it!

Edited by Distraught (see edit history)
  • Love 13
  • Party 1
  • See 1

AxgtRfG.png
C++ programmer at Gameloft

Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


×
×
  • Create New...

Important Information

Terms of Use / Privacy Policy / Guidelines / We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.