Realtime Update of the resources in Ogre App

http://www.ogre3d.org/forums/viewtopic.php?f=5&t=46787&start=25#p329272

This code contains a class, with helper to reload RessourceGroup.
The idea is : you create a ressource group, with your material in it (and texture and shaders). 
then you call an update fonction.

The update function : 
1/ first time you call it, it just make the list of all files, and corresponding last modification date.
2/ then second and + times that you call it, it checks if the last modification date has been modified.
3/ it reloads if needed the content of the RessourceGroup.

What is more, the update function output errors from the log. Very useful if you want to display that 
"at line 6 in mymat.material you made an error", or that "in the shader red.cg at line 20 the compiler got a problem".

So I find it very cool, but it's just a class.

ResourceGroupHelper.h

#ifndef RESOURCEGROUPHELPER_H
#define RESOURCEGROUPHELPER_H

#include <utility>
#include <string>
#include <vector>
#include <OgreRenderable.h>
#include <OgreLog.h>
///\brief a fake class with different useful methods for manipulating Resourcegroups.
/// please note that it was not tested with background loading.

class ResourceGroupHelper
{
private:

   /// \brief anti copy constructor (no use)
   ResourceGroupHelper(const ResourceGroupHelper&);
   /// \brief anti affector (no use)
   ResourceGroupHelper& operator=(const ResourceGroupHelper&);

   /// \brief a helper method to visit all overlay
   /// it uses a recursive call
   void visitRecursivelyRenderablesFrom(Ogre::OverlayContainer* pOverlayContainer, Ogre::Renderable::Visitor& pVisitor, bool debugRenderable = false);

   ///\brief a map [key : nameOfTheRessourceGroup]/[value : last modification time on the hdd from the files of the ResourceGroup]
   std::map<std::string, time_t > mRessourceGroupModificationTimes;

   // ------ helper classes
   
   /// \brief this visitor will be used to set the material on known renderable that allow this operation.
   /// for other user class renderable, must be tweaked/changed
   class BRAND_NAME : public Ogre::Renderable::Visitor
   {
   private:
      BRAND_NAME(const BRAND_NAME&);///<\brief anti copyconstructor
      BRAND_NAME& operator=(const BRAND_NAME&);///<\brief anti affector
   public: 
      /// \brief default constructor
      BRAND_NAME();
      /// \brief called for each renderable
      virtual void visit(Ogre::Renderable *rend, Ogre::ushort lodIndex, bool isDebug, Ogre::Any *pAny=0); 
   };
   
   ///\brief this class will listen to the log.
   /// it will check if there are "errors" or "exception" send to the listener
   /// and allows to keep or not the messages.
   class ResourceGroupHelperLogListener : public Ogre::LogListener
   {
   private:
      ResourceGroupHelperLogListener(const ResourceGroupHelperLogListener&);///<\brief anti copyconstructor
      ResourceGroupHelperLogListener& operator=(const ResourceGroupHelperLogListener&);///<\brief anti affector
      std::stringstream mKeptMessages;///<\brief interesting messages that we choose to keep
   public:
      /// \brief the constructor
      ResourceGroupHelperLogListener();
      ///\brief the destructor de-register itself from the log
      ~ResourceGroupHelperLogListener();
      /// \brief called for each message
      virtual void messageLogged(const Ogre::String &message, Ogre::LogMessageLevel lml, bool maskDebug, const Ogre::String &logName, bool& skipThisMessage);
      /// \brief get a copy of kept messages
      std::string getKeptMessages();
      ///\brief tells if mKeptMessages is empty or not
      bool areMessagesKept();
      ///\brief clear the kept messages
      void clearKeptMessages();
   };

public:
   /// \brief default constructor
   ResourceGroupHelper(void);
   
   /// \brief destructor
   ~ResourceGroupHelper(void);

   /// \brief : a path + the archive type.
   typedef std::pair<std::string,std::string> ArchivePathAndType;

   /// \brief get a vector with directory path and corresponding type.
   /// can be useful if you want to reload some Resourcegroup.
   /// note : does not check the config file.
   /// \param the name of the resourcegroup
   /// \warning there is no garanty that the path order will be the same than during the first loading.
   std::vector<ArchivePathAndType> getAllPathAndTypesNames(const std::string& pResourceGroupName);

   /// \brief tries to suppress a Resourcegroup and reload it completely using the provided ordered locations.
   /// \param the name of the resourcegroup
   /// \param the path and types of the archives to be loaded, and in the order to be loaded
   /// it will not work correctly if some elements of the Resourcegroup are still used.
   /// \return true if the Resourcegroup was found and correctly destroy, false otherwise.
   bool reloadAResourceGroup(const std::string& pResourceGroupName,const std::vector<ArchivePathAndType>& pLocationToAdd);
   
   /// \brief this serves the same purpose than reloadAResourceGroup, but it does not destroy/recreate the ResourceGroup.
   /// \param the name of the resourcegroup
   /// on the one hand, you don't need to worry about the path and order of the locations.
   /// on the other hand, there is no test that the resources were really cleared and reloaded.
   /// personnally I prefer this thought, because it's much smarter & less costly in the end.
   bool reloadAResourceGroupWithoutDestroyingIt(const std::string& pResourceGroupName);

   /// \brief return true if the resourcegroup exists
   /// \param the name of the resourcegroup
   bool resourceGroupExist(const std::string& pResourceGroupName);

   /// \brief updating informations about materials on all 'reachable' renderables
   /// that are currently used by the different scenemanagers and overlays
   void updateOnEveryRenderable();

   /// \brief get the latest modification time : check all files date from the resourcegroup
   /// \param the name of the resourcegroup
   /// \warning time-costly! because access the HDD!
   time_t getLatestModificationTime(const std::string& pResourceGroupName);


   /// \brief test latest modification time.
   /// if it did change since the latest call to this function,
   /// then the resourcegroup is reloaded
   /// and all the renderables are updated
   /// bool returns if reload was tried or not.
   /// if an error happens during reloading (parsing script + glsl), it is likely to be
   ///      described in the pOutLoggedMessages param, if useLog.
   bool checkTimeAndReloadIfNeeded(const std::string& pResourceGroupName,std::string &pOutLoggedMessages, bool useLog=true);
};

#endif

ResourceGroupHelper.cpp

#include "ResourceGroupHelper.h"
#include "OgreResourceGroupManager.h"
#include "OgreLogManager.h"
#include "OgreRoot.h"
#include "OgreMovableObject.h"
#include "OgreMaterialManager.h"
#include "OgreEntity.h"
#include "OgreSubEntity.h"
#include "OgreBillBoardChain.h"
#include "OgreBillBoardSet.h"
#include "OgreOverlayElement.h"
#include "OgreOverlay.h"
#include "OgreOverlayManager.h"
#include "OgreOverlayContainer.h"

ResourceGroupHelper::BRAND_NAME::BRAND_NAME():
Ogre::Renderable::Visitor()
{
}

void ResourceGroupHelper::BRAND_NAME::visit(
   Ogre::Renderable *rend, Ogre::ushort lodIndex, bool isDebug, Ogre::Any *pAny)
{
   const Ogre::MaterialPtr mat = rend->getMaterial();
   if(!mat.isNull())
   {
      std::string newMatName = mat->getName();
      Ogre::MaterialPtr newMat = Ogre::MaterialManager::getSingleton().getByName(newMatName);
      if(newMat.isNull())
      {
         // this can happen if there was error during the reloading of the material.
         // in that case, we keep the ancient one.
         // Ogre::LogManager::getSingleton().logMessage(newMatName+" : new material is null!");
         return;
      }

      // unfortunately, the renderable gives access only to a const MaterialPtr.
      // and there is no 'setMaterial' or 'setMaterialName' method on renderables.
      // so I have to try to down cast with known classes...
      {   
         Ogre::SubEntity* lRend = dynamic_cast<Ogre::SubEntity*>(rend);
         if(lRend){lRend->setMaterialName(newMatName);return;} 
      }
      {
         Ogre::SimpleRenderable* lRend = dynamic_cast<Ogre::SimpleRenderable*>(rend);
         if(lRend){lRend->setMaterial(newMatName);return;} 
      }
      {
         Ogre::ShadowRenderable* lRend = dynamic_cast<Ogre::ShadowRenderable*>(rend);
         if(lRend){lRend->setMaterial(newMat);return;} 
      }
      {   
         Ogre::BillboardChain* lRend = dynamic_cast<Ogre::BillboardChain*>(rend);
         if(lRend){lRend->setMaterialName(newMatName);return;} 
      }
      {   
         Ogre::BillboardSet* lRend = dynamic_cast<Ogre::BillboardSet*>(rend);
         if(lRend){lRend->setMaterialName(newMatName);return;} 
      }
      {   
         Ogre::OverlayElement* lRend = dynamic_cast<Ogre::OverlayElement*>(rend);
         if(lRend){lRend->setMaterialName(newMatName);return;} 
      }
   }else{
      // was there for debug...
      // Ogre::LogManager::getSingleton().logMessage("material of renderable is null!");
   }
}

ResourceGroupHelper::ResourceGroupHelperLogListener::ResourceGroupHelperLogListener():
Ogre::LogListener(),mKeptMessages()
{
   Ogre::LogManager* logMgr = Ogre::LogManager::getSingletonPtr();
   if(logMgr)
   {
      bool logExist = true;
      try{logMgr->getDefaultLog();}catch(Ogre::Exception&)
      {
         logExist = false;
      }
      if(logExist)
      {
         logMgr->getDefaultLog()->addListener(this);
      }
   }
}

ResourceGroupHelper::ResourceGroupHelperLogListener::~ResourceGroupHelperLogListener()
{
   Ogre::LogManager* logMgr = Ogre::LogManager::getSingletonPtr();
   if(logMgr)
   {
      bool logExist = true;
      try{logMgr->getDefaultLog();}catch(Ogre::Exception&)
      {
         logExist = false;
      }
      if(logExist)
      {
         logMgr->getDefaultLog()->removeListener(this);
      }
   }
}

bool ResourceGroupHelper::ResourceGroupHelperLogListener::areMessagesKept()
{
   return mKeptMessages.peek()==EOF;
}

std::string ResourceGroupHelper::ResourceGroupHelperLogListener::getKeptMessages()
{
   return mKeptMessages.str();
}

void ResourceGroupHelper::ResourceGroupHelperLogListener::clearKeptMessages()
{
   mKeptMessages.str(std::string());
}

void ResourceGroupHelper::ResourceGroupHelperLogListener::messageLogged
   (const Ogre::String &message, Ogre::LogMessageLevel lml, bool maskDebug, const Ogre::String &logName, bool& skipThisMessage)
{
   Ogre::String copy = message;
   Ogre::StringUtil::toLowerCase(copy);
   Ogre::String pattern1 = "*error*";
   Ogre::String pattern2 = "*exception*";
   bool lErrorfound = Ogre::StringUtil::match(message,pattern1,true) || Ogre::StringUtil::match(message,pattern2,true);
   if(lErrorfound)
   {
      mKeptMessages<<message;
   }
}


ResourceGroupHelper::ResourceGroupHelper(void):
mRessourceGroupModificationTimes()
{
   // reloading the default resource group is a bad idea.
   // lets make it harder, by giving it a big modification time.
   time_t veryBig = 1;
   {
      int nbBytes = sizeof(time_t);
      for(int i = 0; i<(nbBytes-1)*8;i++)
      {
         veryBig+=2*veryBig;
      }
   }
   mRessourceGroupModificationTimes[Ogre::ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME] = veryBig;
}

ResourceGroupHelper::~ResourceGroupHelper(void)
{
}

std::vector<ResourceGroupHelper::ArchivePathAndType> ResourceGroupHelper::getAllPathAndTypesNames(const std::string& pResourceGroupName)
{
   // note : unfortunately no access to the order in which path where loaded
   Ogre::FileInfoListPtr fli_dirs = Ogre::ResourceGroupManager::getSingleton().listResourceFileInfo(pResourceGroupName,true);

   std::vector<ResourceGroupHelper::ArchivePathAndType> lDirectoryInfos;

   if(!fli_dirs.isNull())
   {
      Ogre::FileInfoList::iterator itFliD = fli_dirs->begin();
      Ogre::FileInfoList::iterator itFliDEnd = fli_dirs->end();
      for(; itFliD!=itFliDEnd;itFliD++)
      {
         Ogre::FileInfo& lFinfoD = (*itFliD);
         if(lFinfoD.archive)
         {
            ResourceGroupHelper::ArchivePathAndType arch;
            arch.first = lFinfoD.path + lFinfoD.archive->getName();
            arch.second = lFinfoD.archive->getType();
            lDirectoryInfos.push_back(arch);
         }
      }
   }
   return lDirectoryInfos;
}



bool ResourceGroupHelper::reloadAResourceGroup(const std::string& pResourceGroupName,const std::vector<ResourceGroupHelper::ArchivePathAndType>& pLocationToAdd)
{
   if(!resourceGroupExist(pResourceGroupName))
   {
      // not present. something wrong.
      return false;
   }

   Ogre::ResourceGroupManager& resGroupMgr = Ogre::ResourceGroupManager::getSingleton();
   resGroupMgr.destroyResourceGroup(pResourceGroupName);

   if(resourceGroupExist(pResourceGroupName))
   {
      // still present. something wrong.
      return false;
   }

   resGroupMgr.createResourceGroup(pResourceGroupName);

   std::vector<ResourceGroupHelper::ArchivePathAndType>::const_iterator iterLoc= pLocationToAdd.begin();
   std::vector<ResourceGroupHelper::ArchivePathAndType>::const_iterator iterLocEnd= pLocationToAdd.end();
   for(;iterLoc!=iterLocEnd;iterLoc++)
   {
      // add the location in the resourceGroup
      resGroupMgr.addResourceLocation(iterLoc->first,iterLoc->second,pResourceGroupName,false);
   }
   resGroupMgr.initialiseResourceGroup(pResourceGroupName);
   return true;
}



bool ResourceGroupHelper::reloadAResourceGroupWithoutDestroyingIt(const std::string& pResourceGroupName)
{
   if(!resourceGroupExist(pResourceGroupName))
   {
      // not present. something wrong.
      return false;
   }
   Ogre::ResourceGroupManager& resGroupMgr = Ogre::ResourceGroupManager::getSingleton();
   resGroupMgr.clearResourceGroup(pResourceGroupName);
   resGroupMgr.initialiseResourceGroup(pResourceGroupName);
   return true;
}


bool ResourceGroupHelper::resourceGroupExist(const std::string& pResourceGroupName)
{
   bool lIsPresent = false;
   Ogre::ResourceGroupManager& resGroupMgr = Ogre::ResourceGroupManager::getSingleton();
   Ogre::StringVector lAllResourceGroups = resGroupMgr.getResourceGroups();
   Ogre::StringVector::iterator iter = lAllResourceGroups.begin();
   Ogre::StringVector::iterator iterEnd = lAllResourceGroups.end();
   for(;iter!=iterEnd;iter++)
   {
      if((*iter) == pResourceGroupName)
      {
         lIsPresent = true;
      }
   }
   return lIsPresent;
}


void ResourceGroupHelper::updateOnEveryRenderable()
{

   //1/ get all the available object type (entity, light, user defined types ...)
   std::vector<std::string> allAvailableTypes; 
   Ogre::Root::MovableObjectFactoryIterator iterFactory = Ogre::Root::getSingleton().getMovableObjectFactoryIterator();
   for(;iterFactory.hasMoreElements();)
   {
      Ogre::MovableObjectFactory* factory = iterFactory.getNext();
      allAvailableTypes.push_back(factory->getType());
   }

   BRAND_NAME BRAND_NAME;

   //2/ for each scene manager, lets visit renderables!
   // unfortunately that does not cover all renderables type... (overlays...)
   Ogre::SceneManagerEnumerator::SceneManagerIterator iterSceneManager = Ogre::Root::getSingleton().getSceneManagerIterator();
   for(;iterSceneManager.hasMoreElements();)
   {
      Ogre::SceneManager * scMgr = iterSceneManager.getNext();

      std::vector<std::string>::iterator iterMovableType = allAvailableTypes.begin();
      std::vector<std::string>::iterator iterMovableTypeEnd = allAvailableTypes.end();
      for(;iterMovableType!=iterMovableTypeEnd;iterMovableType++)
      {
         Ogre::SceneManager::MovableObjectIterator iterMovable = scMgr->getMovableObjectIterator(*iterMovableType);
         for(;iterMovable.hasMoreElements();)
         {
            Ogre::MovableObject * movable = iterMovable.getNext();
            movable->visitRenderables(&BRAND_NAME,false);
         }
      }
   }

   // 3 / visit overlays!
   {
      Ogre::OverlayManager::OverlayMapIterator iterOverlay = Ogre::OverlayManager::getSingleton().getOverlayIterator();
      for(;iterOverlay.hasMoreElements();)
      {
         Ogre::Overlay* lOverlay = iterOverlay.getNext();
         // get the first level of OverlayContainer in the Overlay
         Ogre::Overlay::Overlay2DElementsIterator iterOverlayElem = lOverlay->get2DElementsIterator();
         for(;iterOverlayElem.hasMoreElements();)
         {
            Ogre::OverlayContainer * lOverlayCont = iterOverlayElem.getNext();
            visitRecursivelyRenderablesFrom(lOverlayCont,BRAND_NAME, false);
         }
      }
   }
}


void ResourceGroupHelper::visitRecursivelyRenderablesFrom(Ogre::OverlayContainer* pOverlayContainer, Ogre::Renderable::Visitor& pVisitor, bool debugRenderable)
{
   // call on 'this'
   pOverlayContainer->visitRenderables(&pVisitor,false);
   
   // call on 'leaf' (cf composite pattern)
   {
      Ogre::OverlayContainer::ChildIterator childIter = pOverlayContainer->getChildIterator();
      for(;childIter.hasMoreElements();)
      {
         Ogre::OverlayElement* lOverElem = childIter.getNext();
         lOverElem->visitRenderables(&pVisitor,false);
      }
   }

   // call on 'not-leaf' (cf composite pattern)
   {
      Ogre::OverlayContainer::ChildContainerIterator childContainerIter = pOverlayContainer->getChildContainerIterator();
      for(;childContainerIter.hasMoreElements();)
      {
         Ogre::OverlayContainer * childContainer = childContainerIter.getNext();
         visitRecursivelyRenderablesFrom(childContainer, pVisitor,debugRenderable);
      }
   }
}

time_t ResourceGroupHelper::getLatestModificationTime(const std::string& pResourceGroupName)
{
   time_t result(0);

   Ogre::ResourceGroupManager& rgMgr = Ogre::ResourceGroupManager::getSingleton();
   Ogre::FileInfoListPtr fli_files = rgMgr.listResourceFileInfo(pResourceGroupName,false);
   if(fli_files.isNull())
   {
      // something went wrong (example : no files)!
      return result;
   }

   // for each file, we check the modification date.
   // we keep the latest one.
   Ogre::FileInfoList::iterator iterFiles = fli_files->begin();
   Ogre::FileInfoList::iterator iterFilesEnd = fli_files->end();
   for(;iterFiles!=iterFilesEnd;iterFiles++)
   {
      Ogre::FileInfo& file = *iterFiles;
      if(file.archive)
      {
         //  [4/24/2013 zhangzh]
         Ogre::Archive* pArchive = const_cast<Ogre::Archive*>(file.archive);
         time_t modifTime = pArchive->getModifiedTime(file.filename);
         //time_t modifTime = file.archive->getModifiedTime(file.filename);
         if(result < modifTime)
         {
            result = modifTime;
         }
      }
   }
   return result;
}

bool ResourceGroupHelper::checkTimeAndReloadIfNeeded(const std::string& pResourceGroupName, std::string &pOutLoggedMessages, bool useLog)
{
   bool result = false;

   // 1/ get last modification time.
   time_t lastModificationTime = getLatestModificationTime(pResourceGroupName);

   std::map<std::string, time_t >::iterator iterInfoTime = mRessourceGroupModificationTimes.find(pResourceGroupName);
   if(iterInfoTime!=mRessourceGroupModificationTimes.end())
   {
      if(iterInfoTime->second < lastModificationTime)
      {
         // update the value
         iterInfoTime->second = lastModificationTime;
         
         Ogre::LogManager::getSingleton().logMessage("Time has evaluated ! Reload the ResourceGroup!");// use log if needed
         if(useLog)
         {
            // constructor register itself, and destructor unregister itself
            ResourceGroupHelperLogListener lLogListener;
            // try to reload
            reloadAResourceGroupWithoutDestroyingIt(pResourceGroupName);
            pOutLoggedMessages = lLogListener.getKeptMessages();
         }else{
            // try to reload
            reloadAResourceGroupWithoutDestroyingIt(pResourceGroupName);
         }

         // update the material of reachable renderables
         updateOnEveryRenderable();

         result = true;
      }
   }else{
      mRessourceGroupModificationTimes[pResourceGroupName] = lastModificationTime;
   }
   return result;
}

And in my application I just call it that way :

at initialisation :

#include "ResourceGroupHelper.h"

// ... some code

ResourceGroupHelper resourceGrouphelper;

In the infinit loop (or your framelistener), if I press 'R', then I check for reload (note that I could have done it without checking for R pressed):

if(firstR==0.0f && mKeyboard->isKeyDown(OIS::KC_R))
{            
            firstR = 1.0f;
         
            // name of the resource group that we want to track
            std::string resourceGroupName = "Yaose";

            // to receive the error messages
            std::string errorMessages;

            // look on the hdd the last modification time and try reload changes if needed
            resourceGrouphelper.checkTimeAndReloadIfNeeded(resourceGroupName, errorMessages);

            if(errorMessages.size()>0)
            {
               // you can display them in a message box, or in an overlay for example
               // here I resend them to the log...
               Ogre::LogManager& logMgr = Ogre::LogManager::getSingleton();
               logMgr.logMessage("****************** HERE THE BAD MESSAGES : ");
               logMgr.logMessage(errorMessages);
               logMgr.logMessage("****************** END OF THE BAD MESSAGES *****");
            }
            
}else if(!mKeyboard->isKeyDown(OIS::KC_R))
{
            firstR = 0.0;
}

And so the "Yaose" resourceGroupManager is reloaded when at least 1 file has been modified.

note : neveer try to reload the default resourcegroupmanager, it is not meant to be.

I hope you like it!

Pierre