KKMLL::Orderings Class Reference

Used to maintain multiple orderings of a single list of FeatureVector objects. More...

#include <Orderings.h>

Public Types
typedef Orderings *	OrderingsPtr

Public Member Functions
	Orderings (const FeatureVectorListPtr _data, kkuint32 _numOfOrderings, kkuint32 _numOfFolds, RunLog &_log)
	Constructs Orderings object from ImageFeatursList object. More...
	Orderings (const KKStr &_featureFileName, FeatureFileIOPtr _driver, RunLog &_log, bool &cancelFlag)
	Constructs Orderings of a FeatureVectorList from a previous construction that was saved in a data file. More...
	Orderings (const FeatureVectorListPtr _data, const KKStr &_indexFileName, kkuint32 _numOfOrderings, kkuint32 _numOfFolds, RunLog &_log)
	Constructs a Orderings object from a FeatureLectorList object. More...
	Orderings (FeatureVectorListPtr _data, RunLog &_log)
	Constructs a Orderings objects for a specified FeatureVectorList using a previously built Orderings data index file. More...
	~Orderings ()
FeatureVectorListPtr	Data () const
const KKStr &	FeatureFileName () const
const FileDescPtr	FileDesc () const
const KKStr &	IndexFileName () const
const MLClassListPtr	MLClasses () const
kkuint32	NumOfFolds () const
kkuint32	NumOfOrderings () const
const FeatureVectorListPtr	Ordering (kkuint32 orderingIdx) const
void	Save ()
void	Save (const KKStr &_indexFileName, RunLog &_log)
kkuint32	Size () const
bool	Valid () const

Static Public Member Functions
static OrderingsPtr	CreateOrderingsObjFromFileIfAvaliable (const FeatureVectorListPtr _data, kkuint32 _numOfOrderings, kkuint32 _numOfFolds, RunLog &_log)
	Constructs a Orderings object for a specified FeatureVectorList. More...

Detailed Description

Used to maintain multiple orderings of a single list of FeatureVector objects.

Author

Kurt Kramer

Will maintain multiple orderings of a single FeatureVectorList. These orderings will be saved in a text file for recall later. This will allow the user to be able to repeat experiments using the same ordering of data again. The Idea is that the 1st time this orderings is created the order will be randomly driven with Stratification by Class. This ordering will then be saved in data Files for future recall. More than one order can be maintained for a single list. An example of a good use of this is RandomSplits.

Definition at line 58 of file Orderings.h.

Member Typedef Documentation

typedef Orderings* KKMLL::Orderings::OrderingsPtr

Definition at line 62 of file Orderings.h.

Constructor & Destructor Documentation

Orderings::Orderings	(	const FeatureVectorListPtr	_data,
		kkuint32	_numOfOrderings,
		kkuint32	_numOfFolds,
		RunLog &	_log
	)

Constructs Orderings object from ImageFeatursList object.

Use this when an existing list does not exist. Will create 'numOfOrderings' separate lists of 'data' that are randomly ordered and stratified by 'numOfFolds'.

Parameters

[in]	_data	ImagFeaturesList object,
[in]	_numOfOrderings	Number of separate orderings of data need.
[in]	_numOfFolds	Used to help stratify data in each fold.

Definition at line 25 of file Orderings.cpp.

References KKB::KKStr::Empty(), KKMLL::FeatureVectorList::ExtractListOfClasses(), KKMLL::FeatureVectorList::FeatureVectorList(), KKMLL::FeatureVectorList::FileDesc(), KKMLL::FeatureVectorList::FileName(), KKB::KKStr::operator+(), KKB::KKStr::operator=(), and KKB::osRemoveExtension().

                      :
  
  data           (new FeatureVectorList (*_data, false)),
  fileDesc       (_data->FileDesc ()),
  mlClasses      (_data->ExtractListOfClasses ()),
  numOfFolds     (_numOfFolds),
  numOfOrderings (_numOfOrderings),
  valid          (true)

{
  featureFileName = data->FileName ();
  if  (!featureFileName.Empty ())
    indexFileName = osRemoveExtension (featureFileName) + ".idx";

  CreateOrderings (_log);
}

Orderings::Orderings	(	const KKStr &	_featureFileName,
		FeatureFileIOPtr	_driver,
		RunLog &	_log,
		bool &	cancelFlag
	)

Constructs Orderings of a FeatureVectorList from a previous construction that was saved in a data file.

Will load object from the Feature File '_featureFileName' and retrieve the different orderings from a separate index file who's name will be osExtention (FeatureFileName) + ".idx". The load routine will validate that all FeatureVector instances are accounted for in each ordering. If the index file does not exist it then the 'successful' flag will bet to false.

Parameters

[in]	_featureFileName	File to load FeatureVector' objects from. This will be used as master list for 'Orderings'.
[in]	_driver	Feature File driver to utilize.
[in]	_log	Log file to write messages to.
[in]	v	If flag turns to 'TRUE' then will terminate the load process and return to caller.

Definition at line 120 of file Orderings.cpp.

References KKMLL::FeatureVectorList::FileDesc(), KKB::KKStr::KKStr(), KKMLL::FeatureFileIO::LoadFeatureFile(), KKMLL::MLClassList::MLClassList(), KKB::KKStr::operator+(), KKB::KKStr::operator=(), and KKB::osRemoveExtension().

                     :
  
  data            (NULL),
  featureFileName (_featureFileName),
  fileDesc        (NULL),
  mlClasses       (NULL),
  numOfFolds      (0),
  numOfOrderings  (0),
  valid           (true)

{
  bool  changesdMade = false;
  bool  successful   = true;

  mlClasses = new MLClassList ();

  data = _driver->LoadFeatureFile (featureFileName, 
                                   *mlClasses,
                                   -1,  // Load in all data
                                   _cancelFlag,
                                   successful,
                                   changesdMade,
                                   _log
                                  );

  if  (_cancelFlag)
  {
    _log.Level (-1) << endl
                   << "Orderings   ***ERROR***   CancelFlag was set,   load was canceled." << endl
                   << endl;
    valid = false;
    return;
  }


  if  (!successful)
  {
    _log.Level (-1) << endl
                   << "Orderings   ***ERROR***   Loading Feature File[" << featureFileName << "]" << endl
                   << endl;
    valid = false;
    return;
  }

  fileDesc = data->FileDesc ();

  indexFileName = osRemoveExtension (featureFileName) + ".idx";
  Load (indexFileName, successful, _log);
  if  (!successful)
  {
    _log.Level (-1) << endl
                    << "Orderings   *** ERROR ***   Loading Index File[" << indexFileName << "]" << endl
                    << endl;
    valid = false;
    return;
  }
}

Orderings::Orderings	(	const FeatureVectorListPtr	_data,
		const KKStr &	_indexFileName,
		kkuint32	_numOfOrderings,
		kkuint32	_numOfFolds,
		RunLog &	_log
	)

Constructs a Orderings object from a FeatureLectorList object.

Will use 'data' as master list of FeatureVector objects.The orderings will be loaded from 'indexFileName'. It is expected that the size of the files will match. The load routine will validate that all FeatureVector objects are accounted for in each ordering.

Parameters

[in]	_data	Master List of FeatureVector instances.
[in]	_indexFileName	File where orderings of 'data' are to be loaded from.
[in]	_numOfOrderings
[in]	_numOfFolds

Definition at line 49 of file Orderings.cpp.

References KKMLL::FeatureVectorList::ExtractListOfClasses(), KKMLL::FeatureVectorList::FeatureVectorList(), KKMLL::FeatureVectorList::FileDesc(), KKB::KKStr::KKStr(), KKB::osFileExists(), and Save().

Referenced by CreateOrderingsObjFromFileIfAvaliable().

                      :
  
  data           (new FeatureVectorList (*_data, false)),
  fileDesc       (_data->FileDesc ()),
  mlClasses      (_data->ExtractListOfClasses ()),
  indexFileName  (_indexFileName),
  numOfFolds     (_numOfFolds),
  numOfOrderings (_numOfOrderings),
  valid          (true)

{
  if  (!osFileExists (indexFileName))
  {
    CreateOrderings (_log);
    Save ();
  }
  else
  {
    Load (_log);
  }
}

Orderings::Orderings	(	FeatureVectorListPtr	_data,
		RunLog &	_log
	)

Constructs a Orderings objects for a specified FeatureVectorList using a previously built Orderings data index file.

Will use FileName from "data" parameter to derive both 'featureFileName' and 'indexFileName' using the 'FileName' method from FeatureVectorList. It is expected that a separate index file by the name osDeletExtention (FeatureFileName) + ".idx" will exist. The orderings will be loaded from that file.

Parameters

[in]	_data	FeatureVectorList that we want different orderings of.
[in]	_log	Logger.

Definition at line 77 of file Orderings.cpp.

References KKB::KKStr::Empty(), KKMLL::FeatureVectorList::ExtractListOfClasses(), KKMLL::FeatureVectorList::FeatureVectorList(), KKMLL::FeatureVectorList::FileDesc(), KKMLL::FeatureVectorList::FileName(), KKB::KKStr::KKStr(), KKB::KKStr::operator+(), KKB::KKStr::operator=(), and KKB::osRemoveExtension().

                      :
  
  data           (new FeatureVectorList (*_data, false)),
  fileDesc       (_data->FileDesc ()),
  mlClasses      (_data->ExtractListOfClasses ()),
  indexFileName  (),
  numOfFolds     (0),
  numOfOrderings (0),
  valid          (true)

{
  featureFileName = data->FileName ();
  if  (featureFileName.Empty ())
  {
    _log.Level (-1) << endl << endl
                   << "Orderings   *** ERROR ***   No File Name in FeatureVectorList object." << endl
                   << endl;
    valid = false;
    return;
  }

  bool  successful = true;
  indexFileName = osRemoveExtension (featureFileName) + ".idx";

  Load (indexFileName, successful, _log);
  if  (!successful)
  {
    _log.Level (-1) << endl
                    << "Orderings::Orderings   Error Loading existing ordering[" << indexFileName << "]" << endl
                    << endl;
    valid = false;
  }

  return;
}

Orderings::~Orderings

(

)

Definition at line 185 of file Orderings.cpp.

{
  DeleteOrderings ();
  delete  data;
}

Member Function Documentation

OrderingsPtr Orderings::CreateOrderingsObjFromFileIfAvaliable ( const FeatureVectorListPtr  _data, kkuint32  _numOfOrderings, kkuint32  _numOfFolds, RunLog &  _log  )

static

Constructs a Orderings object for a specified FeatureVectorList.

Will use FileName from "_data" parameter to derive both 'featureFileName' and 'indexFileName' using the 'FileName' method from FeatureVectorList. If a separate Index file does not exist it will randomly create orderings and save the orderings in a new Index file.

Parameters

[in]	_data	FeatureVectorList that we want different orderings of.
[in]	_numOfOrderings	Expected number of orderings.
[in]	_numOfFolds	Number of folds each ordering should be stratified by.
[in]	_log	Logger.

Definition at line 193 of file Orderings.cpp.

References KKB::KKStr::Concat(), KKB::KKStr::Empty(), KKMLL::FeatureVectorList::FileName(), NumOfFolds(), NumOfOrderings(), KKB::KKStr::operator+(), Orderings(), KKB::osRemoveExtension(), Save(), and Valid().

{
  KKStr  _featureFileName = _data->FileName ();

  if  (_featureFileName.Empty ())
  {
    _log.Level (-1) << endl
                   << "CreateOrderingsObjFromFileIfAvaliable  *** ERROR ***  FileName empty." << endl
                   << endl;
    return NULL;
  }

  KKStr _indexFileName = osRemoveExtension (_featureFileName) + ".idx";
  
  OrderingsPtr  orderings = new Orderings (_data, _indexFileName, _numOfOrderings, _numOfFolds, _log);
  if  (orderings->Valid ())
  {
    if  ((orderings->NumOfOrderings () != _numOfOrderings)  ||  
         (orderings->NumOfFolds     () != _numOfFolds)
        )
    {
      _log.Level (-1) << endl
                     << "CreateOrderingsObjFromFileIfAvaliable   *** ERROR ***      Dimension Mismatched." << endl
                     << endl
                     << "Dimensions Expected   NumOfOrderings[" << _numOfOrderings              << "]  NumOfFolds[" << _numOfFolds              << "]" << endl
                     << "Dimensions Found      NumOfOrderings[" << orderings->NumOfOrderings () << "]  NumOfFolds[" << orderings->NumOfFolds () << "]" << endl
                     << endl;
      delete  orderings;
      return NULL;
    }
  }
  else
  {
    delete orderings;
    orderings = new Orderings (_data, _numOfOrderings, _numOfFolds, _log);
    if  (!orderings->Valid ())
    {
      delete  orderings;
      return  NULL;
    }
    else
    {
      orderings->Save ();
    }
  }

  return  orderings;
}  /* CreateOrderingsObjFromFileIfAvaliable */

FeatureVectorListPtr KKMLL::Orderings::Data ( ) const

inline

Definition at line 163 of file Orderings.h.

{return data;}

const KKStr& KKMLL::Orderings::FeatureFileName ( ) const

inline

Definition at line 164 of file Orderings.h.

{return featureFileName;}

const FileDescPtr KKMLL::Orderings::FileDesc ( ) const

inline

Definition at line 165 of file Orderings.h.

{return fileDesc;}

const KKStr& KKMLL::Orderings::IndexFileName ( ) const

inline

Definition at line 167 of file Orderings.h.

{return indexFileName;}

const MLClassListPtr KKMLL::Orderings::MLClasses ( ) const

inline

Definition at line 166 of file Orderings.h.

Referenced by KKMLL::CrossValidationMxN::RunTrainAndTest(), and KKMLL::CrossValidationMxN::RunValidations().

{return mlClasses;}

kkuint32 KKMLL::Orderings::NumOfFolds ( ) const

inline

Definition at line 168 of file Orderings.h.

Referenced by CreateOrderingsObjFromFileIfAvaliable().

{return numOfFolds;}

kkuint32 KKMLL::Orderings::NumOfOrderings ( ) const

inline

Definition at line 169 of file Orderings.h.

Referenced by CreateOrderingsObjFromFileIfAvaliable().

{return numOfOrderings;}

const FeatureVectorListPtr Orderings::Ordering

(

kkuint32

orderingIdx

)

const

Definition at line 614 of file Orderings.cpp.

Referenced by KKMLL::CrossValidationMxN::RunTrainAndTest(), and KKMLL::CrossValidationMxN::RunValidations().

{
  if  ((orderingIdx >= orderings.size ()))
  {
    KKStr errMsg;
    errMsg << "Orderings::Ordering  ***ERROR***  Index Out Of Range;  Number Of Orderings [" << (kkuint32)orderings.size ()  << "]   OrderingIdx [" << (kkuint32)orderingIdx        << "]";
    cerr << endl << errMsg << endl << endl;
    throw KKException (errMsg);
  }

  return  orderings[orderingIdx];
}  /* Ordering */

void Orderings::Save

(

)

Definition at line 551 of file Orderings.cpp.

References KKB::KKStr::Concat(), KKB::KKStr::Empty(), and KKB::KKStr::operator=().

Referenced by CreateOrderingsObjFromFileIfAvaliable(), Orderings(), and Save().

{
  // Build an index by relative location in master list data, so that we can 
  // quickly determine index for other orderings.
  map<FeatureVectorPtr,kkint32> index;
  kkint32  idx = 0;
  for  (idx = 0;  idx < data->QueueSize ();  idx++)
  {
    FeatureVectorPtr  example = data->IdxToPtr (idx);
    index.insert (pair<FeatureVectorPtr,kkint32>(example, idx));
  }

  KKStr  tempName = featureFileName;
  if  (tempName.Empty ())
    tempName = indexFileName;

  map<FeatureVectorPtr,kkint32>::const_iterator  indexIDX;

  ofstream  o (indexFileName.Str ());

  o << "//Orderings"                                        << endl;
  o << "//FeatureFileName" << "\t" << tempName              << endl;
  o << "//NumOfOrderings"  << "\t" << numOfOrderings        << endl;
  o << "//NumOfFolds"      << "\t" << numOfFolds            << endl;
  o << "//DateCreated"     << "\t" << osGetLocalDateTime () << endl;
  o << "//EndOfHeader"                                      << endl;

  kkuint32  orderingIDX;

  for  (orderingIDX = 0;  orderingIDX < orderings.size ();  orderingIDX++)
  {
    FeatureVectorListPtr  ordering = orderings[orderingIDX];
    o << "//OrderingNum" << "\t" << orderingIDX << "\t" << "Count" << "\t" << ordering->QueueSize () << endl;

    FeatureVectorList::const_iterator idx;
    for  (idx = ordering->begin ();  idx != ordering->end ();  idx++)
    {
      const FeatureVectorPtr  example = *idx;
      indexIDX = index.find (example);
      if  (indexIDX == index.end ())
      {
        // We have a very serious problem,  for some reason we 
        // could not locate the FeatureVector object in the master list.

        KKStr errMsg;
        errMsg << "Orderings::Save  ***ERROR***   FileName[" << indexFileName << "]   Could not locate Image in data list";
        cerr << endl << errMsg << endl << endl;
        throw KKException (errMsg);
      }

      o << indexIDX->second << endl;
    }
    o << "//EndOfOrdering" << "\t" << orderingIDX << endl;

    cout << "Saved " << orderingIDX << " of " << orderings.size () << endl;
  }

  o.close ();
}  /* Save */

void Orderings::Save	(	const KKStr &	_indexFileName,
		RunLog &	_log
	)

Definition at line 528 of file Orderings.cpp.

References KKB::KKStr::Empty(), KKB::KKStr::operator=(), KKB::osWaitForEnter(), and Save().

{
  if  (indexFileName.Empty ())
  {
    // We have a problem,  no way of creating a file name.
    _log.Level (-1) << endl << endl
                   << "Orderings::Save    *** ERROR ***    No Index File Name Specified." << endl
                   << endl;
    osWaitForEnter ();
    exit (-1);
    return;
  }

  indexFileName = _indexFileName;
  Save ();
}  /* Save */

kkuint32 KKMLL::Orderings::Size ( ) const

inline

Definition at line 170 of file Orderings.h.

{return (kkuint32)orderings.size ();}

bool KKMLL::Orderings::Valid ( ) const

inline

Definition at line 171 of file Orderings.h.

Referenced by CreateOrderingsObjFromFileIfAvaliable().

{return valid;}

---

The documentation for this class was generated from the following files:

• C:/Users/Kurt/GitHub/KSquareLibraries/KKMachineLearning/Orderings.h
• C:/Users/Kurt/GitHub/KSquareLibraries/KKMachineLearning/Orderings.cpp