What is an Evaluator?¶
Evaluators are modules that measure how well organisms perform under a given circumstance. They generally correspond to problems/fitness landscapes. When used on a population, they assign a trait to each organism indicating how well it performed. In simple configurations, this trait may be used directly as a “fitness” criterion on which to base selection.
Using an evaluator¶
In addition to other configurable parameters that are specific to a given evaluator, all evaluators have a parameter called fitness_trait
.
This parameter indicates the name of the trait where the score calculated by the evaluator should be stored.
All evaluators have an EVAL
member function. This function takes a list of organisms as an argument and runs the evaluation function on all of them, storing the results in the trait designated by the fitness_trait
parameter.
For example, the NK evaluator can be used as follows:
// Create a population
Population main_pop; // Main population for managing candidate solutions.
// Configure an EvalNK module with the name eval_nk
EvalNK eval_nk { // Evaluate bitstrings on an NK fitness lanscape.
N = num_bits; // Number of bits required in output
K = 3; // Number of bits used in each gene
bits_trait = "bits"; // Which trait stores the bit sequence to evaluate?
fitness_trait = "fitness"; // Which trait should we store NK fitness in?
};
// Other modules would be configured here
// Configure what happens on each time step
@UPDATE(Var ud) {
// Run the evaluator on the population
eval_nk.EVAL(main_pop);
}
Writing an evaluator¶
Writing an evaluator module is the same as writing any other type of module, except for the following:
The constructor must call
SetEvaluateMod(true);
to tell MABE that this is an evaluator module.The module must have a config parameter called
fitness_trait
that specifies where the calculated fitness should be stored.The module must have a member function named
EVAL
that takes aCollection
as input, runs the evaluator on each member of the collection, and stores the result infitness_trait
.
Example:
// All modules should be in the MABE namespace
namespace mabe {
// All modules should inherit from Module
class EvalExample : public Module {
private:
// The trait that fitness should be stored in
std::string fitness_trait;
public:
EvalNK(mabe::MABE & control,
const std::string & name="EvalNK",
const std::string & desc="Module to evaluate bitstrings on an NK Fitness Lanscape",
const std::string & _ftrait="fitness")
: Module(control, name, desc)
, fitness_trait(_ftrait)
{
// Notify MABE that this is an Evaluate module
SetEvaluateMod(true);
}
~EvalNK() { }
// Setup member functions associated with this class.
static void InitType(emplode::TypeInfo & info) {
// Setup the EVAL function
// The arguments are a reference to this module and a Collection of organisms
// For convenience, this just calls a member function called Evaluate, which
// we'll define a little later
info.AddMemberFunction("EVAL",
[](EvalExample & mod, Collection list) { return mod.Evaluate(list); },
"Evaluate all orgs in an OrgList.");
// Other functions you want the user to be able to call in the config file
// could be set up here
}
// Setup the config settings for this module
// This needs to include fitness_trait, but could also
// include anything else you want the user to be able to configure
void SetupConfig() override {
// Our fitness_trait member variable will end up containing a string
// (which by default is "fitness_trait") containing the name of the trait
// where the fitness will be stored
LinkVar(fitness_trait, "fitness_trait", "Which trait should we store fitness in?");
// Other config parameter could be set up here
}
// Tell the module about any traits that it depends on
void SetupModule() override {
// Setup the traits.
// This module creates the fitness_trait trait, which defaults to 0
AddOwnedTrait<double>(fitness_trait, "Fitness value", 0.0);
// Other setup could happen here
}
// Actually implement Evaluate method that we used earlier in the
// EVAL function
double Evaluate(const Collection & orgs) {
// Loop through the population and evaluate each organism.
// If you want to only evaluate orgs that are alive,
// it may be helpful to create a new collection that
// only contains the organisms within the original collection
// that are actually alive.
mabe::Collection alive_orgs( orgs.GetAlive() );
// Keep track of the maximum score because
// Evaluate needs to return it
double max_score = 0.0;
// Loop over all the organisms
// (alive_orgs could be replaced with orgs if we didn't bother
// with the filtering step above)
for (Organism & org : alive_orgs) {
// If you're going to look at any other traits to evaluate
// fitness, make sure to call GenerateOutput on the organism
// to ensure all traits are populated
org.GenerateOutput();
// Figure out what you want this organism's fitness to be
// (you probably want to do something more elaborate here)
double fitness = 1;
// Set this organisms fitness trait equal to the calculated fitness
org.SetTrait<double>(fitness_trait, fitness);
if (fitness > max_score) {
max_score = fitness;
}
}
return max_score;
}
// Alternate version of Evaluate that takes a Population instead of a Collection
// If a population is provided to Evaluate, first convert it to a Collection.
double Evaluate(Population & pop) { return Evaluate( Collection(pop) ); }
// Alternate version of Evaluate that takes a string instead of a Collection
// If a string is provided to Evaluate, convert it to a Collection.
double Evaluate(const std::string & in) { return Evaluate( control.ToCollection(in) ); }
};
// You always need to call MABE_REGISTER_MODULE after defining a new module,
// to inform MABE that the module exists.
// Remember to also add it in modules.hpp as an include.
MABE_REGISTER_MODULE(EvalExample, "Example evaluator.");
}