You could do that. It would mean using the instantaneous error (the error on the current trial) instead of a moving average of the error. What is the point of my using the latter, when the fitness itself is already a moving average? Probably this does overdo it. I don't know because I haven't tried the other way.

However, having the prediction error statistic can be useful in its own right. For example, in recent experiments on the explore/exploit tradeoff, I've used the error (meaning the moving average) for decisions as to whether a given action has been sufficiently tried. A large error may mean the prediction is not well established, and more exploration of that action should occur.


The only place the prediction error parameter is used is in the fitness update calculation. Why not simply use the absolute difference between P and p sub j in the fitness update?