org.jscience.computing.ai.reinforcement.qlearning
Class QLearner

java.lang.Object
  org.jscience.computing.ai.reinforcement.qlearning.QLearner

public abstract class QLearner
extends java.lang.Object

Q-learning algorithm implementation. You should override this class for your requirements.

Field Summary

static int

EpsinonGreedy "Epsinon Greedy" selection method for selecting actions

Constructor Summary

QLearner()

Method Summary

java.util.Collection	getBestActions() Returns all of the best actions for each states tried so far
double	getBestValue(java.lang.Object state) Returns best reinforcement value for given state
double	getDiscountFactor() Returns discount factor
double	getLearningRate() Returns learning rate
int	getNumberOfStates() Returns the number of different states in the q-table
int	getQTableSize() Returns the number of entries in q-table
java.util.Enumeration	getStates() Returns different states in the q-table.
void	initialize() Initializes the learner with default parameters.
void	initialize(double learningRate, double discountFactor, int selectionPolicy) Initializes the learner with given parameters.
java.lang.Object	selectAction(java.lang.Object state) Selects an action based on given state
protected abstract java.lang.Object	selectRandomAction(java.lang.Object state) Selects an action randomly.
void	setDefaultValue(int defaultValue) Changes default valee for q-table entries.
void	setDiscountFactor(double dicountFactor) Changes the discount factor
void	setLearningRate(double learningRate) Changes the learning rate
void	setSelectionPolicy(int selectionPolicy) Changes the selection method
void	updateQTable(java.lang.Object state, java.lang.Object action, java.lang.Object newState, double reward) Updates q-table with given state, action, new state and reward information

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

EpsinonGreedy

public static final int EpsinonGreedy

"Epsinon Greedy" selection method for selecting actions

See Also:

Constant Field Values

Constructor Detail

QLearner

public QLearner()

Method Detail

initialize

public void initialize()

Initializes the learner with default parameters. Default parameters are listed below:

learningRate = 0.2; discountFactor = 0.9; epsinon = 0.75; defaultValue = 0.0; selectionPolicy = QLearner.EpsinonGreedy;

initialize

public void initialize(double learningRate,
                       double discountFactor,
                       int selectionPolicy)

Initializes the learner with given parameters.

Parameters:

learningRate - learning rate

discountFactor - discount factor

selectionPolicy - selection method

setLearningRate

public void setLearningRate(double learningRate)

Changes the learning rate

Parameters:

learningRate - new learning rate

setDiscountFactor

public void setDiscountFactor(double dicountFactor)

Changes the discount factor

Parameters:

dicountFactor - new discount factor

setSelectionPolicy

public void setSelectionPolicy(int selectionPolicy)

Changes the selection method

Parameters:

selectionPolicy - new selection method

setDefaultValue

public void setDefaultValue(int defaultValue)

Changes default valee for q-table entries.

Parameters:

defaultValue - new default value

getLearningRate

public double getLearningRate()

Returns learning rate

Returns:

learning rate

getDiscountFactor

public double getDiscountFactor()

Returns discount factor

Returns:

discount factor

getQTableSize

public int getQTableSize()

Returns the number of entries in q-table

Returns:

size of the q-table

getNumberOfStates

public int getNumberOfStates()

Returns the number of different states in the q-table

Returns:

number of different states in the q-table

getStates

public java.util.Enumeration getStates()

Returns different states in the q-table.

Returns:

different states in the q-table

getBestActions

public java.util.Collection getBestActions()

Returns all of the best actions for each states tried so far

Returns:

all of the best actions

getBestValue

public double getBestValue(java.lang.Object state)
                    throws QTableEntryNotFoundException

Returns best reinforcement value for given state

Parameters:

state - state to look for best reinforcement value

Returns:

best reinforcement value for given state

Throws:

QTableEntryNotFoundException - if the state is not found in q-table

updateQTable

public void updateQTable(java.lang.Object state,
                         java.lang.Object action,
                         java.lang.Object newState,
                         double reward)

Updates q-table with given state, action, new state and reward information

Parameters:

state - state before the action is done

action - action performed for given reward

newState - new state observed after the action done

reward - reinforcement taken after the action is done

selectAction

public java.lang.Object selectAction(java.lang.Object state)

Selects an action based on given state

Parameters:

state - state to select action for

Returns:

selected action

selectRandomAction

protected abstract java.lang.Object selectRandomAction(java.lang.Object state)

Selects an action randomly. This method should be implemented in subclasses, since possible actions are not known by QLearner class.

Parameters:

state - state to select an action randomly

Returns:

selected action