Notes

A typical
cycle of training or testing of an autonomous agent involves: i) resetting the NAS to get the start
state, ii) using the state to choose an action, iii) executing the action against the environment
using the step function to receiving the next state and reward, iv) repeat steps (ii) and (iii) until
the goal is reached or the time limit expires. This complete cycle would be a single episode and
an agent may then reset the environment and repeat this for as many episodes as desired.

Parameter Description
subnets The number and size of each subnet
topology Connectivity of each subnet defined using an adjacency matrix
sensitive machines The address and value of each sensitive machine on the network
services The number of possible services running on any given machine
service exploits The cost and success probability of each service exploit
machine configurations Which services are running on each machine in the network
firewalls Which service traffic is permitted along each subnet connection on
network

Specifically, the probabilities were chosen based on the
attack complexity score for CVSSv2, which scores exploits as having either ‘low’, ‘medium’ or
‘high’ attack complexity [41]. We take the same approach as Backes et al [38] and set
probability of success to be 0.2, 0.5 and 0.8 for ‘low’, ‘medium’ or ‘high’ attack complexity
41
respectively. Using this approach we hope to try and model the distribution of exploit success
probabilities found in the real world