Experiments


Introduction

Experiments are powerful tools for deriving insights from simulation models. A single experiment run typically consists of many individual simulation runs, with a large number typically occurring in parallel.

These simulation runs occur with subtle differences (including by varying initial states, agent populations, environmental conditions, and noise parameters).

Creating experiments

In HASH, experiments are defined in the experiments.json file. Currently supported are:

  • linspace - vary a single parameter within a range
  • arange - vary a parameter based on an increment
  • values - manually enter values for a specific parameter
  • monte-carlo - generate random numbers according to a distribution
  • group - group together multiple experiment types into a single experiment
  • multiparameter - create all possible parameter combinations from multiple experiments
  • optimization - efficiently identify optimal parameter combinations that minimize or maximize desired metrics

Experiments can be created with the Experiments Wizard, or by manually defining them in the experiments.json file. You can access the wizard through the Experiments icon (beaker) in the runner controls, or from the menu bar along the top of hCore.

The Experiment Creation Wizard

For example, this values experiment will run seven experiments, setting a different value of [0..6] in the radius field in each one.

experiments.json
{
    "Sweep values": {
        "type": "values",
        "field": "radius",
        "values": [0,1,2,3,4,5,6],
        "steps": 100
    }
}

To run an experiment, click the "Experiment Runner" button in the runner controls, denoted with a beaker icon. The option "Sweep Values" will be available in the selector.

Sweep values demonstration

If you want to run an experiment from another user's published simulation - for instance an example simulation - first save a copy to your drive and then click the experiment runner.

Running this experiment will generate 7 new simulations, each with a slightly different globals.json. If we run the simulation, we can see exactly which parameters get changed in the sidebar:

Radius options

To access the changed varied parameter from within the simulation, we simply access that parameter from context.globals():

const behavior = (state, context) => {
  const { radius } = context.globals();
};

Now, any behaviors that rely on the "radius" parameter from globals.json will use the corresponding value.

You can run experiments locally or in hCloud

For more information on specific syntax, read more about Experiment Types.

Join our community of HASH developers