The Experiment Design is the subset of combinations to implement the experiment. A good design will contain enough systematic variation between scenarios to allow modelling to be successful.
The SurveyEngine system automatically generates designs to fit the current experiment, this screen will also perform simulations on the design and give estimates on the likely accuracy of the design.
- To see currently used experiment design click [Experiment Design] button
- You will be redirected to the experiment design screen. Initially the screen will look like this:
- To generate or update the experiment design click [Generate experiment design] button. This will generate a design to fit the current experiment.
→ Note: if you go back and change the experiment you will need to re-generate the design. - Insert scenarios into the survey.
→ Note: The more scenarios – the more data is collected from each person and therefore the smaller sample that is required. Check the number of rows of the design - the number of scenarios should not exceed this. Typically 8 scenarios per respondent is considered adequate without too much of a burden on the respondent. - To simulate responses click [Simulate Responses] button.
→ Note:- This is optional – however it is useful as it simulates the effectiveness of the current design and shows the accuracy at different sample sizes.
- Automated agents simulate a preference for the different attributes and the system attempts to recover their internal preferences. The estimates are compared with the actual and the differences shown.
- because the simulations use randomised data – estimates may change slightly each time they are run. Simulations may take up to 5 minutes for large experiments.
- Afterwards the screen will look like this
The simulations indicate that 320 respondents required for modelling with each respondent performing 1 scenario for a valid model, however with a simulated variation of about 4.3%.
Attachments:
0 Comments