and your data collection
set-up as well as to test
the training on solutions
evaluation coordinator practitioner coordinator
trial owner
solution providers
CM practitioners
In a nutshell
What this step
is about
In this step, the trial design and all test-bed technical infrastructure arrangements are tested at the location(s) where the actual trial will take place. This concerns both technical and non-technical issues. The aim is to test whether or not the results of all six steps have been implemented correctly and are clear for the involved stakeholders and/or users. As this is focused on functionality, you may start with the use cases and then go through the whole scenario.
In depth
All you need to know
about this step
This step contains the final tests and adaption of each trial sub-system and should end with a complete trial dry run.
From a technical perspective: make sure the test-bed technical infrastructure is up and running under the conditions the trial needs: at the location, with all necessary solutions connected. Do a stress-test. Try all needed kinds of input - and some that a creative end user might come up with. (People usually don’t stick to the script - especially as they are not able to learn it by heart in the short amount of time).
While the technical crew is setting up, review your injects (the things that have to happen, in order to trigger the gap-behaviour). Test those injects! While doing that, check whether you can really collect the data you need to collect (within the test-bed technical infrastructure, the solutions and with the use of human observers). Based on this test, you can assign the number of observers you need to the rooms and points in time - and write down the instruction for their observation. In the end take enough time to hear from everyone what worked well and where there is room for improvement. Create a to-do-list with clear assignments and start the preparation of dry run 2.
Methods
Technical test, roleplay
Tools
Output
Proof of concept of data collection
and evaluation plan, to-do-list
Checklist
- Data collection plan & evaluation approach reviewed in practice
- Scenario and injects reviewed in practice
- Training on solutions tested
- Readiness review of solutions and technical integration conducted
- Local test-bed technical infrastructure adaptation reviewed
- Solutions approved
- Needed roles reviewed in practice
- Make sure legal (e.g. GDPR) and ethical issues (e.g. use of real tweets) concerning the solutions are covered.