The Survey Solutions software is being very actively developed and every
new release adds new features, optimizations and efficiency
improvements. However, sometimes the support team receives inquiries
regarding the decrease of performance of the interviewer’s app, perhaps
after an update. Here it is important to make sure that a valid
comparison is made. A few common explanations to consider:
�
| Reason | Explanation |
| 1. The questionnaire has changed | This is by far the most common reason. The user tests a version of the questionnaire, then sends it to partners for feedback, then incorporates "the finishing touches": add two checks here, two checks there, two hundred checks over there. By the time all of these changes are made the questionnaire may not even resemble the original version before the pilot. Comparisons are valid provided that the questionnaire is invariant! |
| 2. Different testing scenario. | The amount of checks performed by the device is data dependent. If the tests were done for "a family of two" the responsiveness may be very different when you redo it for "a family of twelve". |
| 3. The tablet could have changed. | The user tests the questionnaire using a tablet specifically purchased for the development team. But for the data collection team, a cheaper model questionnaire is purchased. It is expected the hardware will affect the performance. Test the questionnaire performs responsively on the model that you will be using for data collection! |
| 4. The performance affected by 3rd party apps. | Check if there are any other apps running on the tablet that may affect the performance. Remember that they may update themselves even without your involvement if application auto update is activated. They may compete for resources with Survey Solutions and decrease the performance. Install only what you need. Uninstall everything you don't. Restrict interviewers' accounts to eliminate possibility of installing unwanted apps. |
| 5. Imperfect measurement. | User tested the "speed of the questionnaire" by measuring the total time it takes to enter some values. In the field the time became longer. This can be due to any of the reasons related to the real communication between two people (need to repeat questions, revise earlier answers, think about what is being asked, what are the answers, how to record them into the questionnaire, etc) compared to a rehearsed procedure of entering a specific test example. |
| 6. Comparing apples with oranges. | "The questionnaire used to work faster before! Yes, before it was not in Survey Solutions, and not on an Android tablet, but on a Windows server, and we didn't have any validation checks or calculated expressions, but it was faster". No comment. |
| 7. Wishful thinking. | No objective measurement was taken to measure the performance of the questionnaire during testing, then when the survey started and things start to matter, the questionnaire performance is deemed inadequate. Test during preparation. |
Survey Solutions has very flexible and powerful language and multiple strategies for questionnaire implementation. Copying paper questionnaires is usually a common mistake that leads to inefficient questionnaires and inadequate responsiveness. A second opinion of a reviewer helps identify these inefficiencies and get an advice on how to improve the questionnaire.