Data Export

Paradata file format

Every survey collected in Survey Solutions is supplemented with a paradata file. No actions are needed from the questionnaire designers or headquarters team to collect it, it is produced automatically by the Survey Solutions software. The paradata files describe the process of data collection. They explain how the data was entered, detailing all edits, who and when undertaken them. These files may be large and most conveniently processed using specialized statistical packages, rather than general purpose tools.

January 30, 2024

Interview diagnostics file

interview__diagnostics file is a special service file, automatically produced with every data export, that contains useful information for assessing the quality of interviews: number of invalid entities (invalid answers and errors in static texts), number of questions with comments, duration of the interview etc. For a description of available fields and their types, refer to this article. This is especially useful for user-written monitoring systems that work alongside Survey Solutions and continually monitor the quality of incoming data by periodically exporting the data and building customized reports.

December 5, 2023

Audio Audit Files

Survey Solutions has a feature of audio audit: a capability of recording an interview and provide the recording for the audit of the interviewing process. The recording covers all the duration of the interview from the moment it starts to completion, except the time when: the tablet is off; the user switches to a different app; the interview is not active (Interviewer has switched to a different interview or switched to the dashboard); the audio recording is done as an answer to an audio question; when the interview is completed over the web (CAWI mode).

February 14, 2022

How does Survey Solutions identify records and establish connections between data?

Survey Solutions works with various surveys done in different conditions, so trying to come up with one naming scheme that would be equally suitable for all applications is a challenging task. A particular survey may come up with a numbering like 2 digits of the province code, followed by 3 digits of the district code, followed by the household id of 4 digits. Yet it is not a good identifier in the case where several surveys are conducted on the same server, or there may be repetitive visits, and other twists.

October 5, 2020

Organization of meta-data storage in the exported data

Survey Solutions export file is a zip-archive with multiple data files corresponding to various levels of data and system-generated accompanying data reflecting the comments, errors, statuses and other technical information about interviews and assignments. When the corresponding setting is activated in the export interface, the data is accompanied with the meta-data reflecting the structure of the questionnaire. The meta-data is saved to the subfolder Questionnaire of the export archive. The questionnaire subfolder contains the following information:

September 17, 2020

System generated files

Every archive of data exported from the Survey Solutions Headquarters contains among the user data the following system-generated data files: assignment__actions interview__actions interview__comments interview__diagnostics interview__errors The above data files are saved in the same data format as the main survey data, for example, if the main data is requested in Stata format, the system-generated data files will also be exported in Stata format. The data files are also accompanied by an automatically generated text file named export__readme.

October 1, 2019

Export to cloud storage

Binary data files, such as pictures and audio recordings are often large and can be time-consuming to download. One can now opt for pushing the data to own cloud storage, rather than downloading directly. The following cloud storage repositories are supported: DropBox; Google Drive; OneDrive. The direct download of binary data in the form of a single large zip-archive remains available. To push the data to the cloud storage, select the corresponding option in the download dialog, then authenticate yourself with the corresponding cloud storage provider:

June 5, 2019

Merging files of different data levels

Survey Solutions produces export data files separately for each data level. This can be for example households, persons, plots, crops, etc. The process is completely automatic and not configurable by the user. If you need to combine information from different data levels (for example bring household characteristics to the personal level) you can utilize a statistical package to post-process your data. This is a basic operation supported by most packages, such as SAS, SPSS, Stata, R and others, though the exact terms may differ by package.

January 7, 2019

Export of calculated variables

Calculated variables are commonly used in CAPI questionnaires to implement complex checks or indicate to the interviewer some derived information. For example, one may create a variable that calculates the number of children in the household, numChildren = Persons.Count(person => person.age < 18) and use this variable in subsequent questions to check if the number of children in a household exceeds a specific threshold. Starting with version 5.24 of Survey Solutions these variables will be automatically included in the export data.

September 28, 2018

Missing values

Survey Solutions recognizes two situations of missingness: the value is missing because the logic of the questionnaire is disabling (skipping) this question. There is nothing that the interviewer can do to enter a value in such a field and in some cases even to see the question and its options. the value is missing despite the question is enabled (not skipped). This could be because of any of the following: interviewer hasn’t reached this point in the interview; interviewer missed the question, didn’t ask it and hence no answer was recorded; none of the options was suitable to accommodate the respondent’s answer and hence the interviewer didn’t record the answer; other similar situations.

June 13, 2017