Export of data may be requested at any time when the Survey Solutions server is running (during or after the survey) and it doesn’t cause any interruption of the synchronization (of mobile devices) or online work (CAWI surveys). In case the server is busy processing other export requests, the new export jobs are put into a queue and processed sequentially. This article describes how to request a data export from Survey Solutions.
In the main menu select
Data Export.Select the questionnaire corresponding to the survey that needs to be exported.
Select the questionnaire version (from available at the server).
Select the status of interviews that need to be exported or leave blank for export of all interviews (in any status).
Select data type, main survey data, binary data, or paradata.
Select whether to include metadata (does not apply to paradata export).
Select the format of the data file (does not apply to paradata export):
- Tab-separated data (*.tab) - unicode text format with tab-character used as a delimiter.
- Stata format (*.dta) - Stata statistical package format, Stata version 14 or newer can open these files.
- SPSS format (*.sav) - SPSS statistical package format.
Select the export file destination (Binary data files, such as pictures and audio recordings are often large and can be time-consuming to download. One can opt for pushing the data to own cloud storage, rather than downloading directly):
- Download - exported data will be prepared for downloading to the user’s machine.
- Upload to OneDrive - when the data is ready it will be pushed to OneDrive cloud storage.
- Upload to DropBox - when the data is ready it will be pushed to DropBox cloud storage.
- Upload to Google Drive - when the data is ready it will be pushed to Google Drive cloud storage.
Press the
Add to Queuebutton to create a new export job and add it to the queue of export jobs.
If the queue is empty, a new job is immediately picked for processing.
Once an export job is processed, the result is either uploaded to the cloud
destination, or a Download button is presented to the user in a card
corresponding to the export job.
The cards are placed in the queue regardless whether the export was requested directly from the web interface, or by submitting the corresponding request via the Survey Solutions’ API.
If you’ve chosen export to the cloud storage, authenticate yourself with the corresponding cloud storage provider. Note that despite the cloud-to-cloud data transfer is usually faster than direct download, it may still take significant time to complete, especially for large surveys.
Statuses of interviews
In step 4 the user may select all interviews by not specifying any particular status, or rather select one of the following:
interviewer assignedcompletedapproved by supervisorapproved by headquarters
Note that selection is based on the current status of interview when the export
of data is requested, and not on any of the past statuses that the interview
had. For example, if the interview is currently in the status
Approved by Headquarters it will not get exported when the status Completed
is specified, despite having one or more Completed events in the past.
Metadata
If metadata is requested to be included with main survey data, it includes:
- a DDI description file (in XML format);
- PDF documents containing textual representation of the questionnaire (1 document per each translation of the questionnaire);
- a questionnaire document (in JSON format);
- questionnaire attachments (images, and other files that may be embedded into the questionnaire).
See more on this in Organization of meta-data storage in the exported data.
Export card
Export cards corresponding to past export jobs can also be seen here.
Each such card compactly displays a great amount of information about the export job:
| # | Explanation | In this example |
|---|---|---|
| 1. | Export job number | 21131 |
| 2. | Job status | COMPLETED |
| 3. | Timestamp when the export job was created | May 01, 2024 22:08 |
| 4. | Card menu button. | |
| 5. | Title of the questionnaire and version | "2023_24 Kenya Integrated Survey of Agriculture Pilot" version 2 |
| 6. | Format of export data | paradata |
| 7. | Selection of interviews for export | interviews in all statuses |
| 8. | Selection of language/translation | original language |
| 9. | Destination | the job prepared a file for downloading by the user |
| 10. | Time spent in queue | about a minute |
| 11. | Time spent on production of the export data | a few seconds |
| 12. | Timestamp when this job's output was last updated (output can be regenerated, see below) | May 01, 2024 22:08 |
| 13. | Approximate size of the file for download | about 0.5 MB |
| 14. | DOWNLOAD button (during production of the export data, while the DOWNLOAD button is not available, this space is utilized to display the current export step, such as RUNNING or COMPRESSING). |
Export card menu
Export card menu (4) provides only one item, which allows to regenerate the export of data with the same parameters, as for the current card.
When regeneration is started, a new export job is created and placed into the queue (and a new card appears in the export queue with a new job number). However, when the job completes, its result overrides all other jobs’ export data matching the same parameters. Hence there are two timestamps on each card, reflecting the point of processing of the job (3) and update of the data file (12).
See also
- Description of the paradata file format;
- Description of the system-generated data files.