Telephone Surveys

Introduction If researchers wish to have survey interviews carried out by an interviewer but face-to-face interviews are not possible, conducting interviews via telephone (either through a landline or a mobile telephone) can be an alternative. Multinational, multiregional, and multicultural ('3MC') surveys use different standards to determine whether telephone penetration is adequate in a study country. For example, the Gallup World Poll generally uses a telephone survey only in countries where telephone coverage represents at least 80% of the population [zotpressInText item="{2265844:HHWZAXWB}"]. Telephone interviews are generally less costly than face-to-face methods, and can be completed in a shorter amount of time. However, response rates are generally lower, and depending on the available sampling frame for a country, a rigorous telephone-administered sample design can be difficult to develop. See Sample Design for a discussion of the challenges and limitations of a telephone-based frame and sample design. As discussed in Data Collection: General Considerations, 3MC surveys sometimes employ mixed modes, depending on individual country constraints. However, it is important to note that mode effects may occur if the survey is carried out by telephone in some countries and face-to-face in others (see Study Design and Organizational Structure for discussion on mode effects). Virtually all questionnaires administered by interviewers in telephone surveys are completed using an electronic computer-based instrument to record survey responses. This data collection mode is most commonly referred to as computer-assisted telephone interviewing (CATI). These guidelines assume that the interviewer will be using a computer-based instrument, and will refer to the mode as 'CATI' as such. For additional discussion on the advantages and disadvantages of telephone surveys, see Study Design and Organizational Structure.

⇡ Back to top

Guidelines Goal: To achieve an optimal cross-cultural data collection design by maximizing the amount of information obtained per monetary unit spent within the allotted time, while meeting the specified level of precision and producing comparable results, within the context of a telephone survey.

⇡ Back to top

1. Develop the computer-based system(s) that the interviewers will use to administer telephone interviews. Rationale Interviewers can conduct telephone interviews from either a central location or remotely. Software systems can be used to distribute sampled telephone numbers, dial telephone numbers, manage call records, and record survey data. When using CATI, it is crucial to design and implement a system that interviewers can use to reliably collect survey data. Procedural steps

1.1     Decide whether interviewers will work in a centralized and/or decentralized location.

1.1.1    Many survey research firms conducting telephone interviews maintain a 'telephone lab,' which is a central calling center where center supervisors oversee a variable number of interviewers. Each interviewer has access to the electronic instrument, and records responses directly in the electronic file. Interviews can be monitored in real time.

1.1.2    Sometimes, interviewers work from other locations while having access to the electronic system set up by the survey research firm.

1.2     Develop a system and protocol for sample release management, including how cases will be transferred between interviewers when necessary.

1.3     Develop a protocol for dialing sampled telephone numbers. Some projects may use CATI systems that can dial telephone numbers automatically, while other projects may elect to have interviewers dial telephone numbers manually. In some countries, it is against the law to use automation to dial specific types of telephone numbers (e.g., in the United States, it is illegal to use automation to dial mobile numbers). If using automation, be familiar with the local laws about its use.

1.4     Consider the cost structure for telephone calls in each study country. In the United States, respondents are responsible for the cost of incoming telephone calls on mobile telephones. However, in the Persian Gulf, for example, there is no charge, and interviewers based in Nepal were able to telephone Nepali migrant workers living in Gulf countries for a migration survey without any cost to the respondents [zotpressInText item="{2265844:I3LPNV6L}"].

1.5     Decide on a telephone number and name to be displayed to the respondents in their caller ID, and whether the number should be available if people call it back.

1.6     Develop an electronic survey instrument used to record survey responses. There are numerous CATI software packages. However, it is also possible to use a Web-based survey instrument, which may not be as suitable for more complex projects, but would be less expensive. Electronic survey instruments in a telephone survey share many of the same requirements as electronic survey instruments administered in the face-to-face mode. For in-depth discussion of these elements, see Data Collection: Face-To-Face Surveys, Guideline 3.

Lessons learned

1.1     While survey mode can affect survey responses, studies are not unanimous in the direction of the effect observed.

1.1.1    A survey of HPV awareness and knowledge, including sexual behavior, was conducted in Singapore, with half participating via CATI and half through an interviewer-administered face-to-face interview. Few differences between survey modes were found in the information disclosed [zotpressInText item="{2265844:CCXQQK9A}"].

1.1.2    A study in India evaluating accuracy of health data collection through several different interfaces found that telephone interviewing had greatest accuracy in phone interviews when compared to electronic forms on PDAs and text messaging [zotpressInText item="{2265844:UY685SDY}"].

1.2     CATI can be particularly useful in a panel study setting, especially when there is frequent contact with respondents. Experiences vary by country, however.

1.2.1    In a study of farmers in Tanzania, researchers gave respondents prepaid mobile phones for the duration of the field period so that they could receive a phone call from an interviewer and complete a survey every three weeks over a ten-month period, resulting in a high-quality dataset [zotpressInText item="{2265844:MMX3PDUW}"].

1.2.2    Researchers distributed mobile phones to female sex workers in India for use in a diary study on sexual behavior, which resulted in high response rates and high-quality data [zotpressInText item="{2265844:UZ5ARH7A}"].

1.2.3    Researchers on a panel study in South Sudan using CATI found that response rates were affected by irregular fluctuations in the mobile network [zotpressInText item="{2265844:DNG7C4TI}"].

1.3     Beyond the traditional CATI mode, interviewing via text message has been recently used. In this mode, the interviewer sends individual survey questions by text to the respondent, who sends their responses back by text to the interviewer [zotpressInText item="{2265844:WPJHHNRC},{2265844:ASKB4GKV}"].

⇡ Back to top

2. Train interviewers on interviewing strategies specific to telephone interviewing. Rationale The nature of the interaction between the interviewer and the respondent depends on the mode of data collection. Some interviewing strategies that are accessible in a face-to-face mode, such as interpretation of body language, are not possible to implement over the telephone, contributing in part to lower response rates and potential for nonresponse bias. However, there are certain telephone-specific strategies that researchers can introduce to assist interviewers in completing telephone interviews. Procedural steps

2.1     Consider the social context of the study country when hiring interviewers to administer a telephone survey, and whether selection of interviewer based on gender or other characteristics will affect response rates. See 2.1 in 'Lessons learned' below, as well as Interviewer Recruitment, Selection, and Training for additional discussion of interviewer recruitment considerations.

2.2     Develop an introduction appropriate for the interviewer to read upon contact with the respondent.

2.3.1    The introduction is especially important, and may differ depending on cultural norms. The way the opening unfolds between the interviewer and respondent may have significant implications for both survey nonresponse and data quality [zotpressInText item="{2265844:WAYCJPVI}"]. The context of the interview can dictate identification procedures and pace of interview.

2.3.2    Establishing and maintaining rapport is especially important in achieving a telephone survey. Particular care should be taken in the translation stage to ensure an interviewer script that does not violate cultural norms involving politeness and linguistic encoding of status and social distance [zotpressInText item="{2265844:TXPJ33IM}"].

2.3.3    The introduction can be particularly critical in achieving cooperation in some countries. Previous respondent exposure to the telephone as a survey mode can differ across countries, and there can be discomfort in sharing personal information over the phone [zotpressInText item="{2265844:SC8NE559}"].

2.3.4    In countries where there are linguistic differences depending on actors’ social status, translations must also recognize that interviewers and respondents are strangers, and cannot rely on visual cues to establish social distance and appropriate linguistic level, necessitating the opportunity for some social interaction at the beginning of the survey to establish such social distance.

Lessons learned

2.1     Gender norms of the study country can have a significant impact on response rates in CATI surveys.

2.1.1    In France, researchers have found that female interviewers generally have higher refusal rates in telephone surveys [zotpressInText item="{2265844:728H47UB}"].

2.1.2    In Nepal, a highly gendered society, women generally prefer to speak to other women and men to men, even over the telephone. However, in a CATI survey using Nepali-based interviewers contacting (mostly male) Nepali migrant workers in Persian Gulf countries, researchers obtained high response rates using predominantly female interviewers, because of the cultural perception that a woman would not call a man unless it was an important matter [zotpressInText item="{2265844:I3LPNV6L}"].

2.1.3    There is also anecdotal evidence that male respondents in highly gendered countries in the Middle East are more likely to participate in a telephone survey when contacted by a female interviewer.

2.2     Immediate identification by name is standard telephone practice in the United States, but is uncommon in China [zotpressInText item="{2265844:TXPJ33IM}"].

2.3     Acceptable pace of the interview introduction can vary across even otherwise similar cultural contexts. For example, an examination of reactions to phone calls in Hong Kong and Beijing found that Beijing residents were more resistant to a fast-paced, business-like telephone conversation when compared to those from Hong Kong [zotpressInText item="{2265844:A9Z9YQRM}"]. Similarly, a comparison of Greeks and Germans showed that Greeks prefer to have some social interaction before reaching the main point of a telephone conversation, while Germans prefer to discuss the main point immediately [zotpressInText item="{2265844:HVZIY9RL}"].

2.4     Acquiescence bias differs across cultures and can be particularly problematic in a telephone survey, where already-difficult issues can be exaggerated. For example, in many Asian cultures, people tend to avoid 'no' answers to yes/no questions, particularly when there is an asymmetrical relationship between speakers as in a survey interview [zotpressInText item="{2265844:TXPJ33IM}"].

2.5     Introductory scripts can differ dramatically across cultures. For example, in Chinese, the use of expressions like “please” and “thank you” is not common in daily conversation, and thus implies a large social distance between speakers. The mandated repetitive use of such words in a survey among Chinese speakers would be detrimental, particularly in a telephone survey where rapport is especially important, in sharp contrast to a survey in American English, where such phrases are acceptable and expected [zotpressInText item="{2265844:A9Z9YQRM}" etal="yes"].

⇡ Back to top

3. Decide whether a subset of survey questions would best be collected in a self-administered section of the interview. Rationale Interviewer-administered telephone interviewing is subject to social desirability biases similar to those in face-to-face interviewing. Interactive Voice Recognition (IVR) is a telephone mode where the computer plays recordings of the questions over the telephone to respondents, who then respond by using the keypad of the telephone or speaking their answers out loud. IVR can be used as a self-administered mode (SAQ) to administer a portion of an interview, otherwise conducted by CATI, which is particularly sensitive in nature and where accuracy might improve without the presence of an interviewer. It can also be used exclusively as a self-administered mode, with the computer automatically telephoning the respondent and then completing the questionnaire (see Data Collection: Self-Administered Surveys for further discussion of IVR in a completely self-administered mode). Procedural steps

3.1     Design the IVR system so that it is technically well-integrated into the CATI system in use by the project, and so that switching from the CATI to the IVR system is straightforward for the interviewer.

3.2     Decide whether to program the IVR system as touchtone, voice input, or a combination of the two.

3.2.1    When deciding on the programming, consider the target population. Studies in rural India and Botswana found that respondents with less education and lower literacy do better with touchtone, and cited privacy for touchtone preference as well [zotpressInText item="{2265844:PKGIXZ8N},{2265844:I3L3PZDH}"]).

3.2.2    A study in Pakistan found that a well-designed speech interface was more effective than a touchtone system for respondents regardless of literacy level [zotpressInText item="{2265844:796M9DDZ}"].

3.3     Devote sufficient time to the development of a high-quality IVR system to maintain respondent interest and continued cooperation.

3.3.1    The IVR system must have a high quality recording, as the respondent is likely to break off the survey if quality is poor.

3.3.2    See [zotpressInText item="{2265844:HBKSVNAK}" format="%a% (%d%)"] for a guide to the development of an IVR system and the associated speech characteristics which need consideration.

Lessons learned

3.1     Consider the voice used for recording.

3.1.1    In a health helpline project in Botswana, researchers employed a well-known local actress for the IVR recording, and users reacted very positively [zotpressInText item="{2265844:CHXGGNYG},{2265844:PKGIXZ8N}"].

3.1.2    Depending on the social context, using an IVR recording of a man for male respondents and a woman for female respondents may elicit more accurate reporting, particularly of sensitive information.

3.2     [zotpressInText item="{2265844:DFMSKAYM}" format="%a% (%d%)"] developed an innovative approach to the challenge that dialectical variation and multilingualism poses to speech-driven interfaces for IVR in India, applicable to other settings as well. In their approach, people from specific villages are recorded during interactions, and their speech is semi-automatically integrated into the acoustic models for that village, thus generating the linguistic resources needed for automatic recognition of their speech.

3.3     A survey of teachers in Uganda resulted in a number of useful considerations when designing an IVR system to improve response rates and data quality [zotpressInText item="{2265844:CHXGGNYG}" etal="yes"].

3.3.1    The IVR call began with the immediate information that “This is a recorded call from Project X. You are not talking to a real person.”

3.3.2    The IVR call provided very specific instructions about whether to use the keypad or to speak.

3.3.3    Respondents were initially confused by the automation of the IVR system. Researchers had better results when using a chime to get respondents’ attention before the automated voice gave instructions.

3.3.4    Leveraging conversational and turn-taking conventions of normal conversation in the IVR system lead to more success than detailed instructions in eliciting desired user behavior.

3.3.5    An IVR system which projected a loud voice, with prompts recorded as if the speaker were using a poor cell connection, resulted in a survey that was easier for respondents to follow.

3.3.6    When producing the IVR recording, use slow speech to get slow speech—respondents will emulate the voice, and resulting data will be easier to understand.

3.3.7    The IVR recording included 3 seconds of silence before the recorded speakers says “thank you” and moves onto next question, which was reported as well-received by respondents.

⇡ Back to top

References [zotpressInTextBib style="apa" sortby="author"]

⇡ Back to top