PDF 
Peter Mohler, Brita Dorer, Julie de Jong, and Mengyao Hu​, 2016
(2010 Version: Janet Harkness)
 

Introduction

The term adaptation, as used in this chapter, refers to the deliberate modification of a question or questionnaire to create a new question or questionnaire. It is also referred as asking different questions (ADQ) in Questionnaire Design.

Adaptation needs may be considered at different stages in different multinational, multicultural, or multiregional surveys, which we refer to as “3MC” surveys, and it is likely that some adaptation needs only become apparent during translation or during pretesting of a translated questionnaire. It is therefore not possible, in terms of the survey lifecycle, to identify a single unique stage as the stage at which adaptation needs might be recognized or addressed. This may differ, for instance, for different projects or surveys.

However, the most common procedure is to consider and address adaptation needs together with the translation process – therefore, the terms are often used in combination as in ‘Translation and Adaptation’. This is also mirrored in the survey lifecycle.

Overall, there are different ways to integrate adaptation in the survey lifecycle, that is, in the production of survey instruments in the target languages. Examples are creating separate teams and experts for adaptation and translation; another possibility is to deal with adaptation at the same time as translation in a one-team framework: in this case, translation experts would also need to be experienced or knowledgeable in adaptation needs. In this chapter, we will focus on the former approach, where translation and adaptation are conducted through separate teams. It is highly recommended that the two teams collaborate and work closely together for questionnaire design. Translation team will be responsible for the same questions asked in both cultures (see asking same question and translating (ASQT) in Questionnaire Design), and adaptation team will mainly be responsible for asking different questions or asking questions in different ways to adapt new cultural needs (see asking different questions (ADQ) in Questionnaire Design).

 

Why adapt questions?

Adaptation may be made to the content, format, response scales, or visual presentation of any part of a question, questionnaire, or instrument. The purpose of adaptation is to better fit the needs of a new population, location, language, or mode, or any combination of these (Harkness, 2010b; Harkness, 2008b; Harkness, Villar, & Edwards, 2010; see also Instrument Technical Design and Translation: Shared Language Harmonization).

When developing new studies, researchers frequently modify questions that have been used in other studies and then use these modified versions. The motivations for such modifications may or may not be documented. Sometimes changes are related to adapting to meet new needs. However, sometimes changes are made simply because those making the changes consider them to result in a generally "better" version or instrument. This chapter focuses only on changes made to meet new needs as described above (adaptations).

In one language (monolingual) contexts, questions and questionnaires may be deliberately adapted for a variety of reasons. In longitudinal surveys, for example, wording might be updated to stay abreast with current usage; "wireless" could be replaced by "radio" (Smith, 2005), for example. Wording might also be changed to better reflect current social realities, such as adding social media as means of communication or the Internet as an information source in media usage questions (Smith, 2005). Changes might also be made to accommodate a new population; modifying vocabulary, presentation, and instructions to suit a child population rather than an adult one, for example.

In 3MC projects, adaptation is often related to the need to translate a questionnaire into another language in order to study new populations. In the following chapters, the terms "source language" and "target language" are used to distinguish between the language translated out of (the source language) and the language translated into (the target language).

In some projects, adaptations may already be anticipated in the source questionnaire, that is, the questionnaire on which other language versions are based and derived. Thus a source questionnaire question about pride in one's nationality, "How proud are you to be [nationality]?" anticipates a country-specific adaptation inside the square brackets, with each participating country entering the relevant nationality (e.g., Spanish, German, Chinese) in the slot indicated by the square brackets in their version of the questionnaire.

Socio-demographic questions often require adaptations to be made in different locations and languages (see Translation: Shared Language Harmonization).

The need to make some adaptations might only become apparent in the course of translating the source questionnaire into a given target language. This could be because features of the target language itself make adaptation necessary or because a translated version of the source question, although possible, would not achieve the required measurement goals.

Response scales provide examples of adaptations occasioned by features of the target language. Agreement scale response categories developed in English frequently have a middle category "neither agree nor disagree." In languages such as Hebrew and Swahili, this phrase cannot properly be translated by simply translating the words. The closest semantic option available to translate "disagree" in Hebrew, for example, corresponds to "no agree." In addition, the words "neither" and "nor" are the same as the target language element corresponding to "no." Thus "neither agree nor disagree," if translated element for element, would produce something like "no agree, no no agree" this makes little sense in Hebrew (Harkness, 2003). The Hebrew phrase thus used in ISSP studies for the category "neither agree nor disagree" corresponds to "in the middle." On a study on adapting or translating response scales, see Villar (2009)

Frequently adaptations are motivated less by features of the target language than by the need to fit social, cultural, or other needs of the new linguistic group to be studied. Examples of adaptation not directly related to linguistic considerations abound. A recent international project proposed fielding the question, "Can you lift a two-liter bottle of water or soda…," in multiple countries. The source question itself was not developed cross-culturally (see Questionnaire Design). Several locations (countries) noted that (a) the normal size of bottle in their context was 1.5 liter bottles, not 2 liter bottles, (b) that they were unsure whether the bottle referred to was intended to be glass or plastic (which would affect the lifting task), (c) that "soda" was not a salient generic concept in their locations, and (d) that the formulation in English which indicates that the bottle is not empty ("bottle of water or soda" ) needed to become "a full bottle of water" or "a bottle full of water" in their translations. However, there was some concern that these much more explicit renderings of "bottle of water" might alter respondent perceptions of the lifting task.

Usually, as reflected also in these examples, the needs of translation and those of adaptation are entangled. Thus, the appropriate or viable translation for a given context may also be a translation that includes adaptation of content, format, or some other questionnaire feature. For example, translations of an American question referring to being able to walk "several blocks" also needed to adapt the phrase "several blocks" for Great Britain and provide the distance for European locations in terms of yards or meters (Harkness, 2008b). It is not always possible, therefore, to distinguish neatly between translation needs and the need to adapt other features of the question or questionnaire. Therefore, in several studies, both terms can be used in combination as “Translation and Adaptation.” It is also thus essential that translation team and adaptation team work closely together for questionnaire development.

There can be a delicate balance between when adaption is needed and when the changes required are so great that they indicate that the original question should be discarded because even the best adaption cannot result in a question that could be considered equivalent. It is also important to note that the scale of the adaptation work involved is likely to be very different when there are only two languages involved, and useful back and forth can optimize an item and its best translation or adaption, versus when dozens of languages are involved.

⇡ Back to top

Common forms of adaptation

The categories identified below are based on distinctions found in Harkness, (2010b), Harkness (2008b), Behr & Shishido (forthcoming) and Harkness, Villar, & Edwards (2010).

System-driven adaptation

Units of measurement differ across countries and may require adaptation (e.g., Imperial [yards, pounds] versus Metric [meters, kilos]; Fahrenheit versus Celsius). Direct conversions may be exact and completely equivalent but can produce an odd sounding question. For example, asking about 100 yards would mean asking about 91.4 meters, which while precisely equivalent is an odd metric distance. Adaptations will need to be considered for any mention in instruments of length, area, dry volume, liquid capacity, weight or mass, and also currency.

Adaptation may also be needed to account for structural differences in government, government policies, and laws. For example, a question involving the head of state would ask about the prime minister in the United Kingdom and the President in the United States. Perhaps less straightforward is how to how to adapt questions about a law or policy that may exist in some contexts but not others, such as a minimum wage.

Questions involving currency can also raise adaptation challenges. There are different exchange rates (e.g. official vs. informal) and straight conversion is often not meaningful because of differences in purchasing power. Some economic questions try to achieve equivalence by using some standard such as the median wage as the reference point in each country.

Adaptation to improve or guide comprehension

In preparing to use the question, "Can you run 100 yards?" in Vietnam, local researchers worried that the distance would not be clear to Vietnamese respondents and adapted it to, "Can you run 100 yards or the distance of three light poles?" to help respondents envision the distance intended (Hanh et al., 2005; Harkness, 2008b). In this particular example, the distance mentioned in the source version is retained but also supplemented by a localized indication of the intended distance.

Adaptation to improve conceptual coverage

Sometimes question components are added for a given location to better tap the intended dimension or construct. For example, the symptoms shown by patients with a given disease (as well as the treatments, the attributed causes, and the places to get help) can differ across cultures. Including mention of local symptoms, as relevant, can improve the accuracy of information collected at the local level and for the combined data set.

Adaptation related to cultural discourse norms

Speech communities differ in the way in which they frame and carry out communication. Depending on the culture and language involved, indicators of politeness or deference may be required in the interview script or the self-completion questionnaire (polite imperatives, acknowledgment of relative status of interviewer and respondent, apologies for asking a question, etc.).

In some contexts, adaptations are made without the scientific community currently acknowledging these as part of questionnaire adaptation needs. For example, Korean is a language with a systematic honorifics system reflecting social status, age, interpersonal relationships between participants in a discourse, and, indeed, much more (Strauss & Eun, 2005). In interviewer-assisted applications, such discourse and etiquette requirements affect what interviewers say, depending on whom they are interviewing. In some diglossic linguistic contexts, the gap between written forms of a language and spoken forms can be quite large. This can mean that interviewers have a written script that conforms to the norms of the written standard of the language but are required, in "speaking the script," to conform to spoken norms of the language (see Harkness et al. (2008b) and Paulston & Tucker (2003)).

Adaptation and cultural sensibilities

Cultural sensibilities with regard to a wide range of topics differ from culture to culture. Such differences motivated adaptations for Japan in the Picture Completion section of the Wechsler Intelligence Scale for Children (WISC-III). Instead of a depiction of a person with a body part missing, the Japanese version used an inanimate object with a part of that object missing (Ueno & Nakatani, 2003).

Adapting design components or characteristics

Changes to the technical design of an instrument can be motivated by many factors. The direction languages are read or written in, a population's familiarity with certain visual representations (thermometers, ladders, scales using faces (Kunin, 1955)), and a wide range of culturally anchored conventions related to visual presentation, including color symbolism, representational preferences, and conventions of emphasis, may call for adaptation of components of the source questionnaire (see also Instrument Technical Design).

Adaptation related to lexicon and grammar

The lexicon (a language's vocabulary) and grammar of a language may also make changes in design necessary. An example already discussed is the response category "neither agree nor disagree" which has been rendered in Hebrew International Social Survey Programme questionnaires as "in the middle."

Adaptation to maintain or to reduce level of difficulty

Educational and cognitive ability tests are biased if it is easier for one population to answer correctly or perform a task required than it is for another population of equal ability on that item (van de Vijver & Leung, 1997). A wide range of question types is thus sometimes adapted to maintain the same level of difficulty across different populations. Research in educational and psychological testing discusses such issues (see, for example, Georgas, Weiss, van de Vijver, & Saklofske, 2003 and Hambleton, Merenda, & Spielberger, 2005).

In studies of opinions, behaviors and attitudes, the goal is generally more one of keeping respondent burden low. Adjustments may thus sometimes be made to simplify the vocabulary used in a translation for populations with expected low levels of education or to increase instructions and explanations for those unfamiliar with the procedures of survey research. Response scale presentation is sometimes supplemented for populations unfamiliar with the notions of rating, for example, or for those unfamiliar with conceptualizing the response scale concepts in relation to entities asked about (Bullinger, Kirchberger, & Ware, 1995; Struwig & Roberts, 2006).

⇡ Back to top

Guidelines

Goal: To make a survey instrument better fit the needs of a new population, location, language, or mode.

1. Determine the policy, people, and procedures for adaptation for the project.

Rationale

Adaptation needs will arise in most comparative projects and should therefore be sufficiently prepared for. Any quality assurance and quality monitoring framework must therefore include a plan for how to deal with adaptation. This plan should propose procedures to identify and address adaptation needs for each location and how to make decisions about documentation. It should also determine how any effort to coordinate adaptations or their documentation is to be organized (see Harkness (2010b)).

Procedural steps

1.1    Plan coordination of adaptation development and the tools to be used to develop and document the process and outputs.

1.2    Identify a suitable team with the necessary skills to work on adaptation problems (see Guideline 2 below).

1.3    Decide on a procedure for approving adaptation by the persons assigned to decide and approve adaptations. In projects aiming to ask the same questions (ASQ) of each population, substantive adaptations should only be made if they are required to ensure comparable measurement or avoid some other important negative consequence.

1.4    Decide on a strategy to ensure that participating groups (locations, countries, etc.) are informed about adaptations being proposed by other members and can contribute their own proposals or reactions.

Lessons learned

1.1    By anticipating certain adaptations in an ASQ source and translate (ASQT) model, the translated versions are likely to be more consistent with the measurement intended in the source questionnaire. However, it is very likely that some adaptation needs will not be recognized until translated versions are available.

⇡ Back to top

2. Recruit a team to work on adaptations.

Rationale

Adaptations are made to address modifications necessary to be able to interview multiple populations. The spread of skills and range of cultural experience required cannot be provided by one person.

The team should bring together knowledge about and an understanding of (1) adaptation needs in general, (2) the types of adaptation, (3) the strategies commonly used to adapt, (4) measurement comparability needs, (5) language proficiency in whatever languages are involved, and (6) relevant cultural information.

The team should work in close cooperation with the translation team. Depending on the project and the team composition, it may be that the same people carry out translation and adaptation tasks. In any case it is important to have people with adaptation knowledge and skills in the overall team for ‘transforming’ a source survey instrument / source questionnaire for use by target populations.

Procedural steps

2.1    Identify a small group of people who can, as a team, provide the skills and competencies needed for the six points mentioned above.

2.2    Identify at least two people for each given location or instrument to work as an adaptation team. This team should supplement the translation team to carry out adaptations as needed. These additional team members contribute only to the specific instrument they can provide input on. They provide the specific cultural awareness and language competence needed for a given location and language. However, issues identified for one location and population may prove relevant for others too.

2.3    Brief all team members on the goals of the adaptation steps, the procedures, any tools to be used, and the documentation required.

Lessons learned

2.1    Briefing and providing examples of what is desired and not desired is important. Members of such teams might be working consciously on adaptation for the first time. In addition, some team members with experience with adaptation might have learned practices the current team does not want to endorse. Providing examples for discussion during briefing and training reduces the likelihood of members making incorrect assumptions about what is required and how to proceed.

⇡ Back to top

3.   Review, as relevant, the source questionnaire for adaptation needs.

Rationale

Identifying and resolving adaptation needs in the source questionnaire may result in a better source questionnaire (that is, one that is easier to work with as a source questionnaire). By identifying and resolving elements to consider for adaptation in the source document, comparability across different questionnaire versions can also be enhanced.

Procedural steps

3.1    Assign the work to a person or persons familiar with the common forms of adaptation in surveys, knowledgeable about the questionnaire as well as the measurement goals of each question, and with a good understanding of the cultural and social realities of both source and target populations. Provide a format for indicating potential adaptation elements.

3.2    Keep a record of all elements identified and the rationale for these.

3.3    Provide examples of what is required in terms of adaptation in the record.

3.4    Check the suggestions made with a range of locations participating in the project. The members engaged for local consultation would be useful contacts for this.

3.5    Adjust the adaptation proposals for the source questionnaire as seems appropriate.

Lessons learned

3.1    It may not be easy to find people with experience in adaptation procedures. People with extensive experience in drafting questionnaires for multicultural projects and translators may be good first choices; each can provide different insights based on their different knowledge and experience.

3.2    The ability to look at a questionnaire with an awareness of other cultures' needs can be trained but it needs to be based on some background of cross-cultural experiences and awareness. Translators develop the ability to think across and between cultures in the course of their training. Their insights and their explication of motivations for suggested changes could help others in the team learn what is needed. At the same time, translators cannot be expected to understand all the measurement factors to be considered in question adaptation. In addition, translators are not necessarily in touch with the on-the-street reality of interviewing and the everyday language of the target population. This is why a team providing a spread of expertise is recommended.

4.   4. Review the translated questionnaire or instrument for adaptation needs.

Rationale

A review with respect to adaptation can be incorporated into the translation phases. Some adaptation proposals are likely to result from the translation process in any case. However, some adaptation needs that are unrelated to translation may not be apparent to the translation team. It is, therefore, important to check for other adaptation needs once the translation is completed. In addition, the adaptation team may have access to knowledge about adaptation undertaken in other languages involved in a multi-lingual project that an individual translation team does not.

Procedural steps

4.1    The adaptation team should collaborate closely with the translation team. The persons chosen should, together, provide language and translation skills and a good understanding of the cultural contexts of target versions. The team producing the local target version of the questionnaire could help them as necessary to be aware of source version implications and cultural assumptions inherent in it. These people need not be extremely proficient in the language of the source questionnaire. If suitable local people are readily available, using two different people from those advising on adaptation for the source questionnaire could minimize repetition and transfer of topics from the source questionnaire review to the current review.

4.2    Provide a format for indicating potential adaptation elements, along with examples.

4.3    Keep a record of all elements identified and the rationale for these.

4.4    Check the suggestions made by the adaptation team with groups formed from other locations and adjust the adaptation proposals accordingly. This step might best be undertaken as a late step in deciding adaptations for the entire project.

Lessons learned

4.1    Given the meager literature on the rationale and procedures of adaptation in surveys (for an example, see Behr & Shishido (forthcoming)), adaptation teams may end up making decisions based on common sense and best guesses. Pretesting adaptation decisions before implementation is thus essential.

⇡ Back to top

5. Document adaptations and the rationale for making them

Rationale

Documentation of adaptations is important for version control across locations and adaptations in one round of a survey. It also makes it possible to check content and presentation through any longitudinal iterations of a survey or a question. Such documentation can also ultimately inform the development of a more refined understanding of adaptation practices.

Lessons learned

5.1    Ensure the documentation of changes and their rationale is made publicly available. At the moment it is not easy to find literature on adaptation that presents procedures and motivations in detail (for an example, see Behr & Shishido (forthcoming)). The documentation taken by teams as proposed above will form an important basis for surveys in the future and help advance this area of methodology.

5.2    The motivation for adaptations may also not be evident to those not involved in the adaptation process. Secondary analysts, for example, would benefit from a record of the rationale behind adaptations.

6. Test adaptations made with the target population.

Rationale

Adaptation results in new questions. New questions should be tested with people representative of the target population.

Procedural steps

6.1    Pretest adapted instruments to find out whether the questions are understood as intended and can be answered without undue burden.

6.2    Include quantitative assessment (see Pretesting).

Lessons learned

6.1    It is important to streamline development of adapted instruments as much as possible in order to have enough time and resources to undertake the various steps and testing of these steps. Adaptation needs should be considered at each stage of development; however, in several surveys, they may be mainly dealt with during the translation stage. Development and pretesting of the source questionnaire should keep adaptation needs in mind. The question about being able to lift a 2-liter bottle of water or soda, for example, could have been evaluated in terms of the availability of bottled beverages, the saliency of the size of the bottles, and the material of which they might be made. So the need to make adaptations from a final source instrument can be reduced during the questionnaire design phase, as translation alone cannot remedy such matters.

6.2    If adaptation is left until the last moment, there may be no more time or resources to pretest.

6.3    If sharing findings and conclusions about adaptation across locations involved in a project is not organized in an efficient and timely fashion, individual locations are not able to benefit from solutions or problems found in other locations.

6.4    Extensive evaluations of various kinds are needed to establish whether adapted or translated questions result in comparable measurement. The health-related quality of life literature on translated instruments, even on just the SF-36 Health Survey, is revealing in this respect. See, for example, Bolton & Tang (2002) and references cited there.

⇡ Back to top