PDF 
Janet Harkness, Brita Dorer, and Peter Ph. Mohler, 2016
 
Appendices:  A | B 

Introduction

Shared language harmonization is developing a common version (vocabulary or structure) across questionnaires for different regional varieties of a "shared" language. The guidelines in this chapter address the fact that it is important for countries or locations that share a language to take steps to avoid unnecessary differences across their questionnaires (Harkness, 2000/2008; Harkness, 2007; Harkness et al., 2008a).

Why harmonize language?

In multinational, multicultural, or multiregional surveys, which we refer to as “3MC” surveys, multiple countries or communities may field surveys in the same language. Languages such as Russian, French, German, Spanish, and Chinese, for example, are spoken as a first language by populations in a number of countries. However, the regional standard variety of a language used in one country usually differs to varying degrees in vocabulary and structure from regional standard varieties of the same language used in other countries. For example, American English, British English, and Indian English differ systematically in many ways. Often differences relate to vocabulary and pronunciation, but differences in syntax and other grammatical features of the language are also found.

As a result, translations produced in different locations may differ considerably—not only because there is usually more than one way to translate a question (see Translation: Overview) but because of regional differences in language, social reality, and culture. Thus differences in translation may reflect the given regional standard (e.g., Mexican Spanish versus Castilian Spanish), may simply reflect the fact that there is more than one way to say and to translate the same source text, may actually reflect different interpretations of what the source text intends to convey, or may stem from different social and cultural realities.

Which differences are ‘necessary’ – and should thus be kept – or ‘unnecessary’ – and should therefore be avoided – needs to be defined within each study. In general, the following rule of thumb may be useful: any differences due to (a) factual differences (e.g. referring to different political, educational, or social security systems) or (b) different language use (boot/trunk, grill/broil or storm in a teacup/tempest in a teapot) should be kept. However, where representatives of each country sharing one language agree that a common version can be found, this common version should be used: often, this is the case where the different national teams had – in their initial translations – synonyms or expressions that may equally be used in several countries using one language.

A further complicating factor is that the written regional standard variety of a language may differ systematically and markedly from the spoken form of that language the same community uses. Spoken Swiss German, for example, differs notably from region to region. There is no standard written Swiss German. What is normally used in survey instruments is written Standard-German with some vocabulary and grammatical adaptations to get closer to a kind of “least common denominator oral Swiss-German” so that oral adaptations are less complicated for the interviewer. The interviewer has then to convert written Standard-German to oral Swiss-German, and additionally to the regional needs.

When there are shared languages across one or more countries, each country sharing a language with another is asked to produce and discuss its own initial translation (that is, to carry out the TRA steps of the TRAPD model at the national level) and then consult with other countries fielding in this language. Consultation may provide a fresh perspective on questions a given country may have ‘struggled’ with. In addition, it provides the opportunity for country A to benefit from a neater or better translation made by country B but also suitable for country A. Most importantly, unnecessary and potentially harmful differences in wording can be avoided. Comparing versions may sometimes lead both country A and country B to modify what they have and arrive at a better (perhaps) common version (European Social Survey, 2014). 

⇡ Back to top

Guidelines

1. Harmonize the wording of questionnaires in one language whenever possible.

Rationale

All else being equal, it is preferable to keep the wording constant within a language across locations. If no policy of harmonization is followed, unnecessary differences may proliferate. Some of these, such as differences in translating response scales, may negatively affect measurement (Villar, 2009).

Procedural steps

1.1    Decide upon the policy and procedures to be adopted on harmonization (obligatory or optional, full or optimized, top-down or bottom-up; see Appendix A), as well as whether a full or split translation procedure will be used in the case of a bottom-up procedure.

1.2    Decide on the tools to be used; these should include a documentation component.

1.3    Inform all locations sharing a language of the harmonization policy and procedures and related requirements.

1.4    Schedule and organize any translations so that harmonization is possible given the project’s overall schedule and constraints. If working from a single translated questionnaire towards localized versions, prepare and distribute the single translation. If such a top-down approach is used, the single translated version should be produced in a  team translation  approach that includes input for the different regional varieties of the languages that are to be accommodated.

Lessons learned

1.1    The increased effort, time, and outlay to undertake harmonization may be an obstacle to implementing it.

1.2    Without advance planning, the short time often available for translation may make harmonization preparation and meetings to discuss versions difficult and makes  pretesting of alternatives unlikely.

1.3    Without clearly defined protocols and some training, the local teams asked to harmonize may have difficulty making informed decisions about harmonization. They may also not properly record their decisions and their motivations.

1.4    When new locations join an ongoing study, new harmonization needs may arise in previously harmonized versions of questions. No research could be identified on whether it is better for the older harmonization decisions to be kept and the new country to deviate or for all to change. There is "received wisdom" about changing as little as possible but this is always over-ruled when change becomes necessary. These will be case-by-case decisions, depending on the study and also on the translation issue at hand.

1.5     Content management  system and localization software can aid identification of text requiring harmonization and provide a documentation option for differences retained (see Translation: Tools).

1.5.1   Keeping the words the same across questionnaires in different locations does not automatically mean that perceived meaning and intended measurement are retained across populations. Pragmatic meaning also needs to be considered (see Braun & Harkness, 2005; Harkness, 2003Harkness, Villar, & Edwards, 2010; Schwarz, Oyserman, & Peytcheva, 2010). At the same time, there is little research available that clarifies how to keep both semantic meaning and pragmatic meaning stable across surveys in different languages. Pragmatic considerations might also stand in conflict with retaining semantic meaning. It remains to be established how "sameness" and  comparability  are best ascertained at the textual level (see Braun & Harkness, 2005; Harkness, 2003; Harkness, 2010a).

1.5.2   Localized versions based on a single common translation may have fewer differences across versions in a shared language. This does not mean that the instruments are necessarily better than those with more differences. Careful testing should be carried out to make sure that each population does understand the questions as intended (Harkness et al., 2010b; McKay et al., 1996).

1.5.3   In instances where there is a language shared across more than two countries – Russian, for example – the following procedure may be applied: Two of the affected countries in a 3MC project (e.g., Russia and Ukraine) agree on a common, de-centered ‘master’ translation. A de-centered translation is one that does not use terms that have precise linguistic equivalence, but rather phrases that are more general and do not rely on a specific linguistic context (e.g., rather than using the English-specific phrase “every cloud has a silver lining”, using instead “something good comes from any misfortune”) (Smith, Bond, & Kagitcibasi, 2006). This master version would then be used by all countries sharing this language as the 2nd initial translation in their ‘national’ TRAPD process, i.e. it should be used as one of the two translations in the review session. Also with this option, care must be taken to keep up a communication between all countries involved in order to discuss any criticisms or questions arising during the different review meetings and reconciliation efforts. There must be a thorough review meeting when using the de-centered master translation as the second translation in the TRAPD process in every country. Like in all review meetings, the participation of both people with linguistic and/or translation expertise and with survey knowledge is crucial; and it would be useful if a representative from one of the countries producing the master version could participate in the review meetings (European Social Survey, 2014).

1.5.4   A ‘lighter’ approach along the line of a ‘de-centered master translation’ is acceptable in case it is not at all possible to create such a ‘de-centered master translation’ in instances of shared language across more than two countries. The reason for this may be that schedules of the translation processes in the countries sharing one language vary so much that it is not even possible to organize any reconciliation efforts between two countries. In this exceptional case, countries should be allowed to use the final translation from another country using the same ‘shared language’ as the 2nd initial translation in the national TRAPD process even if this translation has not been agreed upon with a second country. Again, in the Russian language, for example, this would preferably be the final translation from Russia or Ukraine.

     However, some points need to be considered: (a) this option should only be applied in exceptional cases, that is, if the translation schedules are so distant from each other, that no other reconciliation methods detailed above are possible; in any case, reconciliation methods where all participating countries make a more active contribution to the final translation(s) will be more rewarding for all those participating; (b) there must be a thorough review meeting when using the final translation from another country as the second translation in the TRAPD process in every country; if possible, there should be a communication with the country producing this first translation, giving feedback and also asking questions or providing comments in cases of criticism of this translation; like in all review meetings, the participation of both people with linguistic and/or translation expertise and with survey knowledge is crucial; (c) the disadvantage of this option is that the country finalizing their translation first would normally not benefit from the opportunity of discussing their translation with experienced native speakers from other countries (European Social Survey, 2014).

1.5.5   Splitting a questionnaire between translators can save time and effort, particularly if a questionnaire is long (Martinez, Marin, & Schoua-Glusberg, 2006; Schoua-Glusberg, 1992). At least one translator from each country plus a reviewer and adjudicator (or reviewer-cum-adjudicator) is needed. The translation is divided up between translators in the alternating fashion used to deal cards in card games. The questionnaire should not, for example, be divided into a first half and a second half, nor divided by thematic module. By giving each translator material from the same sections, possible translator bias is avoided and translator input is maximized evenly across the material. Each translator translates his/her own questions (European Social Survey, 2014).

1.5.6   Care is needed to ensure that consistency is maintained across the translation, and ‘split’ questionnaires require particular care. Steps should be taken to ensure that material or terms which re-occur across the questionnaires are translated consistently. At the same time, it is important to remember that although the source language may use one and the same expression in different contexts, target languages may need to choose different terms for different contexts (e.g., the term “government) (European Social Survey, 2014).

⇡ Back to top

2. Only keep necessary differences.

Rationale

There are often several ways to formulate a survey question, an explanation, or even instructions. Teams cooperating in a harmonizing effort must try to lay aside personal preferences. Differences that are maintained across questionnaires should be considered genuinely necessary—and, preferably, demonstrated through testing to be so. It is also possible that countries decide they need different versions. However, countries should try and follow the ‘as close as possible to each other but as different as necessary’ principle. In all cases, the emphasis must be on ‘better’ versions, not on ‘word level sameness’ for the sake of ‘word level sameness’. In such cases, countries should document changes made as a result of consultation with each other as well as any differences across sharing countries which are necessary to keep in a form such as the TVFF (see Translation: Overview , Appendix A and European Social Survey, 2014).

Procedural steps

If harmonization takes place on the basis of individual translations made by each national or regional group (bottom-up approach):

2.1    Organize templates to enable easy comparison of the initial translations to be reviewed for harmonization.

2.2    Organize the harmonization meeting(s).

2.2.1   These can be face-to-face, perhaps piggy-backing on another meeting. However, webcasting, webinars, or "skyping" may be the only affordable modes of meeting.

2.2.2   Share versions prior to the meeting and produce a central document aligning them side by side; use a format that also allows each user to see the source and target questions easily (see Appendix B).

2.2.3   If possible, appoint someone to identify types of difference (or just differences) ahead of the meeting, both on the basis of any past experience and by checking the translations to be harmonized. If this person is someone who also attends the meeting, he or she might usefully introduce each question, summarizing points noticed.

2.2.4   Appoint a meeting chair and determine how group/location decisions will be made, ensuring fair representation of each group/location.

2.2.5   At a reconciliation meeting, translators and the translation reviewer(s) go through the questionnaire question by question discussing versions and agreeing on a common version. The adjudicator(s) may attend the review process or already be involved as reviewer(s). Alternatively, the reviewed version moves on to adjudication.

If common wording in the form of a single translated version is the starting point (top-down approach):

2.3    Organize templates to enable easy comparison of the suggested localizations.

2.4    Have each local team propose modifications it considers necessary to the common version.

2.5    Organize the reconciliation meeting(s).

2.5.1   These can be face-to-face if possible, perhaps piggy-backing on another meeting. However, webcasting, webinars, or "skyping" may be the only affordable modes of meeting.

2.5.2   Define the goals of this meeting (e.g., to review suggested changes, to try to find new shared alternatives, to share questions about the single translation).

2.5.3   Share localization suggestions prior to the meeting and produce a central document aligning them side by side; use a format that also allows the users to see the source questions easily.

2.5.4   If possible, appoint someone to identify the types of localization proposed ahead of the meeting, both on the basis of any past experience and by checking the localizations proposed. If this person is someone who also attends the meeting, he or she might usefully introduce each question, summarizing the suggestions made and questions raised.

2.5.5   Appoint a meeting chair and determine how decisions will be made, ensuring a fair representation of each group/location.

2.5.6   At a reconciliation meeting, translators and the translation reviewer(s) go through the questionnaire question by question discussing versions and agreeing on a common version. The adjudicator(s) may attend the review process or already be involved as a reviewer. Alternatively, the reviewed version moves on to adjudication.

Lessons learned

2.1    Personal language perception and usage can be mistaken for generic language usage. It would be mistaken to assume that because one or more speakers make a distinction that these are then distinctions made by all speakers of a given speech community.

2.2    It may not serve the study's purpose to make decisions on the principle of a "majority" vote. The aim is ultimately to allow necessary difference in any given version.

2.3    Harmonization is not limited to the choice of words or phrases; it can include decisions, for example, about how sentences are structured and response scales organized.

2.4    Sometimes harmonization takes the form of adding a term or an example to whatever is common with other shared language versions. Thus if a question about tobacco use does not cover a special form that is only relevant (but important) for one population, mention of it could be added for that population alongside the other forms of tobacco use mentioned in the other versions of the question. This strategy of keeping what is common but adding a local requirement is frequently found in adaptations (see Adaptation).

2.5    If the top-down localization model is used, teams may spend more time discussing the single translation than any of their localizations. This has advantages and disadvantages. One benefit in discussing the available translation is that the group may have new ideas about a possible common version or a common version with occasional "add-ons" as just described. One possible disadvantage is that consideration of the range of localized suggestions is reduced, with each team member ultimately focusing more on resolving what to choose for his or her own version.

⇡ Back to top

3. Schedule harmonization at an appropriate time.

Rationale

Harmonization efforts can result in changes in one or all national questionnaires. The harmonization decisions need to be made when each questionnaire version (or the single translation) is at an advanced stage of development. Although desirable, iterative rounds of pretesting are not likely to be feasible. Thus if a team translation procedure (documented translation review,  adjudication , and pretesting) is followed, harmonization should precede pretesting and thus final adjudication (see Appendix A and Translation: Overview , and in particular, Figure 2. European Social Survey Translation Process). Pretesting can be used to check harmonization decisions. It may also indicate that further changes are required in one or more versions.

Procedural steps

3.1    Identify the time at which a well-developed version of each questionnaire to be harmonized will be pretested  (or the single common version is well advanced) and arrange for harmonization before that time. In cases where expert assessment, such as verification or survey quality predictor software (SQP), are part of the translation processes, shared language harmonization might intervene at different point in time: before submitting national translations to verification, and after receiving back the expert checks from verification and survey quality predictor software (SQP) coding (see also “Figure 2. European Social Survey Translation Process” in Translation: Overview ). Before harmonization occurs, each country should complete the initial translation process as outlined in Translation: Overview  and a preliminary review and revision of these translations.

3.2    Countries then exchange translations with the other country or countries sharing a given language; the arrangements between these countries will be decided on by the countries themselves; the procedure chosen and the different steps should be documented accordingly.

3.3    Countries consult together on the final version for each country. They “harmonize” and remove differences across countries as appropriate and comment on any difference retained, and document every decision accordingly.

3.4    Schedule in-person reconciliation meetings whenever possible. Representatives from all countries involved meet in person in order to discuss all newly translated or adapted questions. At least one person per country must participate in this meeting – ideally this would be the person acting as reviewer/adjudicator from each country; it is recommended that at least one translation expert participates in the meeting (e.g. from the host country of the meeting so that there are no further travel expenses). Of course, additional people can participate, such as translators or technical experts. The outcomes of these reconciliation meetings must also be documented (e.g., in the appropriate column in the TVFF called ‘Shared Languages Discussion’) (see Figure 3 in Translation: Overview , Appendix A).

3.5    If in-person meetings are not possible, plan to exchange translated questionnaires via email and/or telephone. In this case, it is important that the countries involved have a thorough discussion on all critical issues or discussions and also document the outcomes of their deliberations. Similarly, discussions can be held in the form of a web- or telephone-based conference, which may require higher financial and organizational efforts.

3.6    Demographic questions which are country-specific or questions that require national consultation processes between the project leaders and the national teams do not need commentary on differences between national versions (e.g. country specific education variables, alcoholic drinks and quantities common in each country).

Lessons learned

3.1    If countries are fielding at different times, a group fielding much later than others may have trouble carrying out (or funding) harmonization preparations in time for groups fielding earlier. The sooner harmonization is organized and scheduled, the greater the chances are of successful schedule coordination between countries or locations.

3.1.1   In practice, recommending harmonization rather than requiring it may not be sufficient to motivate countries or locations to engage in the extra effort. The European Social Survey (ESS) has been aiming for optimized harmonization and recommending it to participating countries. Since harmonization is not a requirement in the ESS, countries are left with considerable freedom as to whether they harmonize or not. Historically, the countries' various time schedules also did not always easily accommodate a harmonization step. Harmonization has further been complicated by countries with shared languages joining the project at different times (Andreenkova, 2008). But in the 7th round of the ESS, there were harmonization steps for almost all shared languages. Even ex-post comparison of other versions of the same language – that is, no proper ‘harmonization’ effort, but a mere comparison – can be rewarding in terms of enhanced harmonization and similarity of shared language versions.

3.1.2   Without harmonization, the differences that may arise across different regional versions of questionnaires in a shared language can be considerable and may often be unnecessary (Andreenkova, 2008Harkness, 2000/2008).

3.2    The differences in regional varieties of languages, at least in terms of what needs to be captured in questionnaires, may sometimes also be overestimated.

3.3    While recognizing and emphasizing that same wording does not mean same meaning or  comparable  measurement, differences across questionnaires may introduce unnecessary and potentially serious measurement error . It is, therefore, important to include harmonization procedures in the study design.

⇡ Back to top

4. Determine and stipulate documentation requirements and tools for the process and outcomes.

Rationale

Those undertaking documentation should have a clear understanding of what is required and should be provided with aids that enable them to maintain documentation without undue burden. Documentation templates play an essential role while deliberating on harmonization as described above. Documentation also provides the evidence examined in  quality monitoring and assurance steps, for any coordination of harmonization efforts that may exist in a project and provides secondary analysts and other users of data with information about differences across instruments.

Procedural steps

4.1    Determine documentation needs and create stipulations to be followed by those involved in harmonization in order to achieve these needs.

4.2    Develop templates for the language harmonization process and the harmonization outcomes (see Appendix B).

4.3    Distribute templates and specifications to all those involved well in advance and ensure they are familiar with their purpose and how to use them.

4.4    Provide examples of what is sufficient documentation and what is not.

4.5    Differences should be documented (e.g., in the TVFF). (See Translation: Overview , Appendix A)

Lessons learned

4.1    Good and accessible documentation is essential to shared language harmonization efforts. It enables teams to compare options more easily while making decisions and also to record clearly the decisions taken. Users of data also benefit from documentation on differences across instruments.

⇡ Back to top

5. Undertake shared language harmonization within a quality assurance and control framework as that relates to translation quality.

Rationale

Language harmonization is undertaken to reduce unnecessary variance across versions of a questionnaire in one language that may negatively affect measurement in any of a variety of ways. The purpose of harmonization is, thus, to enhance measurement  quality .

Procedural steps

5.1    Plan and undertake harmonization in controlled procedures as described above.

5.2    Plan to follow harmonization with a pretesting phase.

5.3    Develop the relevant materials needed as described above.

5.4    Identify and engage suitable people to be involved in harmonization as described above.

5.5    Brief team members on the materials, purpose and strategies used in harmonization.

5.6    Complete the main harmonization process.

5.7    Pretest and then modify instruments as relevant.

5.8    Share findings in a well-documented and timely fashion with any coordinating center , as relevant.

Lessons learned

5.1    The more rigorous the translation procedures and the various sub-activities such as harmonization and pretesting  become, the more important scheduling, budgeting, and briefing are.

5.2    Long-term, the benefits of having and being able to share well-developed, well-translated and tested instruments can be very considerable.

5.3    It may be more effective to require locations to engage in harmonization than to recommend that they do.

⇡ Back to top