Appendix E (Changes in Existing Translations)

Survey instruments often contain items that were used in previous rounds or in other studies. In this appendix, we draw directly from the ESS Round 7 Translation Guidelines, which provide suggestions to translators on managing translations across multiple waves of a survey [zotpressInText item="{2265844:883WBJP7}"].

The policy adapted by researchers is generally to maintain continuity, which is essential for measuring differences across countries and/or change within countries and change over time. However, it is also critical that the translations used in each country are equivalent to the source language and indeed measure what is intended by the researcher.

Both the ESS and the International Social Survey Programme (ISSP) apply the following rule: where translations used in a target country are not equivalent to the source language, and indeed don’t measure what is intended by the researchers, and changes are therefore considered to be absolutely necessary, changes in existing translations should be made. This is mainly the case where (a) clear mistakes have been made in previous survey rounds or waves or, for instance, (b) the language use in a target language has changed in the meantime and a previously used translation would not be used or even correctly understood anymore.

Due to the unknown impact of even minor changes to the questionnaire, it may be unwise to make desirable but inessential changes (even if they are thought to improve equivalence with the source questionnaire) in the middle of the time series. Translators are explicitly advised against amending a translation simply to improve it with small changes or enhance consistency across the questionnaire post hoc. Only real mistakes, that is, justified concerns, should be corrected and subsequently documented in a documentation template such as the TVFF in Appendix A. The ESS has started compiling changes applied to existing translations in a specific report in Round 5 (cf. [zotpressInText item="{2265844:GXPLUJSH}" format="%a% (%d%)"]); such reports will be made available for all subsequent rounds of the ESS and may be consulted by data users.

Spelling mistakes and typos can be adjusted at any time, but should be documented appropriately. Below are recommendations for consideration of changes in existing questionnaire translations.

A country should be able to make a case for any change, noted by the translator, that they want to implement. If possible, countries that wish to change existing translations should provide some evidence for the benefit of this change (in the case of obvious translation mistakes, however, no evidence would be required). This evidence could take the form of a result from a pre-test or some other assessment. The evidence provided will facilitate the decision-making process for the project coordinators on the acceptability of a change. By discussing any desired changes with the project coordinators, tinkering with the translation can be avoided.

As [zotpressInText item="{2265844:HPLGQN2K}" format="%a%"] notes: “Sometimes a researcher realizes that a question has not been worded perfectly but it is still useful, such as when it is decided that it is better to maintain old question wording so that time trends can be analyzed, even if some alternations could lead to a better question” (2005, p. 112).

However, words and their use may change over time. This change may, for instance, be triggered by altered social conditions or the introduction of politically correct language. Example from the German General Social Survey (ALLBUS)—“Gastarbeiter” [zotpressInText item="{2265844:H9W3KEYW}"]: a word that in the past was used for 'immigrants' can no longer be used since the immigrant flow has changed in its composition, qualifications and countries of origins; in addition, current language use also plays a role here. Similarly one can observe dramatic language use in naming of ethnic groups in many countries over time (e.g., [zotpressInText item="{2265844:23D6BBNR}" format="%a% (%d%)"]). Awareness of needed change over time should lead to regularly reviewing any core translations and adapting them where necessary.

A translation should be changed if a real deviation between the source and the target text can be corrected. This may, for instance, be the case when:

(a)   adding an interviewer instruction that was mistakenly left out of the translation previously;

(b)   adding a word or phrase that was left out of the translation previously (e.g. source question asked about full-time work but translated version left out reference to ‘full-time’);

(c)   deleting a word or phrase that had previously been included in the translated questionnaire but was not present in the source questionnaire (example: adding examples of different sources of income to the household income question when no such examples were in the source questionnaire);

(d)   changing a word that is no longer in common usage in a country, e.g., because it is no longer politically correct; and

(e)   changing a word or phrase in the target language so that its translation more closely matches the intended meaning in the source language.

In some cases, the decision will depend on the evidence provided by a country. An example may be changing a word that is thought to cause serious comprehension problems where countries will need to demonstrate that the wording has caused serious problems.

Changes for the sole purpose of improving the translation in the absence of a mistake, even if it does not change the meaning in the target language, should be considered closely before implementation. It is rather advised against making such changes.

Examples of changes to the target questionnaire that should not take place between rounds include:

(a)   making small amendments to tidy up the question wording e.g. using a more parsimonious phrase rather than a lengthy description;

(b)   adding more words or phrases in order to match the source questionnaire more precisely;

(c)   trying to harmonize response scales across all parts of the core questionnaire e.g. ensure agree/disagree scales are always translated consistently—if the translations had not been erroneous before (here the time series is more important than consistency within the questionnaire); and

(d)   trying to harmonize translations with other countries sharing the same language—if the translations had not been erroneous before (here the time series is more important than consistency within the shared languages).

References [zotpressInTextBib style="apa" sortby="author"]