PDF 
Lesli Scott, Julie de Jong, and Kristen Cibelli Hibben, 2016
 

Appendices: A | B | C | D | E | F | G | H | I

Further Reading


Introduction
Conducting a multinational, multiregional, or multicultural survey, which we refer to as a “3MC” survey, involves careful coordination of many elements defined in the survey production lifecycle. Reflecting on the historical development of comparative studies, it is “…a quantum leap in complexity when one moves from the national to the multi-national arena in survey design and implementation” (Mohler, 2007, page 159). Research teams face many challenges in their attempts to manage all the requirements, elements, stakeholders and constraints of 3MC studies. The following guidelines provide a suggested framework for study management activities, incorporating aspects of the Project Management Body of Knowledge, often called PMI PMBOK® (Project Management Institute, 2013). After reviewing the study management guidelines below, we suggest reading the guidelines for each of the elements of the survey lifecycle relevant to your study (see the Chapters page for an overview and figure of the survey lifecycle).

This introduction begins by discussing why study management is important in the context of 3MC surveys. The next section provides generic project management concepts including definitions for key roles (project manager, team members and stakeholders), followed by a description of core project management topics including scope, schedule, cost and quality. Guidelines and procedural steps are then presented for the main phases of projects including the initiation, planning, execution, and closing phases. Throughout the chapter, there are tools and examples that may be helpful for 3MC study management. The appendices provide templates for some of these tools. Appendix I, presents a table with links to useful project management examples and resources from the European Social Survey (ESS) and the Teaching & Learning International Survey (TALIS).

Study management is critical to successful completion of survey projects. It embodies techniques that can be used to set and attain project goals and to manage activities effectively. Additionally, study management is essential for achieving the comparability and quality standards demanded by 3MC studies. “In comparative survey research, much more than the problems common to all mono-cultural surveys and measures need to be taken into consideration. In addition to depending on the quality of each individual national or cultural survey and measurement component, cross-cultural research is also dependent on their comparability.” (Johnson & Braun (2016), page 41). In the past, it was often assumed that countries were able to follow instructions or specifications “without much guidance or explanation” but many collaborators in study countries have found it challenging to institute the required protocols because of a lack of experience and infrastructure or may have taken short-cuts in quality assurance and quality control procedures (Lyberg, Japec & Tongur, 2017). Over the past few decades, comparative researchers have attended more rigorously to the planning, execution and evaluation processes that comprise study management. As a result, greater “methodological equivalence” such as that which defines current rounds of the European Social Survey (ESS), are being achieved (Jowell, Kaase, Fitzgerald & Eva, 2007).

The central coordination and local data collection efforts of many 3MC surveys are implemented within the structure of pre-existing programs that carry out ongoing operational activities. The organizations’ ongoing program activities may extend through long timeframes and can include many types of endeavors in addition to a specific survey project. The ESS European Research Infrastructure (ESS ERIC), for example, has implemented many surveys as well as conferences since 2001 within a program structure including a headquarters, a general assembly, a core scientific team, a national coordinators’ forum and others bodies (European Social Survey, 2016)).

But a specific survey needs to be managed as a distinct project, separate from the program(s) it may be associated with. A project (as opposed to a program) has some characteristic features. It is built around a specific and unique goal or set of goals. The project has a beginning and an end. There are limited resources, often revealed through the budget, for implementing the project. The goal or goals are closely tied to the research questions and the methods used to help answer these questions (see Guideline 1 of Study Design and Organizational Structure). The project ends after the goals have been met and all assigned resources have been utilized.

Round 8 of the ESS, for example, is a distinct survey project. There are specific goals which include standard ESS objectives plus new items representing changes from Round 7. The Round 8 survey has a beginning and an end (targeted as May 2015 – October 2017). The resources and budget for implementing Round 8 are pre-determined. The research questions and methods are outlined in the Round 8 Survey Specification (European Social Survey, 2015b). The project ends after the team members finalize and disseminate all Round 8 deliverables.

Note that not all projects are executed as part of the ongoing operations of an organization. Some projects are implemented within new and independent organizational structures that are formed specifically for the project and that dissolve at the end of the project.

The organizational structure chart below shows how a hypothetical organization might utilize employees from several program areas to implement a project. In this case, the organization has a top-level director (sometimes called minister or department head or other). This organization has two program areas including a research department and an operations department (where there are five specific units). The project structure includes individuals from all the program areas. The individuals are assigned to three sub-projects and the sub-projects are coordinated by three managers. Note that the set of individuals assigned to work on “sub-project C” are also separately assigned to work on “sub-project A”.

Example of how an organization might utilize employees from several functional areas to implement a project
study managment image 01

Some survey projects may be unidimensional, where a single project manager and project team handle all the roles and activities to meet the goals of the project. For example, a country’s education ministry might delegate responsibility to a project manager for implementing a survey of paper-based and self-administered questionnaires to regional education superintendents. A small project team might work with the project manager to handle all aspects including sampling, questionnaire development, pretesting, data collection, data processing, and final reporting. All the project phases and processes discussed in this chapter would apply to the tasks carried out by such a unidimensional team.

On the other hand, 3MC surveys typically include multiple dimensions where a central project management team may coordinate the efforts of several country-based or regional-based project teams that each individually implements local components of the survey. For example, a university-based coordinating team might plan and oversee a project that includes contracts with four local data collection teams in separate countries. The central coordinating team (see Guideline 2 of Study Design and Organizational Structure) and the local country teams would focus on different elements of the survey production lifecycle. But all the project phases and processes discussed in this chapter would apply to the tasks carried out by both the central coordinating and the local country teams. 

At a central level, a central coordinating project manager(s) and project team may specify and develop survey elements then contract with local data collection teams that manage the execution of local efforts.
  study managment image 02
       

The relationships between many organizations and entities involved in a given 3MC project become increasing complex as multiple countries and cultures join the effort. Distinct study management efforts (for example, discrete management plans) may be warranted at several levels. The coordinating center, each local country team, and each field data collection company may enact the principles that will be discussed in following sections. High levels of communication and clear accountability are required so that multiple study management efforts on a given survey remain synchronized.

Project Management Key Roles

The project manager role. The overall director (who might be known as the organization’s director, or the scientific lead, or principal investigator, or another title) typically delegates study management authority to a lead project manager. The project manager (who might also be the project director) is the person responsible for accomplishing the project or sub-project objectives. This means completing the project or sub-project on-time and within budget while meeting project specifications and quality. Project managers plan and direct a sequence of activities which involves: identifying requirements, addressing needs, concerns and expectations of stakeholders, maintaining strong communication channels, balancing competing constraints, and completing core processes and phases of the project. An effective project manager has well developed technical skills specific to project management, a strong understanding of the content area for the project, and excellent leadership skills.

The project team. The project team is comprised of the people who have assigned roles and responsibilities for completing aspects of the project. Project team members have budgeted effort and cost that will be monitored. The lead project manager may delegate oversight of components of the study to project leads. In the diagram below, twelve project leads have responsibilities for overseeing work and people associated with each of the elements in the survey lifecycle. For larger and more complex projects (as in the diagram), there may be a separate project lead for each element. For smaller projects, any given project lead and project team member may have oversight responsibilities associated with a set of many of the elements. 

Project leads may have responsibilities for overseeing work and people associated with the elements in the survey lifecycle.study managment image 03
 

The stakeholders. The PMI PMBOKdefines stakeholders as follows: “A Stakeholderis an individual, group, or organization who may affect, be affected by, or perceive itself to be affected by a decision, activity, or outcome of a project.” Examples of possible stakeholders in 3MC project include the funding sponsor, government representatives in participating countries, a technical advisory board, survey respondents, users of final reports and data systems, and others. Project managers and teams need to influence but do not directly manage stakeholders. 

Stakeholder influences diminishes and costs for making change increases.
  study managment image 04

It is most useful to obtain stakeholders’ input at early stages of the project (or lifecycle element) when stakeholders can successfully influence design decisions. If stakeholders try to influence the project (or lifecycle element) at later stages, there will likely be costly revisions and the impact of the stakeholders’ input may be less successful. 

Project management encompasses the application of knowledge, skills, tools and techniques to accomplish the project goals. Some of the broad elements of project management are:

-     defining goals, specifying requirements and establishing clear/achievable objectives;

-     collecting input from team members and stakeholders then weighing benefits and costs of different approaches;

-     balancing competing project constraints of scope, time and cost;

-     managing team member and contractor activities; and

-     producing and delivering the projects’ services and final products.

There are three dominant constraints (often called ‘triple constraints’) that interact on all projects: scope, time and costs. Quality influences the triple constraints and itself can be influenced by attempts to balance scope, time and costs.

Project Management Core Areas

  study managment image 05

The scope constraint. Scope pertains to the work necessary to deliver a product. Scope is about project scope and product scope. Project scope includes the work and activities necessary to complete all the deliverables and requirements. Product scope involves the requirements (e.g., statistical soundness, technical feasibility, ethical integrity, usefulness) that indicate when project deliverables are acceptable.

Scope considers a project’s boundaries: what work will be completed during the project lifecycle, and also what work will not be included. Project teams watch out for scope creep, which is a phrase used to describe uncontrolled changes or growth in the scope that must be constrained or may be harmful to costs, schedule and quality.

The time constraint. To assure project success, all aspects of the project need to be completed in a timely manner. Project teams typically use schedules to track and adjust time constraints throughout the project lifecycle. The process includes several steps which, depending on the nature of the project might be completed all at once by a single person or might be completed in stages by multiple people. These steps are: creating a detailed list of the activities; putting the activities into a sequence (earliest to latest); estimating the duration of each activity (the work effort and days); and putting them into one or multiple schedules. The following example includes columns for common items on schedule templates. This example also includes a graphical display of scheduling information. When the graphical display is present, this schedule format is called a Gantt chart (see also Tenders, Bids and ContractsAppendix A).

This schedule format is called a Gantt chart.
  study managment image 06

Throughout the life of the project, the project team continuously reviews the schedule(s) and periodically makes adjustments when actual progress occurs differently than originally estimated on the schedule. Some teams use a tool called the “Critical Path” to consider the best ways to adjust a schedule. In a critical path diagram, the tasks are listed in sequence and according to their dependencies. The longest path identifies the timelines for the set of tasks that would need to be adjusted if the overall schedule needs to decrease. The time period for the critical path might be shortened if additional people and their work efforts are added to a task (called crashing) or if some tasks in the sequence have slack and/or they can be rescheduled in parallel with earlier tasks (called fast tracking). The diagram, below, explains how to find the critical path. 

Since the critical path is the longest path through the network diagram, Path 2, B-E-H-J, is the critical path for this project.
  study managment image 07

Assume the durations are shown in months.

Path 1:        A-D-H-J      Length = 1+4+6+3 = 14 months

Path 2:       B-E-H-J      Length = 2+5+6+3 = 16 months

Path 3:        B-F-J          Length = 2+4+3 = 9 months

Path 4:        C-G-I-J       Length = 3+6+2+3 = 14 months

The cost constraint. Staying within the budget is one of the most important expectations faced during study management. Usually, 3MC projects start with a rough initial budget that was part of the tender or bid packet that led to the project award. The project manager and/or team will then build a more detailed budget by estimating costs to complete work specified in the activity lists and deliverables schedule. The budget includes labor items (salaries and relevant employee benefit costs), and non-salary items (travel, equipment, materials, contractor costs and such).

One tool that can help with labor cost estimation is called work breakdown structure (WBS). A WBS divides the work activities into small pieces that can be assigned to workers and that can be tracked to assure the project stays on track. A WBS has two levels – the summary tasks and the work packets. Summary tasks pertain to elements of the project such as”create the questionnaires’ and ‘train the interviewers”. There may be a second level of summary tasks. For example, “produce paper versions of questionnaires” may have a second level that includes “produce paper versions of the parent questionnaire” and “produce paper versions of the children questionnaire”. For complex projects, there may be many levels of summary tasks.

The work packets are the lower level tasks that provide details of work that will be assigned. As a rule of thumb, work packets might require about eight to eighty work hours to complete – a reasonable numbers of hours that can be specified and that makes sense as an assignment. In the WBS diagram, below, there are four work packets for the summary task called “4.1 Prepare Interview Manual.” The work packet 4.1.1 might require about 20 work hours and might be completed by a small team. The work packet 4.1.2 might require about 8 work hours and might be handled by one project manager. That same project manager might complete the work packet 4.1.3 which might require about 8 work hours. And the work packet 4.1.4 might require 10 work hours that the small work team members could split. 

A work breakdown structure has two levels–summary tasks and work packets.
  study managment image 08

Project managers will consider how to distribute the work packets to specific team members. All the work packet hours for a given team member will be summed and multiplied by that person’s hourly rate (cost per unit). Adjustments can be made if the overall WBS budget total does not match the budget amount that the project was awarded.

After estimates are created, the project manager will input all team members’ information plus the non-salary cost estimates into a master budget. A basic master budget example is given, below.

Basic master budget example.

  Number Number of Months Cost per Unit Total Cost
Base salaries        
         
  Project Manager        
  Data Manager        
  Fieldwork Manager        
  Accountant        
  Assistants        
  Supervisors        
  Interviewers        
  Data entry operators        
  Drivers        
  Translators        
  Computer programmers        
  Incentive payments        
         
Travel        
  Researchers        
  Interviewers        
         
Materials        
  Computers        
  Printers, etc.        
  Computer/printer supplies        
  Photocopier/Fax machine        
  Office supplies        
  Communications (phone, fax, postage, etc.)        
  Equipment maintenance        
         
Printing costs        
  Questionnaires        
  Training manuals        
  Reports        
  Miscellaneous (maps, listings, manuals, etc.)        
         
Consultant costs        
  International consultants        
  International per diem        
  Local consultants        
  Local per diem        
  Local travel        
         
Contingency (100%)        
         
TOTAL COST        

As the project is implemented, there are different techniques for monitoring whether project expenses are in line with project progress. Some projects incorporate the cost information into the project schedule so they can project the expected budget balance at several defined time points. Project management software can be useful for this. Some projects use a technique called ”earned value analysis” which determines if the work effort expended and the costs incurred (i.e., actual cost) yielded the expected progress (i.e., planned value).

Earned value analysis shows if work erform better or worse than planned
study managment image 09

The quality dimension. Project managers are always thinking about quality issues as they balance the ‘triple constraints.’ The guidelines on Survey Quality provide guidance for assessing and managing quality of 3MC survey deliverables and outputs (for example, data sets and response rates and bias). Those guidelines discuss project management processes and tools used to address quality, including: quality planning, quality assurance steps, as well as monitoring and controlling activities.

Beyond these core project management areas, there are other areas, including: human resources, communications, risk, procurement and change-management. These may be discussed as part of the core areas in the base management plan or these may have distinct importance on some 3MC projects and thus warrant their own management plans. More information about all project management areas are available through the PMI website (Project Management Institute, 2016).

Project Management Phases

There are four project management phases that all projects pass through: initiation, planning, executing and closing phases (some project teams may break the executing phase into two separate pieces – implementing and controlling/monitoring). During the initiation phase, the project managers gather information about the project and obtain authorization to move forward with project work. During the planning phase, the project managers create a management plan that addresses the core areas of scope, time, costs, and quality as well as other relevant areas (human resources, communications, risk, procurement and change-management). During the executing phase, the project team members complete the work activities and produce survey deliverables using project management monitoring and controlling processes. During the closing phase, the project managers archive project elements, obtain acceptance of deliverables, and document the lessons-learned which may help future survey efforts.

The guidelines below outline activities that are completed in each of these phases. In large and complex projects, there may be sub-project components that independently pass through the phases at different times. The four core project management phases are useful for all projects, regardless of size and complexity.

Guidelines

Goal: To establish a project structure for managing the 3MC survey lifecycle and to use project management processes and tools to effectively complete the study management phases: initiation, planning, execution, and closing.

1.   Implement the Initiation Phase of the project.

Rationale

After there is a trigger indicating the project will occur (e.g., a funding announcement), it is important to establish that project sponsors and decision-makers are committed to moving forward. Early in the initiation phase, project managers educate project leaders in ways to help everyone agree about the goals and approaches. In this phase, project leaders authorize the project managers and team to carry out the project.

Procedural Steps

1.1    Develop a clear understanding of the project. This begins by clarifying the research questions, aims and objectives described in Study Design and Organizational Structure Incorporating and sharing this understanding will help keep project leaders, project stakeholders and team members aligned. Steps that can be taken include:

1.1.1   Create a project summary that is easy to share and can be included in future documents. The summary can include:

  • A problem statement that describes what needs to be solved and why;
  • Project goals (which integrate research aims and management aims) that are high level targets and that state the end results; TALIS
  •  Clear, measureable and realistic objectives that provide specific details for the goals and may include: research objectives, financial objectives, business objectives, quality objectives, technical objectives, performance/completion objectives and such.

1.1.2   Begin to create a detailed list of project stakeholders and anticipate their relationships to the work activities.

1.1.3   Determine the approaches that will work best within the culture and organizational structure of the project. For example, if there are multiple project managers, determine whether each will have a distinct budget to manage or if all managers will use the same master budget. There will be different strategy options and it may be helpful to assemble project team members to brainstorm ideas, assess feasibility and consider the desirability of different approaches.

1.1.4   Gather requirements which include details of what the research and management outcomes will look like. Some requirements might be stated in the contract with the sponsor (e.g., the project might need to complete certain deliverables before the second year of funding is released). Some requirements will describe the products (e.g., the sample design, survey questionnaires, etc.) and details might be gathered from stakeholders including the sponsor and the principal investigator. To verify understanding of product requirements, some projects create prototypes before moving into the planning phase. Many of the requirements will pertain to the management aspect: what are the quality standards, what ethical issues need to be met, what milestones belong on the schedule, what level of expertise is required from scientific/technical human resources, what are the budget issues?

1.1.5   List the deliverables and each of their success criteria. For example, the list might include the item ”survey questionnaire” and the success criteria might include “short enough so response rate is not compromised” and “sufficiently tested so response bias is low/acceptable”.  

1.1.6   Identify the assumptions for the project. Stakeholders may have specific expectations. For example, household respondents may expect that husbands may join their wives’ interviews. Or, data end users may expect to receive a specific file format. Early interactions with stakeholders can help avoid later misunderstandings.

1.1.7   Identify potential risks.

1.1.8   Gather information from past similar projects.

1.2    Draft a scope statement. A scope statement provides the road map that guides the project team throughout the project and lets everyone know what is expected. As quoted from baseball-legend Yogi Berra, "You've got to be very careful if you don't know where you're going, because you might not get there” (FREEP, 2015).

1.2.1   The components of a typical scope statement are listed below.

  • Scope description, based on project summary.
  • Deliverables list, which may be elaborated during planning.
  • Acceptance criteria, which indicates what the sponsor and/or director (a.k.a. principal investigator or scientific lead) require in order to accept the deliverables, and may include quality elements.
  • Exclusions, which include those things that are out-of-scope for the project team.
  • Constraints, which are factors that may limit or have an impact on final project results.
  • Assumptions, which may be a list of major cost drivers that impact the deliverables (e.g., the period of performance and the target response rates).
  • Staffing/scheduling plan indicating the individuals that will work on the project during what time periods.

1.2.2   Use the scope statement to define the boundaries of the project – what team members should expect to work on and especially what is out-of-bounds. For example, separate from the project budget, the sponsor might deliver a sample frame from a “sampling database vendor” to the project team. In this case, it would be out-of-scope (and duplicative) for the project team to design and create or to seek a sample frame for the project.

1.3    Write a project charter and obtain signatures from the project decision-makers. The project charter is a high-level document that provides a synthesis and authorizes the project. It is short and concise and does not change over the life of a project unless there is a dramatic scope revision. Signatures are added to the document to: a) demonstrate the project team commitment; b) provide authorization to start the project from the top-level decision makers; and c) specify the project manager, PI, or scientific lead, additional managers, and pre-committed core staff members.

1.3.1   The components of an example project charter are listed below.

  • Basic project information such as name and related projects.
  • Project management team members and internal authorities to whom the team reports.
  • Sponsor and significant stakeholders (often called the customer).
  • Business objective (why the performing organization is interested in conducting the project).
  • Project objective (top-level statement of research aims).
  • Deliverables (top-level list of primary deliverables).
  • Risks, constraints and assumptions (top-level lists that all parties understand and agree can be tolerated).
  • Schedule milestones (most significant events on the schedule).
  • Overall budget.
  • Signature section.

Lessons Learned

1.1  Taking time to complete the steps in the initiation phase yields great benefits. The process helps build team cohesion and sponsor confidence. The outputs are useful for future communications and they facilitate the steps in the other project phases. Especially important, the initiation activities result in full commitment to the project from team members and stakeholders.

1.2  Sponsors and directors (e.g. principal investigators / scientific leads) will vary on how much they desire to participate in the initiation phase. They may not be interested in learning specific project management vocabulary or tools. And, they may not understand the value behind disciplined use of documents like the scope statement and project charter. Whether or not these specific documents are shared, the decision makers will need to be involved in two things: verifying the project scope and budget, and providing authorization for the project team to launch the study.

1.3  Establishing effective communication channels with project stakeholders early in the project can help reduce the chance that barriers will slow down the project progress. During the initiation phase, project managers should be able to identify most of the stakeholders they need to consider during the project life cycle.

1.4  As the project team begins to form, project managers should spend time informing team members about the elaborating details of the project and seeking input from those individuals that have knowledge about specific elements of the project. It takes time to build an effective team. Especially in the early phase of the project, project managers will benefit if they invest time building team understanding and commitment.

1.5. When there are clear sub-projects, especially when they have their own distinct budgets, the relevant project team members may participate in multiple initiation phase activities. For example, a late decision to fund the creation of a data repository at the end of a project might result in a distinctly budgeted sub-project and a few of the project team members will carry out the initiation phase activities when it is time to launch this component.

2.   Implement the ‘Planning Phase’ of the project.

Rationale

Project plans are used to guide how projects will be implemented, including what will be done, who will do what, how stakeholders and team members will receive information, how progress will be tracked, ways the project plan might be corrected when risks are encountered and how quality will be ensured.

Procedural Steps

2.1    Hold an initial planning meeting with all the team members which might include review of the documents developed during the initiation phase, discussion of team members’ roles/assignments across the project elements, and consideration of the project milestones.

2.2    Working through the elements in the production lifecycle, create activities lists and work breakdown structures.

2.2.1   In addition to the elements in the production lifecycle, use scope and deliverables documents to list top level summary activities.

2.2.2   Under each summary activity, list the specific tasks.

2.2.3   Create groupings that can be completed in 8-80 hours (i.e., WBSs).

2.2.4   Describe the work using a detail level that clarifies ‘what to do’ but also recognizes that the person or team that will implement it knows more about ‘how to do it’.

2.3    Put together the project schedule. There are project management software packages that provide technical tools for developing schedules. Some software integrates schedules with resource allocation and budgets. 3MC projects may have sub-projects and some teams might develop separate schedules for the sub-projects. Steps for developing the schedule include:

2.3.1   Put activities in order from earliest to latest.

2.3.2   Estimate the duration of activities.

  • Most projects estimate activities’ durations from the top to the bottom (major phases broken to smaller) but it sometimes works better to estimate from the bottom to the top.
  • If available, historical information from past projects and input from experienced colleagues might help with estimating.
  • Techniques can be used to analyze whether or not the combined durations can be accomplished within the scheduled project period. The critical path tool (discussed in the Introduction of this chapter) uses activity durations to find the longest sequence of tasks in the project. The ‘program evaluation and review technique’ known as PERT graphically displays the pathway and durations between milestone activities. Information about these techniques is available at the PMI website.

2.3.3   Specify the dependencies among activities – what is required to be complete before each next task? The most commonly used type of dependency is ‘finish-to-start’ (finish task-A then start task-B). Other types of task dependencies include ‘finish-to-finish,’ ‘start-to-finish,’ and ‘start-to-start.’

2.3.4   Place the activities on a calendar.

  • Consider if there are pre-determined deadlines for some tasks then work backwards from them.
  • Include start and end dates for the activities based on the duration estimates.
  • Add milestones which are key project events that don’t have durations but mark important things like achievements and due dates.

2.4    Assign activities to individuals.

2.4.1   Based on activity durations, determine the number of hours needed then consider how many individuals are needed. Then, based on their available hours per day, determine how many days the activity will take. Take into account that individuals will have other commitments (for example, department meetings) and some may work on multiple projects in the same week.  

2.4.2   Consider how efficient individuals might be when estimating the hours and days needed. Think about all things that influence productivity – for example, multi-tasking tends to decrease productivity.

2.4.3   Integrate the calendar and the results of assigning activities to see if any changes in the calendar are needed. For example, is the overall schedule delayed because some specialists are only available to start activities at later points than anticipated? If needed, consider techniques for shortening the schedule such as ‘crashing’ or ‘fast-tracking’ as discussed in the Introduction of these guidelines.

2.4.4   Obtain agreement from staff supervisors that the project plan can include the individuals for time periods and numbers of hours desired. 

2.5    Specify details about the people and resources.

2.5.1   Add the names of individuals to the schedule and using actual pay rates, calculate salary cost of the labor.

2.5.2   Create a responsibility matrix which designates who leads and who works on major areas of the project.

2.5.3   Create an organizational chart for the project.

2.6    Create a project budget.

2.6.1   The project budget will include both labor costs and non-salary costs. A summary budget may include a line (row) for each broad category, for example, managers, programmers, supervisors, interviewers and such. A detailed budget may include line (row) for each individual (by name) and each specific non salary item in the budget. 

2.6.2   Typically, there is a rollup budget showing the total budget combining all years of the study as well as individual year-by-year budgets. Often this is accomplished by using tabs in a spreadsheet for yearly budgets that link to a master tab/sheet with the rollup budget.

2.6.3   The budget may be broken in to finer time periods if these will be needed for monitoring and reporting purposes.

2.6.4   Some projects hold a budget line with ‘contingency funds,’ that is undesignated funds that can help cover unanticipated costs.

2.7    Assess and plan for project risks. “Risk can be defined as the function of three variables, an event that could disrupt the project, the probability that the event could happen, and the impact the event will have on the project if it does happen” (Cook, 2005). Every project faces the chance that anticipated risks, as well as risks that were never imagined, can interrupt the project. By planning ahead, the impact of risks on the project can be reduced.

2.7.1   During early planning stages, the project team should attempt to identify potential project risks. Team members as well as others that have handled projects in the past may be able to help create lists of risks. There are several general areas that may introduce risk. Examples of conditions that may increase risk include: high-levels of project complexity, new technology that may be poorly tested or not work as promised; geographic dispersion of team members that may increase miscommunication; and lower-level of experience by team members.

2.72    There are two key questions that can be asked for each identified risk: what is the potential impact if the risk occurs and what is the likelihood that the risk might threaten the project?

2.7.3   For planning purposes, the team might use a form to list and analyze the risks.

2.7.4   For each risk, the team can indicate a response. Some of the techniques include:

  • Acceptance (planning no action and living with the consequences which makes sense if the costs of handling the risk are greater than the cost of the risk).
  • Avoidance (changing the project plan to eliminate the risk).
  • Transference (hiring a third party to handle the risk).
  • Mitigation (making small project updates that won’t eliminate but will reduce the probability or impact of the risk).

2.8    Determine the communication needs for the project. It is especially important to establish good communication when a project is complex and geographically dispersed, like many 3MC projects. Steps to help plan for effective communication include:

2.8.1   Determine what stakeholders and team members need to know about the project.

2.8.2   Consider what communication channels work best and under what circumstances.

  • When and how often are written/posted status reports most effective?
  • When is it beneficial to hold face-to-face sessions?
  • Which team members can effectively receive and participate in email exchanges, conference calls and videoconferencing?

2.9    Specify what project changes the team should track and manage. Since all projects experience changes, it is essential to create a plan for change management. A useful strategy is to choose baseline documents, such as the scope statement, the schedule, the management plan, and then handle changes through version control as these documents are updated to reflect change. Some projects require team members to submit written change requests and receive approval before aspects of the project can be amended.

2.10 Compile the written project management plan. Often kept as an electronic document, the project management plan includes sections for the project components and it contains project planning documents and project monitoring materials. The project management plan will address all phases (initiation, planning, execution and closing) and core processes (scope, schedule, cost – all informed by quality). One single project management plan might include all the project components (for example, the survey production lifecycle components) and all the additional processes (communication, risk, change, and procurement). Alternatively, the project team may decide to work with multiple project management plans that break processes and/or components for convenient oversight.

Lessons Learned

2.1. As projects become more complex, it is usually necessary to add extra effort and time specifically to account for the complexity. This is because there will be more interactions with increased communications requirements and extra management needs.

2.2. Many aspects of planning (and project management, in general) require high levels of communication with project team members and stakeholders. Special steps may be required on 3MC projects to account for different languages and communication norms.

2.3. Developing plans that work well in specific local areas may require project managers to consult with local residents or experts. Several examples of ways local stakeholders contribute to the sample plan development are discussed in Guideline 2 of Sample Design.  And, Guideline 3 of Questionnaire Design discusses the importance of including local participants when defining the approach for creating questionnaires.  Guideline 5 of Instrument Technical Design suggests  that plans for usability tests should consider the involvement of interviewers and it discusses issues when the interviewer and participants are from the same or from different cultures.

2.4. Plans that include the use of technology will need to include extra schedule time for development and testing. Guideline 6 of Instrument Technical Design discusses lessoned learned in this regard in Burkina Faso.

2.5. Quality assurance and quality monitoring should be addressed early in the design planning process. An example of how Thornton et al. (2008) handled this is discussed in Guideline 6 of Questionnaire Design.

2.6. The project team should develop and share thorough written documentation of plans and adjustments to plan. Beginning this work in early phases of the project and continuing through the lifecycle will result in high quality final written products.

3.  Implement the ‘Execution Phase’ of the project.

 Rationale

The “Execution Phase’ of the project includes implementing steps, as well as monitoring and controlling activities.

Procedural Steps

 3.1    Build and strengthen the project team. The most important assets on a project are the people. Project managers use strong interpersonal and leadership skills to gain high performance from individuals and the overall team. Specific techniques include:

3.1.1   Through the project period, review and clarify the roles and responsibilities each team member holds. Help a given team member understand his or her part and importance on the project. Communicate information to all team members so they understand how their own area of responsibility interacts with their colleagues’ areas of responsibility.

3.1.2   Provide specific and achievable goals. Through work packets or other means of assigning work, provide directions that help a given team member determine what will result in ‘success’ and how to measure progress towards completing goals.

3.1.3   Show respect and be honest in all interactions with the team. Individuals enjoy their work more and perform better when they feel valued and believe they can trust leaders.

3.1.4   Provide feedback to individuals and small teams in a timely manner. When effective, correcting feedback and positive/affirming feedback can help teams understand how well they are performing and help build self-confidence.

3.1.5   Support the individuals and the overall team. If there are obstacles hindering success, provide support and commitment to mitigate the problems.

3.1.6   If the team suffers from ‘people problems’ (which might include conflicts or mixed commitment to the project or other issues), address these issues immediately.

3.1.7   Devote sufficient time in meeting and communicating about the project with the team members. Encourage team leaders to communicate frequently with their team members. Up to 80% of project management involves communication and this activity will significantly enhance project success.

3.2    Maintain the project planning documents. As the project is executed, many aspects of the original plan need to be updated to fit actual and changing circumstances.

3.2.1   Retain the baseline documents to help with ‘lessons learned’ and to provide evidence when justifications for changes are required.

3.2.2   Use version control practices to produce and maintain revised planning documents. Team members should have easy access to those planning documents that help them with their work.

3.3    Carry out monitoring activities for all project activities. Do all activities comply with ethics and standards set for the project? Is the specified quality being met? Are all aspects of the scope (product and project) being met? Are there any scheduling problems? Is the work being completed as budgeted? Have new risks been identified?

3.3.1   Gather and analyze data about the survey production elements. The data may cover status, budget, quality and auxiliary areas.

3.3.2   Define what reports will be useful and how they will be produced.

  • On a regular basis, gather information from teams (for example, in regular team meetings) and from technical systems.
  • Consider using electronic reporting systems and/or web-based ‘dashboards’ that provide information to team members and stakeholders, on-demand.

3.3.3   Compare data to expectations in the planning resources.

  • Status information may focus on progress towards meeting the scheduled milestones. Did tasks begin on time and does the actual duration match the planned duration? A Gantt chart may help identify any variances.
  • Cost variances may occur if there are extra hours per task or higher costs for units of work or materials. As discussed in the Introduction of this chapter, earned value analysis can help determine if schedule and cost variances are significant.
  • Survey data may be analyzed during the data collection phase to determine if the questionnaires are performing as expected and with acceptable levels of bias (see Paradata and Other Auxiliary Data).

3.4    When projects are ‘off-track,’ carry out corrective steps in ways that manage and control the changes.  

3.4.1   ‘Responsive design as discussed in Survey Quality provides guidance for correcting project issues.

3.4.2   Consider whether or not the project manager and project team have authority to implement specific changes. Some changes require higher authority approval.

3.4.3   Consider what type of impact the proposed changes might have on the budget. Might they require approval to utilize contingency funds? 

Lessons Learned

3.1    During the executing phase, project team members and their managers may become consumed with work activities related to production of the survey. But more than ever, project managers need to spend time on project management activities in order to keep the project on track. Almost always, this is the most costly portion of the project and it is critical to pay attention to scope, schedule, cost and quality performance.

3.2    The value of communication with team members and stakeholders in this phase cannot be over-stated.  As noted in Guideline 4 of Questionnaire Design, not all  participating groups in a 3MC project will be confident about providing input. It is important to emphasize that every contribution is valued even when not all suggestions are incorporated into design modifications.

3.3    Implementing quality control protocols from the start of a project permits the survey organization and coordinating center to monitor performance and take corrective action when required. Guideline 5 of Interviewer Recruitment, Selection, and Training, for example, discusses how interviewer certification protocols might be implemented in conjunction with additional interviewer training when interviewer candidates fail to pass on their first try.

3.4    Many organizations have used a quality control technique known as adaptive or responsive design which uses paradata collected during survey implementation to determine if performance such as non-response and response bias indicates the project needs to adapt/correct the original design (Groves & Heeringa, 2006).

3.5    Lessons from other projects can be helpful and theEuropean Social Survey (ESS) provides evidence of success using continuous improvement techniques for planning and implementing the survey (Pennell, Cibelli Hibben, Lyberg, Mohler, & Worku, 2017).

3.6    A growing number of organizations are adopting professional project management frameworks to conduct their project activities. The Project Management Institute (PMI) and the International Project Management Association (IPMA) provide two of the most commonly followed frameworks. Many organizations follow institute-wide professional project management best practices and encourage staff members to become certified project managers.

4.   Implement the ‘Closing Phase’ of the project.

Rationale

 As work is ‘wrapped up’ for the final elements of the survey production cycle, managers need to take steps to effectively close the project.

Procedural Steps

4.1    Distribute the final deliverables. These deliverables may include data systems, reports, instruments, and other products. See Data Dissemination for additional details.

4.1.1   Data sets for surveys typically need to be transferred with special consideration to assure that privacy and identities of survey participants are protected. Many projects are required to operate under data management plans. These plans may specify:

  • Some data may be restricted from distribution beyond the protection of the project’s secure storage system and may need to be destroyed during the closing phase.
  • There may be specific methods that must be used to transfer data from the project to other parties. Legal agreements such as data transfer and data use agreements may be required.

4.1.2   Final reports and other products may be developed for public dissemination or may be transferred only to the sponsor and specific stakeholders.

  • The final report should be nearly complete at the end of the study if the project team keeps the management plan updated throughout the project.
  • Final reports and data products may be compiled for hard copy production (for example, the Survey of Health, Ageing and Retirement in Europe (SHARE)) or for on-line access (for example, the ESS).

4.2    Obtain formal notice of acceptance of deliverables.

4.2.1   The project might use basic procedures such as email confirmation that each deliverable is acceptable.

4.2.2   Some projects provide ‘acceptance’ documents that verify that ‘success criteria’ are fully met.

 4.3   Carry out steps to gather ‘lessons learned’ information.

4.3.1   Throughout all phases of the project, managers will gather information that can inform a final summation of the ‘lessons learned.’

4.3.2   At the end of the project, it is useful to hold debriefings with team members and stakeholders to consider ‘what went well’ and ‘what could have gone better.’

4.4    Close all contracts and complete all requirements in legal agreements such as non-disclosure documents, memos of understanding, data use agreements, human subject protections documents, and such.

4.5    Archive project items, including a project close out report.

4.6    Verify that all team members are transitioned off the project.

4.7    Acknowledge successful completion of project. Congratulations!

Lessons Learned

4.1. The last parts of the project ‘execution phase’ may experience very tight timelines which may compromise the timeframe for the ‘closing phase.’ With this in mind, many of the closing activities can be started even as earlier phases are in progress.

 4.2. The archiving activities and production of final reports are much easier when project management processes are maintained throughout the project life and when project teams produce on-going survey and project management documentation.