Statistical Methods, Standards and Guidelines 24
2.5
DATA ANALYSIS
2.5.1 Analysis and Report Planning
Standard 2.5.1: MDAs, LGAs and other stakeholders must develop a plan for the analysis of
survey data prior to the start of a specific analysis to ensure that statistical tests are used
appropriately and that adequate resources are available to complete the analysis.
The guidelines for this standard are:
Guideline 2.5.1.1: Include the following in the analysis plan:
(a)
An introduction that describes the purpose, the research question, relevant
literature, data sources (including a brief description of the survey data and any
limitations of the data), key variables to be used in the analysis, type of analysis,
and significance level to be used;
(b)
Tables and figures that support the analysis; and
(c)
A framework for technical notes including;
(i)
History of the survey program;
(ii)
Data collection methods and procedures;
(iii)
Sample design;
(iv)
Response rates and the treatment of missing data;
(v)
Weighting methods;
(vi)
Computation of standard errors;
(vii)
Instructions for constructed or derived variables;
(viii) Limitations of the data; and
(ix)
Sources of error in the data.
2.5.2 Inference and Comparisons
Standard 2.5.2: MDAs, LGAs and other stakeholders must base statements of comparisons and
other statistical conclusions derived from survey data on acceptable statistical practice.
The guidelines for this standard are:
Guideline 2.5.2.1: Specify the criterion for judging statistical significance for tests of hypotheses (Type I error) before conducting the testing.
Statistical Methods, Standards and Guidelines 25
Guideline 2.5.2.2: If part of an historical series is revised, data for both the old and the new series should be published for a suitable overlap period for the use of analysts.
2.6
REVIEW AND REBASING PROCEDURES
2.6.1 Review of Information Products
Standard 2.6.1: MDAs, LGAs and other stakeholders are responsible for the quality of
information that they disseminate and must institute appropriate content/subject matter, statistical,
and methodological review procedures to comply with NBS and MDAs, LGAs and other
stakeholders.
The guidelines for this standard are: Guideline 2.6.1.1: Conduct a subject-matter review of all information products that present a description or interpretation of results from the survey, such as analytic reports or “briefs.” Select reviewers with appropriate expertise in the subject matter, operation, or statistical program discussed in the document. Among the areas that reviewers should consider are the following:
(a)
Subject-matter literature is referenced in the document appropriately;
(b)
Information is factually correct; and
(c)
Information is presented clearly and logically, conclusions follow from analysis,
and no inconsistent findings are ignored.
Guideline 2.6.1.2: Conduct a statistical and methodological review of all information products. Select reviewers with appropriate expertise in the methodology described in the document. Among the tasks that reviewers should consider are the following:
(a)
Review assumptions and limitations for accuracy and appropriateness;
(b)
Ensure that appropriate statistical methods are used and reported;
(c)
Review calculations and formulae for accuracy and statistical soundness;
(d)
Review data and presentations of data (e.g., tables) for disclosure risk;
(e)
Review contents, conclusions, and technical (statistical and operational areas)
recommendations to ensure that they are supported by the methodology used; and
Statistical Methods, Standards and Guidelines 26
(f) Ensure that data sources and technical documentation, including data limitations, are included or referenced.
Guideline 2.6.1.3: Review all information products that will be disseminated electronically for compliance with existing policies and legislations governing statistical activities in the country for easy accessibility. Ensure that any product that is disseminated via special software is tested for accessibility and interpretability prior to dissemination.
Guideline 2.6.1.4: Rebase and update information products with new and current data produced from survey and routine data.
Guideline 2.6.1.5: Review and harmonize information products using common standards and levels of classification such as ISIC, SNA, etc. Avoid aggregating information products using different standards and classifications before harmonizing them. If it is necessary to customize to local conditions or branch out of main classifications, maintain the link to international standard.
2.7
DISSEMINATION
2.7.1 Information Release Standard 2.7.1: MDAs, LGAs and other stakeholders must release information intended for the general public according to a dissemination policy that provides for equal and timely access to all users and provide information to the public about the MDAs, LGAs and other stakeholders’ dissemination policies and procedures including those related to any planned or unpredicted data revisions.
The guidelines for this standard are:
Guideline 2.7.1.1: Dissemination procedures for major information products may include the
following:
(a)
Develop release calendar and method for the release of information products and
services;
(b)
Inform targeted stakeholders; and
(c)
Ensure equal and timely access to all users.
Statistical Methods, Standards and Guidelines 27
Guideline 2.7.1.2: Protect information against any unauthorized pre-release, and release information only according to established release procedures.
Guideline 2.7.1.3: If revisions to estimates are planned, establish a schedule for expected revisions, make it available to users, and identify initial releases as preliminary.
Guideline 2.7.1.4: Establish a policy for handling unscheduled corrections due to previously unrecognized errors. The policy may include threshold criteria (e.g., the correction will change a national level total value by more than one percent or a regional value by more than five percent) identifying conditions under which data will be corrected and re-disseminated.
Guideline 2.7.1.5: When information products are disseminated, provide users with access to the
following information:
(a)
Definitions of key variables;
(b)
Source information, such as a survey form number and description of methodology
used to produce the information or links to the methodology;
(c)
Quality-related documentation such as conceptual limitations;
(d)
Variance estimation documentation;
(e)
Time period covered by the information and units of measure;
(f)
Point of contact to whom further questions can be directed;
(g)
Software or links to software needed to read/access the information and
installation/operating instructions, if applicable;
(h)
Date the product was last updated; and
(i)
Standard dissemination policies and procedures.
2.7.2 Documentation and Archiving Standard 2.7.2: MDAs, LGAs and other stakeholders must produce survey documentation that includes those materials necessary to understand how to properly analyze data from each survey, as well as the information necessary to replicate and evaluate each survey’s results. Survey documentation must be readily accessible to users, unless it is necessary to restrict access to protect confidentiality.
The guidelines for this standard are:
Statistical Methods, Standards and Guidelines 28
Guideline 2.7.2.1: Survey system documentation (metadata) includes all information necessary to
analyze the data properly. It includes the following:
(i)
Survey instruments;
(ii)
Description of variables used to uniquely identify records in the data file;
(iii)
Description of the sample design, and sampling unit identifiers to be used for
analysis;
(iv)
Definitions of all variables, including all modifications;
(v)
Data file layout;
(vi)
Descriptions of constructed variables on the data file that are computed from
responses to other variables on the file;
(vii)
Description of sample weights, including adjustments for non-response and
benchmarking and how to apply them;
(viii) Description of how to calculate variance estimates appropriate for the survey
design;
(ix)
Description of all editing and imputation methods applied to the data (including
evaluations of the methods) and how to remove imputed values from the data;
Guideline 2.7.2.2: To ensure that a survey can be replicated and evaluated, the MDAs, LGAs and
stakeholders’ internal archived portion of the survey system documentation, at a minimum, must
include the following:
(i)
Survey planning and design decisions, including the NBS Information Collection
Request package;
(ii)
Field test design and results;
(iii)
Selected sample;
(iv)
Sampling frame;
(v)
Justifications for the items on the survey instrument, including why the final items
were selected;
(vi)
All instructions to respondents and/or interviewers either about how to properly
respond to a survey item or how to properly present a survey item;
(vii)
Description of the data collection methodology;
(viii) Sampling plan and justifications, including any deviations from the plan;
(ix)
Data processing plan specifications and justifications;
(x)
Final weighting plan specifications, including calculations for how the final
weights were derived, and justifications;
Statistical Methods, Standards and Guidelines 29
(xi)
Final imputation plan specifications and justifications;
(xii)
Data editing plan specifications and justifications;
(xiii) Evaluation reports;
(xiv) Descriptions of models used for indirect estimates and projections;
(xv)
Analysis plans;
(xvi) Time schedule for revised data; and
(xvii) Documentation made publicly available in conjunction with the release of data.
Guideline 2.7.2.3: For recurring surveys, produce a periodic evaluation report, such as a
methodology report, that itemizes all sources of identified error. Where possible, provide
estimates or bounds on the magnitudes of these errors; discuss the total error model for the survey;
and assess the survey in terms of this model.
Guideline 2.7.2.4: Retain all survey documentation according to appropriate Tanzania records disposition and archival policy and law.
Statistical Methods, Standards and Guidelines 30
PART II:
STANDARD CLASSIFICATIONS IN STATISTICAL PRODUCTION
Classification systems are used to group statistical data according to criteria that make
them more homogeneous and more likely to be used for accurate analysis. The
harmonization and adaptation of these reference classifications play a fundamental role in
the overall harmonization process of integration statistics.
At the international level, there are several reference classifications, recorded in the
classification registry of the United Nations Statistics Division (for the most part,
economic classifications). Several categories of economic classifications exist, which are:
Classifications of economic activities, e.g. International Standard Industrial Classification
(ISIC); Product classifications, e.g. Harmonized commodity description and coding
System (HS), Central Product Classification (CPC), Standard International Trade
Classification (SITC); Functional classifications, e.g. Classification Of Individual
Consumption by Purpose (COICOP), International Classification of Activities for Time –
Use Statistics (ICATUS); and Occupational classifications, e.g. Tanzania Standard
Classification of Occupations (TASCO).
3.0
THE INTERNATIONAL STANDARD INDUSTRIAL CLASSIFICATION OF ALL
ECONOMIC ACTIVITIES (ISIC)
ISIC is the international reference classification of a coherent and consistent structure of economic
activities based on a set of internationally agreed concepts, definitions, principles and
classification rules.
ISIC structure
The hierarchy in ISIC consists of:
(i)
Section
(one letter code)
(ii)
Division (two digits code)
(iii)
Group
(three digits code)
(iv)
Class
(four digits code)
Statistical Methods, Standards and Guidelines 31
The objective and uses of ISIC ISIC is intended to be a standard classification of productive activities. Its main purpose is to provide a set of activity categories that can be utilized for the collection and presentation of statistics according to such activities. Therefore, ISIC aims to present this set of activity categories in such a way that entities can be classified according to the economic activity they carry out.
It provides a comprehensive framework within which economic data can be collected and reported in a format that is designed for purposes of economic analysis, decision-taking and policy-making. The classification structure represents a standard format to organize detailed information about the state of an economy according to economic principles and perceptions.
The ISIC has widely being used both nationally and internationally in classifying data according to kind of economic activity in the fields of production, employment, gross domestic product and other statistical areas. ISIC is a basic tool for studying economic phenomena, fostering international comparability of data, providing guidance for the development of national classifications and for promoting the development of sound national statistical systems.
Classification rule The classification is used to classify statistical units, such as establishments or enterprises, according to the economic activity in which they mainly engage. At each level of ISIC, each statistical unit is assigned to one and only one ISIC code.
The set of statistical units that are classified into the same ISIC category is then often referred to as an industry, such as “the furniture industry”, which would refer to all units classified in ISIC division 31 (Manufacture of furniture)
This standardized categorization or subdivision of the complete set of producing units in an economy makes ISIC an important tool for socio-economic statistics that need to be arranged in accordance with the productive system of the economy.
The individual categories of ISIC have been aggregated into the following 21 sections:
A
01–03
Agriculture, forestry and fishing
B
05–09
Mining and quarrying
C
10–33
Manufacturing