Data Discrepancies between National Surveys and Official Reporting

The growing discrepancies between the survey and official data have become a serious issue in recent years. This has allowed researchers, administrators, policymakers, and individuals to take opposite positions on policies, programme or even the political regimes. It is in such a context that this article looks at some of the data anomalies in institutionalised surveys and official sources.

One of the strong points of the Indian statistical system is the availability of information from multiple sources on important key socio-economic indicators. Information from independent sample surveys, carried out through an established institutional system, has often been used to cross-validate statistics, available from official/departmental sources and major government programmes. In the past, data from the National Sample Surveys (NSS) were used to cross-check the data collected from the agricultural census, livestock census, etc. Deficiencies in civil registration were one of the reasons for initiating the Sample Registration System in the 1970s, which is still continuing. In recent years, in most government programmes, data is mandated to be placed in public domain through online reporting as part of the real-time management information systems. Such information can be accessed through dashboards on the websites of the different departments. The progress of major national programmes is also widely published by the official agencies unlike in the past where such progress used to be reported only in their annual reports. Understandably, there are differences in the magnitude and even the direction of change in the information from government departments and those from established institutional surveys.

The issue of data mismatches between different sources is not new. When the agricultural statistical system was evolving after independence, the Indian Statistical Institute (ISI) played a pioneering role in developing scientific crop-cutting experiments. However, the parallel estimates, one from official agencies and the other based on the ISI methodology, did create problems. Consequently, the ISI-based estimation was dispensed with but an independent scheme for improvement in the official crop statistics got introduced, whereby an external checking of the procedure was established. The estimates on the number of operational holdings from the agricultural census and that from the NSS could never be reconciled. The same is the case of balance of payment’s statistics from the Reserve Bank of India and that from the Ministry of Foreign Trade. The literacy rates from the Census of India were always a shade higher than the NSS-derived rates. Similarly, the enrolment rates reported by the education ministry and the National Education Survey of the National Council of Educational Research and Training (NCERT), during the first decade of the present century, were at variance with the difference increasing over time, although both used the same machinery and collected information from the same schools. The path-breaking cross-validation studies done by B S Minhas on estimates of household expenditure from the NSS and that on private consumption from the national accounts are well known. This particular study did much to help understand the strength of the survey-based estimates.