A A A
Change Text Size

Status of Opinion Polls

Media Gimmick and Political Communication in India

BibtexBibtexEndnoteEndnoteRISRISGoogle ScholarGoogle ScholarPrintPrintEmailEmail

Election surveys are seen as covert instruments used by political parties for making seat predictions and influencing the electorate in India. It is high time opinion polls take cognizance of the situation to establish their credibility and impartiality. 

Praveen Rai (praveenrai@csds.in) is a political analyst at the Centre for the Study of Developing Societies, Delhi. 

The very mention of the word opinion Poll[i]” immediately brings to the mind of people in India election surveys, exit polls[ii] and seat predictions that appear in mass media every time an election takes place in the country. Psephology, the study of elections, began as an academic exercise at the Centre for the Study of Developing Studies (CSDS), Delhi in the 1960s for the purpose of studying the voting behaviour and attitudes of the voters. Psephology is now equated with pre poll surveys and exit polls which are being rampantly done by media houses to predict the winners during the elections. It has now been reduced to a media gimmick with allegations that it is used as communication tool for influencing the voters by a conglomerate of political parties, media and business houses with vested interests. Media houses and television anchors in India have become the modern day “Nostradamus” in using opinion poll findings in forecasting election results, before the actual votes are cast, which have gone wrong on many occasions.

Accuracy of sample surveys depend on the following factors: one, the sample should be large enough to yield the desired level of precision. The size of the required sample for any survey can be statistically determined. Those who do not have the experience can use statistical tables that provide various sample sizes based on the population size of the universe. However in some cases, the sample size depends upon the disaggregation level for which the data is required; two, everyone in the population should have an equal chance of being selected in the sample. Probability sampling based on random method is the best way for ensuring that everyone in the universe stands an equal chance of getting selected; three, survey questions should be asked of the sampled respondents in a standardised manner. (Standardisation ensures that questions are asked in the same manner to all the sampled respondents as that will enable the respondents to respond accurately) and four, there should not be any pre-determined arbitrariness in interviewing the sampled respondents.

An accurate survey should follow some basic norms:

‒Every member of the targeted population should have an equal chance of being selected for the survey. Probability sampling ensures everyone a fair and equal chance of getting selected which results in avoiding coverage error.

‒The size of the sample to be selected should be adequate enough to achieve the required level of precision. The attempt should be to minimise sampling error.

‒The questions to be asked should be simple and clearly worded so that the respondents can understand it and answer it easily. The question to be asked should be worded in such a manner that it stimulates the respondents to answer them correctly. This reduces the measurement error though it cannot be totally avoided.

‒The sampled respondents who could be contacted and interviewed during the survey should have similar in traits as those who could not be interviewed. Everyone in the sample who responds to the survey should have corresponding characteristics with those who did not respond. This helps in avoiding non-response error.

Opinion Polls :A History

The popular media surveys started in the 1980s when media baron Prannoy Roy conducted opinion polls during elections to find out the mood of Indian voters. The proliferation of electronic media in the 1990s made the election surveys and exit polls popular in India, and it started capturing the imagination of the people. Pre-election surveys and exit polls have become a regular feature in the last one and half decades.

At the very beginning, most of the poll results were published only in news magazines like India Today, Outlook and Frontline. Slowly and gradually, the leading newspaper groups also started showing interest in publishing results of election surveys. The demand from the print media further increased the number of opinion polls being conducted in the country. What added to this growing demand of surveys was the advent of various television channels. With a large number of news channels competing against each other, the race for conducting elections surveys and airing them as quickly as possible after election dates are announced has become the norm of the day in India.

While the election polls are of different kinds, it is the pre-poll and the exit poll, which catches the attention of most of the people. The reason is simple ‒ people are eager to know which party or alliance is likely to win the elections, and how many seats they will win. Exit polls became very popular in the year 1996, when Doordarshan, the government owned television channel, commissioned an all India exit poll. The fieldwork and data collection for this poll was done by the team at CSDS, and its findings were reported and discussed in  a five hour programme aired live on Doordarshan. Since then, there has been no election in India when the exit poll results have not been televised  the day polling gets over.

A Mixed Baggage

The history of opinion polls and seat predictions during the last four general elections has been a mixed baggage of success as well as failure. Seat predictions based on election surveys done by various media houses during general elections held in 1998 and 1999 were fairly accurate, and it gave a big boost to opinion polling in India. Table 1 and 2 shows  seats predictions by the various surveys and the level of accuracy vis-vis the actual results.

 

Table 1: General Elections 1998 – Almost accurate forecast

Seat Forecast

BJP allies

Congress allies

NF+LF

Others

DRS

249

155

102

37

Outlook/AC Nielsen

238

149

123

33

India Today/CSDS

214

164

127

38

Frontline/CMS

225-235

145-155

120-130

32-52

Actual result

252

166

96

23

Note: The seat predictions were made on opinion polls conducted before the elections.

 

 

Table 2: General Elections 1999- NDA slightly overestimated

Seat Forecast

BJP allies

Congress allies

Others

Timespoll/DRS

332

138

 -

Outlook/CMS

319-329

135-145

34-39

India Today/Insight

332-336

132-146

 70-80

HT-AC Nielsen

300

146

95

Pioneer-RDI

313-318

140-150

84-86

Actual result

296

134

113

Note: The seat predictions were made on opinion polls conducted before the elections

 

During the Lok Sabha elections held in 2004, all the polls conducted by media houses predicted that the National Democratic Alliance (NDA) led by Bharatiya Janata Party (BJP) would be able to retain power at the centre. The only difference between different polls was that while some suggested that the NDA would come back with an increased tally, others predicted loss of some seats for the NDA alliance. Among the polling agencies and pundits there was a complete unanimity in their predictions that NDA would win the elections. But the result was a shocker -  NDA lost the elections and the United Progressive Alliance (UPA) led by Congress came back to power. Thus poll predictions based on election surveys during the 2004 elections (as seen in Table 3) went completely haywire for most of the polling agencies and pollsters.  

 

Table 3: General Elections 2004 - UPA completely underestimated

Seat Forecast

BJP allies

Congress allies

Others

NDTV-Indian Express

230-250

190-205

100-120

Aaj Tak-ORG Marg

248

190

105

Zee-Taleem

249

176

117

Star-C-Voter

263-275

174-186

86-98

Sahara-DRS

263-278

171-181

92-102

Outlook-MDRA

280-29

159-169

89-99

Actual result

189

222

132

Note: The seat predictions were made on opinion poll/exit polls conducted during the elections

 

With such a failure of pre-polls and exit polls, questions were raised about the failure to predict NDA’s defeat during the 2004 Lok Sabha elections. Were the polls biased, incorrectly done or was there any political interference in showing the results of such polls? Was the sample size of the survey too small for forecasting or was the methodology wrong? Why do we need to conduct election surveys when the predictions made by all polling agencies go wrong? Thus a lot of misconceptions and misgivings developed about the polling industry, with most people regarding it not as a serious discipline but an instrument of political communication. As a result people  believed that it was done with a hidden agenda for benefiting some political party or a lobby.

 

Table 4: General Elections 2009 - Predictions failed to see the Congress upsurge

Seat Forecast

BJP allies

Congress allies

Others

STAR News - AC Nielson

197

199

136

CNN-IBN

165-185

185-205

165-195

NDTV

177

216

150

Headlines Today

180

191

172

News X

199

191

152

Times Now

183

198

162

Actual result

159

262

79

Note: The seat predictions were made on opinion poll/exit poll conducted during the elections

Five years down the line, the poll predictions made by different media houses and poll pundits during the 2009 general elections once again failed to predict the victory of Congress led UPA.  Except for the poll conducted by the Congress themselves that predicted it will cross the 200 mark, everybody failed to predict the upsurge in favour of the Congress. The questions that arose after Lok Sabha elections 2004 once again resurfaced after this election.

The opinion polling industry has failed to learn from its past mistakes, and the recent spurt in opinion polls giving a clear lead to the BJP led NDA once again brings forth the dangers fraught in the misuse of opinion polls. Election surveys are now perceived as covert instruments used by media houses in collusion with political parties for falsely predicting their fortunes with the aim of influencing the electorate in India.

Opinion polls: The Methodology

Opinion poll as a tool of research and poll results, especially in Indian conditions, is fallible. Election surveys are the best means of deriving a more accurate picture of the voting intentions and political attitudes of the Indian electorate. (Butler, Lahiri & Roy:1989). Though election opinion polls are fallible in India, exit polls and post-poll surveys[iii] could be relatively more accurate because of their timing; they gauge the voting intention of voters after they have already voted.

Psephologist Yogendra Yadav says that in addition to timing, other crucial factors, including sample size, sample design and the representativeness of the sample, ensure accuracy. (The Tribune: 1999). Choice of the survey method, sampling method and the level of training of enumerators who collect data determine the accuracy of the survey conducted.

The choice of survey method is important for election studies to determine whether it is a good survey or a bad survey. Election surveys, whether pre poll or post poll, are based  on a random sample drawn from the voter list of the Election Commission of India are generally accurate, as they  yield a representative sample ruling out coverage errors and minimising sampling errors.

On the other hand, exit polls,  done on the day of elections, are based on quota sampling and are fraught with the risk of leaving out some sections/subsamples of the population completely. Thus the sample frame of exit polls is in most cases unrepresentative and suffers from both coverage and sampling errors.

The adequate number of respondents that should be interviewed in an election survey is statistically determined, and the size of sample does not to a great extent determine its quality and credibility. The election surveys in India do not go wrong as as most of the polls have big samples.  However, even if the sample size is big it has been observed that choice of sampling methods have been unscientific to statistically cover the universe of the study. As a result the surveys are not representative and the seat predictions based on it goes wrong on several ocassions.

Random sampling methods which are popularly used for pre-poll and post-poll election surveys have a greater chance of getting a representative sample as compared to a sample drawn purposively. Purposive or quota sampling of voters is popularly used by market research agencies for exit poll where a quota is fixed for sampling the respondents based on gender, education, caste communities,  different age groups, occupational background and class.

What accounts for such unrepresentative sample selections? India is a country with 70 % people living in rural areas. Effectively that means that out of every 100 Indian voters, 70 live in villages while only 30 live in towns and cities. Any election survey in order to be representative, should try to conduct about 70 % interviews in the villages and rural areas and only about 30 % interviews in the towns and cities.

But this is not followed by most of the market research agencies that are engaged in election polling in India. The reasons for this anomaly can be attributed to inconvenience and high costs of conducting surveys in rural areas. Most of the polls suffer from an urban bias, which ultimately results in making the sample unrepresentative. A sample with more interviews in towns and cities is most likely to have more educated, rich and middle class respondents. Thus the whole sample becomes biased and skewed, making  it completely unrepresentative and rendering the survey inaccurate.

The third important component of an election survey, crucial for conducting it accurately, is the training of enumerators who conduct field investigations. Unlike the United States and other European countries where election survey is conducted telephonically, in India an election survey is conducted by enumerators who contact the respondents and conduct interviews in person. So for every opinion poll or election survey, training of enumerators a few days before field investigations is imperative so as to ensure standardisation.

However except for a few academic institutions, most of the market research organisations do not spend time and invest financial resources on fieldwork training and practices. Instead they pick one time-trained enumerator from their pool and ask them to do the fieldwork for subsequent round of election surveys. But there are certain aspects of field investigators training ‒ rapport building with sampled respondents, reading out questions from interview schedules and using survey instruments, following accepted fieldwork procedure and practices, standardised method of asking questions and recording responses and do and dont’s of survey ‒ that need to be done afresh for each round of election survey. The absence of rigorous training leads to inaccuracies in data collection that sometimes lead to the failure of surveys and any data analysis based on it becomes fallible and questionable.  

Factors that propel inaccurate poll predictions

The reasons for election predictions going wrong would be analysed at four levels:

First ‒ the socio-cultural diversity and volatility of India voters. Do voters form their voting decisions after the elections are declared or are there a significant number of  voters who are floating voters and decide at the last moment? Do the Indian voters reveal their voting intentions to the surveyors or do they conceal them arising out of extraneous factors?

Second ‒ whether the survey can capture the complexities of elections arising out of multipolarity of contests, party alliances and transfer of votes for each other and geographical concentration of votes for some parties in some regions/states? Similarly Indian elections witness factionalism in parties, rebel candidates and local level settings that is difficult to ascertain by a survey;

Third whether the sample selection is representative of the demographic of the voters and does size of the sample determine the accuracy of the survey?

Fourth ‒ how far are the various statistical models for seat predictions developed by pollsters in making an accurate election forecasting?

Socio-cultural diversity and volatility:  Election studies conducted in the past have indicated that Indian voters are highly heterogeneous, with different socio-cultural practices and demographic backgrounds, and their voting patterns and preferences are varied. But at times, the multiple identities of Indian voters based on regional, caste community, linguistic and religious identity overlap, making it difficult to ascertain  their political affiliation and electoral choices.

For example, Muslims in a state like Uttar Pradesh (UP) do not vote for the BJP as reported by election surveys conducted in the state in the last decade. However, there are variations in voting patterns of the Muslims residing in different regions of the state. Muslims in UP do not form a heterogeneous group and there are differences among them based on region, language and religious sect, which are reflected in their voting preferences. Thus a large sample survey would capture that certain sections of Muslims in the state have voted for the BJP. Similarly, the voting patterns of voters in India have revealed that they are  volatile. For example, certain issues in elections may have regional and sectional appeal and can form the basis of voting decisions. For others, the issues  might not have any appeal and may not affect their voting decisions.

Pre- poll surveys might make off the mark seat projections. Election studies, done before or during the election campaign, have revealed that around one-fourth of the voters do not decide beforehand  about who they are going to vote for and are termed as floating voters by pollsters. The floating voters make up their mind during election campaigns, and if there is a bandwagon effect in favour of any particular political party, the predictions can seriously go wrong.

Though pollsters use the rider while making election forecast based on pre-poll surveys that it can change in subsequent weeks, its end users remember the predictions and judge it right or wrong accordingly. During Uttar Pradesh Assembly election 2007, a post-poll survey revealed that a large number of voters from upper castes who actually voted for Bahujan Samaj Party (BSP), a dalit party, did not reveal their voting behaviour due to fear of caste-based violence.  As a result most of the election surveys conducted during this election failed to predict a clear cut win for the BSP led by Mayawati.

Similarly exit polls, in which the voters are interviewed outside the polling booth after they have voted, do not always reveal their correct voting intentions. This happens because correct revelation of party they voted for could be used by other political parties in identifying and targeting them individually.  

Multi-polar contest, factionalism etc. : It is easier to forecast election outcome in a state with two main political parties, if the survey gets their vote shares correctly. However in states where there are multipolar contests, making predictions become difficult, as a slight error in estimating vote share for any one party can completely upset the seat prediction.

Parties that contest elections in alliance also pose a problem in seat prediction, as it is very difficult to compute how the votes are shared between one another in an alliance. The geographical concentration of votes for some parties in some regions of the states also make it difficult to make correct seat prediction in spite of getting the vote estimates correctly. The result of Karnataka Assembly election held in 2008 revealed that BJP had one percent vote share less than Congress in the election but went on to capture the majority of  seats in the election. Congress party got more votes than BJP in the state, but its vote was evenly distributed throughout the state which lead to huge loses in winning seats. 

Similarly, Indian elections witness factionalism in parties, and the effect of this on the electoral outcome cannot be captured by an election survey. The effect of rebel candidates and micro level dynamics of elections also cannot be ascertained by a sample survey arising out of limitations of its sample size and huge costs involved.

Method of Sampling and Size of Sample:The most significant aspect for any opinion poll surveys is about sample size and method of sampling. The sample size for any national and state level election study depends upon the level of analysis one intends to do. Thus if one wants to analyse the voting behaviour and attitudes of voters only at the state level, a survey of 1500 respondents would be enough. But if one also wants to do a region wise analysis in the state, then the sample size should be bigger as there should be sufficient number of cases for disaggregate analysis.

Thus sample sizes for any election survey depends upon the level of disaggregate data one requires for analysis.  Seat predictions based on even a small sample size, if it is representative can get the correct results. On the contrary, a survey based on a big sample size if it is unrepresentative will yield wrong seat forecast. Yogendra Yadav says that there is no guarantee that a bigger sample size will get you the right result. Bigger surveys only multiply errors ten times. (The Indian Express: 2008)

The method of sampling used in election survey and its accuracy also plays an important role in making a fair election forecasting. Thus a smaller representative sample can help make an accurate predication as compared to a bigger unrepresentative sample. The method of sampling used by surveys done by market research organisations is mostly not the most scientific method of survey research.

This becomes clear in the voting preferences and intentions of caste communities. For example, in the state of UP, majority of dalits, especially the Jatavas, have been voting for the BSP while the majority among the upper caste voters have been supporters of the BJP and Congress. Similarly the voters belonging to the Yadavs have been staunch supporters of the Samajwadi Party. Thus the sample in UP should be representative of these caste and communities approximating the percentage of the population of these communities in the state.

So if a sample survey fails to gather the opinion of any important caste and community, the election predictions based on it will be highly susceptible to failure.  Political analyst Yogendra Yadav says  the method through which the sample is selected is crucial for the survey. Most Indian polls go wrong because their sampling methodology is poor which makes the sample profile unrepresentative (The Indian Express :2008). Though a scientific and representative sample determines the accuracy of the survey, it is not a guarantee that a forecast based on it would be always right. The reason is that survey has its limitations, as it cannot capture the diverse and nuanced complexities and undercurrents of electoral behaviour and choices in India.

Conclusion

Contrary to their foreign counterparts, media opinion polls on elections in India have focused more on predicting the number of seats major political parties are going to win or lose in the elections rather than understanding the key issues facing the electorate. A recent sting operation on polling agencies have also revealed that seat prediction figures are manipulated in favour of their clients. 

Thus election surveys have been reduced to a media gimmick of only predicting the election results that are quite often wrong or off the mark. As a result election surveys are seen as covert instruments used by political parties for making seat predictions and influencing the electorate in India. Therefore, the Election Commission should ban forecasting before the elections but not the opinion polls.



Notes:

 

[i] Post Poll survey is an indigenous method of survey for measuring voting behaviour developed in India. Post Poll survey is a unique method of conducting election survey which was pioneered by CSDS in the 1960s wherein the voters were interviewed after the polling was complete in the relaxed confines of their homes. The post poll survey for measuring voting behavior is purely an academic exercise done with the purpose of doing the post mortem analysis of the elections. (Kumar & Rai: 2003).

 

[ii] Exit Poll as its name overtly suggests is an election survey, which is conducted among voters as they come out or exit from the polling station after casting their vote on the polling day. This survey is also known as Election Day polling as the survey is conducted and completed on the day of polling. (Kumar & Rai: 2003).

 

[iii] Pre poll survey as the name indicates is an election survey that is conducted before the elections are held for measuring popular choices about political parties, contesting candidates and political leaders. It also helps in measuring voting behaviour amongst different section of voters. (Kumar & Rai: 2003).

 

References: 

Butler, David, Lahiri, Ashok, Roy, Prannoy (1995): India Decides. Elections 1952-1995, Delhi: Books & Things.
 
Kumar, Sanjay & Rai, Praveen (2013): Measuring Voting Behaviour in India, Delhi: Sage Publications.
 
Nagaraj, Anuradha (2008): “‘Psephology is not a science like microbiology... it’s poll studies. But everyone thinks only of seat forecasts’”, The Indian Express, 27 January, available at http://archive.indianexpress.com/news/-psephology-is-not-a-science-like-..., accessed on 11 April, 2014. 
 
Nath, Tripti and Suryamurthy R. (1999): “Poll Pundits or Punters?”, The Tribune, 28 August, available at http://www.tribuneindia.com/1999/99aug28/saturday/head1.htm, accessed on 11 April, 2014. 
 
Rao, Bhaskara (2009): A Handbook of Poll Surveys in Media – An Indian Perspective, Delhi: Gyan Publications.
 
Yadav, Yogendra (2008): ‘Whither Survey Research? Reflections on the State of Survey Research on Politics in Most of the World’, Malcolm Adiseshiah Memorial Lecture, Chennai.

 

Comments

EPW looks forward to your comments. Please note that comments are moderated as per our comments policy. They may take some time to appear. comment, if suitable, may be selected for publication in the Letters pages of EPW.