A high-quality data stream that researchers had tapped for the purposes of an academic study showed promise in answering day-to-day questions for the Excise and Taxation Department in Punjab province, Pakistan. In a pilot project, Evidence for Policy Design (EPoD) and the Center for Economic Research in Pakistan (CERP) built a channel for the data’s continued use and demonstrated its benefits. This case study tells how the project produced gains in unexpected areas, as senior bureaucrats became sophisticated consumers of evidence and began shopping for it outside the research community.
In the energy industry, when you strike oil you also invariably tap other valuable resources; the question is whether you put them to use. Owners of oil wells used to ‘flare’ the natural gas that exists alongside oil in underground reserves. It was less valuable and portable than the liquid petroleum they were after, so they would burn it off in a continual flame at the top of a flare tower. Today, however, if a market for natural gas exists near the oil well, the owner pipes the gas to it. This represents a benefit not only for the oil producer, but also for the local energy customer and the environment.
Similarly, researchers conducting studies often create products that have value outside the academic sphere – especially if they are conducting data-intensive work in data-poor environments. The path to data’s continued use, however, is rarely clear.
A case in point is a large-scale experiment conducted in 2011–13 by Asim Khwaja, of Harvard Kennedy School, Adnan Khan of London School of Economics, and Ben Olken of MIT, in conjunction with the Excise and Taxation (E&T) department of Punjab, Pakistan. The team wanted to determine whether a pay-for-performance scheme would increase revenues. They found that it could – substantially and cost-effectively. Areas where tax inspectors were offered rewards saw a 46–64 percent higher growth rate in revenue, depending on the incentive structure. However, in a puzzling turn, most property owners in incentivized areas didn’t pay higher taxes, but instead reported paying higher bribes to tax authorities. Apparently, giving tax inspectors this new option – to receive a bonus for acting legitimately and reporting a property’s true value – granted them more bargaining power in their relationship with owners. It pushed up the price owners had to pay for inspectors to keep the property value low or to “overlook” the building altogether.
In order to conduct the study, the research team digitized ten years’ worth of paper-based tax records at the level of the tax circle, the unit below district level at which Pakistan’s property tax system is organized. Tax circles contain two to ten thousand properties delineated by streets or landmarks. Each is overseen by one tax inspector. The researchers realized early on that there was no hope in reliably measuring tax inspector performance, or fairly applying incentives, or really finding any results at all if a key piece of data – income generated by the tax circle – existed only on paper or in the informal knowledge of the inspectors themselves. So they conducted the massive exercise of digitizing the information.
In 2014 the study was complete, and the data was sitting there like an expensive power tool left out at a construction site: highly valuable if used and maintained, but bound to become worthless if abandoned.
The researchers and their policy partners could see that, if kept current and put in an easily accessible form, this data could help senior officials at the E&T Department make management decisions – ones that, at that time, were based on hunches and imperfect observations of how their tax inspectors performed. However, research funding only covered knowledge generation – it did not extend to adapting research byproducts for day-to-day use by local actors.
In most cases, that would have been the end of the story. The data would have been flared, so to speak, as often happens after the research findings have been extracted. However, in this case the Building Capacity to Use Research Evidence (BCURE) initiative funded by UK Aid created a pipeline for its continued use. In this BCURE pilot project, researchers worked with technicians at the E&T Department to create a mapping system that would allow managers to track income generated by tax circles. Armed with this high-quality evidence, the Department would be able to make better staffing decisions and give a much-needed boost to revenues.
What follows is a description of the project, but a warning to the reader: it doesn’t end with researchers handing over the completed tool to policymakers. Rather, the project’s successes came in a very different form. The collaboration changed what the policymakers demanded from their own department and from the organizations that would be able to deliver technical support sustainably, long after Harvard researchers and foreign funding were gone. This case speaks to one of the most important questions that has surfaced over the three-year course of the BCURE programme: once demand for high-quality data has been created in government ministries, who should deliver it on an ongoing basis? It also touches on issues that become important as countries like Pakistan grow from low-income to larger economies: how to harness that growth so the poor share in its benefits.
In collaborating with Punjab’s E&T Department, Khwaja and his colleagues focused on a relationship with high potential to make a difference in many lives. There are several reasons for this, not all of which are obvious.
First, Punjab is Pakistan’s largest province, with a population greater than any country on the African continent, and its most industrialized. The province was the engine for Pakistan’s growth, yet the benefits were slow in reaching its citizens. While Pakistan’s GDP grew at a rate of 2 to 5 percent in 2009 to 2014, its rating on the UN’s Human Development Index almost stagnated during that time.
Second, the nation of Pakistan struggled to raise taxes, even as it graduated from low- to lower-middle income status. A 2006 World Bank Report on taxation in Punjab specifically found that,
"While the property tax accounts for at least half – although often significantly more – of total local revenues in many countries, it represents less than 10% of [local] revenues in the Punjab. To carry this comparison further, property taxes represent an average of 1.42% of GDP in industrialized countries, 0.42% in twenty developing countries, and 0.54% in transitional countries; yet, they account for a fairly insignificant share of GDP, estimated at 0.09%, in the Punjab."
The report went on to give strong recommendations to broaden the tax base and strengthen property tax collection capabilities.
Third, researchers have long observed that rich countries raise more tax revenue as a share of national income than poor countries by roughly twice the proportion: a higher proportion of a higher national income. The reasons for this are myriad and range from the fiscal to the cultural: a higher share of poor countries’ economic activity is off the books; overstretched tax departments in low-income countries have limited statistical and data-gathering capabilities; and elites’ capture of power structures can lead to weak enforcement of tax codes.
It is important to note that we are speaking of a correlation here. There is a strong link between higher tax revenues and prosperity as both co-evolve over time; yet taxation rate is not a simple lever you can pull to force a country to develop. Still, it is beyond question that countries like the UK can afford services that benefit the poor – clean water, quality sanitation and public transportation, nationalised healthcare, and effective social protection programmes – in part because they collect over 25 percent of GDP in taxes, while Pakistan struggles to provide modest public services on 10 percent of a much lower GDP.
When countries depend on a narrow tax base, they are over-reliant on taxpayers within that base. In a context where so many citizens are making their living off the books (and off the tax registers), those in the formal economy can justifiably feel put-upon, which reduces their incentive to pay taxes. This sets into motion a vicious cycle, with low tax revenues financing low levels of public services, which in turn further reduces citizens’ willingness to pay. Meanwhile, tax departments are caught in their own cycle of lacking the resources needed to collect resources.
Pakistan – and Punjab in particular – needed ways to help tax revenues catch up with growth. The data visualization project – and the line of research from which it sprung – sought to give Punjab’s E&T Department new ways to document the economy and bring more activity into the formal sector. As with other BCURE pilot projects that dealt with a massive workfare programme in India and public health system in Lahore, a guiding insight was that new technology could improve information flows within government and fast-track improvements that would lead to better service delivery.
Punjab’s property tax inspectors received low pay and faced little threat of punishment for wrongdoing – an atmosphere that led to the widespread collusion that Khwaja and co-authors had measured in their study and which is common across developing countries.
Better monitoring promised not only to sanction poor performance and reduce corruption, but also to increase the potential of even high-performing tax inspectors. As we have described in another BCURE case, pay-for-performance (P4P) schemes can rapidly improve public service delivery, and indeed, the study that Khwaja and co-authors had just completed showed that such a programme worked in this context. Quality data flows like the one the visualization project created are essential for such incentive systems to work; otherwise, employees could claim that they weren’t fair. But even on a simpler level, quality data could help managers with their day-to-day decisions about how to motivate their staff. Processes that were based on hunches, observations, recommendations from lower-tier supervisors, or information provided by the inspectors themselves would now also be informed by data, which is much harder to falsify or misinterpret.
Adnan Khan of the London School of Economics, one of the project’s lead investigators, describes the difficulty of working in departments with low levels of technological expertise with systems based on colonial administrative structures. He says a central question to the project was, “How do we bring 21st century thinking and technology to a 19th century system?”
The researchers had digitized paper records for a ten-year window, but the essence of the tool was that it would provide current information. So a first step to the BCURE pilot project was to standardize the digitization process in the department. This meant training staff to change their practice and include computer-based methods, then dealing with some of the resistance.
The team then digitized several dimensions of tax-circle data that would be key to measuring the performance of the tax inspectors. Then they matched the data with GIS maps, a process which entailed sitting down with tax inspectors, determining the actual boundaries of tax circles, and spot-checking the results with on-the-ground surveying methods. They then constructed heat maps and put them on an online portal, where managers could analyse long-term trends, as well as dips and spikes in performance.
The mapping aspect was particularly important in this project, because urban Punjab is in a period of rapid population growth, and the opportunities for collusion were multiplying as a result. When a new property appeared, tax inspectors could record it or perhaps take a side-payment from the owner in return for allowing it to remain invisible to the E&T Department. Khan sees continued digitization as a way of gradually shutting down such opportunities. “Once a property is discovered it cannot be undiscovered,” he says. “It’s not just giving you value for one year, but for the future too.”
The construction of the portal took a year and a half, during which tax circle boundaries needed to be adjusted twice, as they change year by year. The portal was launched in the Department in September 2014.
As Project Manager for BCURE Pilots, Adeel Shafqat of CERP spent a lot of time in the E&T Department both before and after the launch. He observed how members of the Department used the new tool:
"It was something they could use for analytics, for measuring performances, for looking at departures from trends so that they could then request follow-up with the relevant person. We also had HR data on the portal. That way they could just look at which circle was underperforming and see the name of the person who was posted there, and immediately call the guy over and have a discussion with him."
Reports that previously would have taken several days of painstaking effort were now generated with a few clicks of a mouse. Traffic on the portal confirmed that the E&T Department made use of it frequently.
However, the E&T Department also had complaints about the system: that updates were made based on quarterly (not real-time) information, and that the data was at the level of the tax circle and not at the finer level of individual properties. “This was something that the Department grew sophisticated enough to demand from us,” Shafqat says.
Such requests were beyond the scope that the BCURE program could deliver in the context of the pilot project. But that doesn’t mean that they were outside the realm of what could be done.
As the team worked on the tax visualization project, they were aware that a parallel effort had been underway for a number of years.
The Urban Unit is a semi-public body established in 2006 to provide a number of services across the government of Punjab – mainly technical assistance that would ease the ill effects of the province’s rapid urbanization and help realize the economic growth that it made possible. It is a well-funded organization with a large staff and access to the latest technological tools. For eight years, the Urban Unit had been digitizing tax data at the level of individual properties, but this project had slowed as it encountered some of the same obstacles as the BCURE team witnessed: departmental inertia and hesitancy at the lowest level to learn the new technology and automate processes.
The Urban Unit’s project promised some of the higher functionality that Department officials were demanding because it operated at the individual-unit level. However, it lacked a mapping feature – initially, at least.
As the E&T Department used their tax visualization tool, their requirements changed. Shafqat says, “Both the Secretary and the Director General were keen to negotiate with the Urban Unit in a more technical way, so that they would get something more useful out of that project as well.” They requested that the Urban Unit incorporate GIS maps and a higher level of analytic capability into their portal, and asked the BCURE team to provide guidance – both with their requests and ultimately to Urban Unit technicians in fulfilling them. Shafqat and the team were more than happy to provide this service, which was outside their Memorandum of Understanding with the Department. “We wanted them to use these systems,” he says. “It was less important that they used our system – that served as a demonstration.”
Furthermore, it appears that the researchers’ presence might have lit a fire under a project that had gone cold. In 2015 the Urban Unit completed digitization and launched a new monitoring system. It is likely that the BCURE pilot project served as a catalyst, not only in raising expectations of what could be delivered, but also contributing to an atmosphere where it would get done.
Across the BCURE agenda, ultimately it will be up to government bodies or the organizations that they work with to carry the work forward. In this case, the Urban Unit must play a role, since the E&T Department has neither the staffing level nor the technical expertise to maintain the data channel. The BCURE pilot project highlights an unexpected role that researchers can play: a third party who can drive better performance by showing what is possible through better leveraging available data and by enabling progress through collaboration.
Dr. Khan served many years in Pakistan’s civil service before moving into academia. He says the full meaning behind the tax visualization project – and its applicability outside property tax – become apparent when viewed through a theoretical lens, “not as just a technological shift, but a market shift.”
In a seminal 1970 paper, economist George Akerlof used the market for used cars in California to show how problems can arise when one party, the principal, who has incomplete information, delegates a task to another, the agent, who has complete information. There are many situations where conflicts of interest can lead agents to take advantage of this information asymmetry, to obscure fact and engage in opportunistic behaviour. Akerlof showed that this resulted in adverse selection in the used car market where buyers suffer and sellers who are offloading “lemons” have an advantage. The same problem can lead to inefficient outcomes in bureaucratic functioning. Just as the buyer does not know if the car is normal or a lemon, government (as principal) often does not know if the bureaucrats it recruits to perform various functions (agents) exert effort or shirk their responsibilities, and whether they engage in corruption and other undesirable behaviours.
Incomplete and asymmetric information play a big role in the relationship between senior and frontline workers, in government and the private sector. Without adequate information, supervisors (principals) cannot evaluate the performance of their staff (agents), design rewards for behaviour that benefits them and punishment for behaviour that doesn’t, or identify the sources of inefficiency. Using information technology to open up a data flow can help address these challenges and enable the design of more efficient contracts between supervisors and employees. These can have a near-immediate benefit for principals, similar to those we saw when price-comparison websites swiftly brought down the costs of a number of goods whose prices had been inflated via information asymmetries.
Khan understands the problems around taxation in Pakistan in principal-agent terms. This is clear in the relationship between managers and tax inspectors: quality data can solve the performance-related information asymmetry and allow managers to use incentives to make inspectors work harder and cleaner to raise revenues. But Khan also sees asymmetries between the citizen (as principal) and the government (as agent). Above, we mentioned how distrust can arise in citizens who receive a low level of services for the amount the government charges in tax. A next step for the research team is see if showing taxpayers how their money is spent – and giving them more of a say in the process – will increase their willingness to pay taxes.
“Government should not be a black box,” Khan says. “A learning and experimental government that collaborates with researchers and makes smart use of evidence can go a long way in delivering services to citizens and becoming more answerable.” ■
Text by V. McIntyre. Design by Gabrielle Hauray.