Worker Evaluations of Managers and Productivity in China
Factories in high-income countries are generally more productive than factories in low- and middle-income countries. One key driver of this production disparity may be low management quality. Researchers randomly assigned automobile manufacturing workers to provide monthly feedback on their managers to measure the impact of feedback on worker productivity, turnover, and job satisfaction in China. Team-level productivity was higher, turnover was lower, and workers’ happiness increased when workers could provide feedback on their managers.
Policy issue
Factories in high-income countries are generally more productive than factories in low- and middle-income countries. Evidence from India shows that one key driver of this productivity disparity may be lower management quality in factories in low- and middle-income countries. There are many possible strategies to improve management quality, including financial incentives, training, feedback, and consulting. In particular, feedback may be an important cost-effect method of improving management. Yet, factories can record team-level productivity much more easily than they measure management quality. This may lead factory leadership to assume, for example, that a manager is effective when, in fact, the manager is ineffective but their workers are effective. Can workers’ feedback on managers be used to improve management performance and worker satisfaction?
Context of the evaluation
At the time of this study in 2016, the Chinese automobile manufacturing industry was growing rapidly. Average sales growth was five percent, and average export growth was 30 percent. Many Chinese manufacturers face problems hiring and retaining workers, including the study company where average annual worker turnover was greater than 20 percent.
The researchers partnered with a large automobile manufacturing company. This study took place at one of the company’s plants that produces passenger vehicles. The plant is composed of four factories, each responsible for a different part of the production process.
Each team of production line workers in the plant has a manager. Managers are usually promoted from among the production line workers. Each day, production line teams have a meeting in which production line workers receive individual-level performance feedback from managers. Managers receive monthly team-level performance evaluations based on productivity. Feedback is posted publicly on a bulletin accessible to the whole team.
Both managers and workers receive a flat salary and a bonus. For production line workers, the flat salary accounts for 90 percent of their pay. The remaining ten percent bonus depends on both company-level and individual-level performance. For managers, the flat salary accounts for 80 percent of their pay. The remaining 20 percent bonus depends on company- and team-level performance. Semi-annual and annual promotions and pay raises also depend on these performance metrics.
On average, workers are in their early thirties and have worked at the factory for more than four years. Ninety percent are men and about half of workers have at least a high school degree. Average monthly earnings were 3,700 RMB (US$500), excluding bonus.
Details of the intervention
From 2016-2017, researchers partnered with the automobile manufacturing company to test the impact of upward feedback on managers on worker performance and wellbeing. The study took place in three of the manufacturing plant’s four factories and included 1,250 workers across 76 production teams.
The research team randomly and evenly divided production teams into two groups:
- Monthly manager feedback. Across eight months, visited the plant to gather workers’ evaluations of their managers during monthly production team meetings. Production line workers used anonymous paper forms to score their managers on a scale of one (not effective at all) to five (extremely effective) on five characteristics: organization, adaptability, fairness, openness to suggestions, and empathy. This monthly feedback had a direct impact on managers’ monthly bonuses for the first six months of the program, accounting for up to four percent of managers’ pay. For the last two months of the program, managers’ bonuses were calculated using the pre-program method, meaning that production line worker feedback had no impact on their pay. Aggregated feedback was also posted on the team’s public bulletin board.
- Comparison. As was normal for the plant, workers were not invited to submit feedback about their managers. The existing performance tracking processes outlined above remained in place.
To measure work performance and worker well-being, researchers used two administrative datasets from the company and collected individual survey data. The first administrative dataset covers individual-level production line worker performance. Workers are assigned an initial total performance score of 100 that is adjusted based on six key individual performance metrics: production, safety, quality, attitude, attendance, and routine. This score is updated based on daily feedback from managers to workers, but researchers had access to the data only at the month-level. The second administrative dataset covers team-level performance on key performance indicators (KPIs), including production, management, equipment, quality, and safety. Researchers also collected survey data on study participants individual well-being, mood, and work satisfaction prior to the intervention and shortly after it ended one year later.
Results and policy lessons
Allowing workers to evaluate their managers improved key outcomes for workers, managers, and teams. Workers experienced improved happiness and mood and were less likely to quit, managers performed better and were paid more, and teams performed better on their KPIs. These positive impacts seem to be driven by improved behavior among managers, who were less likely to criticize their workers or lack empathy toward them and more likely to encourage their workers.
Workers who could provide manager feedback were 6.2 percentage points less likely to quit relative to workers who could not provide feedback on their managers. There was no impact on manager turnover.
Team-level performance improved significantly despite no significant change in individual-level performance. Team-level attainment of KPIs increased by 2.3 percent relative to 97 percent KPI attainment in the comparison group, with significant improvements in the production and management subcategories. Improvement in team performance despite consistent individual-level performance can be explained by reduced worker turnover: Higher turnover limits team productivity because new workers are less productive and require more training.
Workers who could provide manager feedback reported higher life satisfaction and positive mood. On average, workers who could provide manager feedback reported being 0.3 units higher on a scale from one (lowest life satisfaction) to ten (highest life satisfaction). These workers did not experience improvements in their health and job satisfaction or reductions in negative moods. There are no significant changes in any of the measures of wellbeing for managers.
Managers’ improved treatment of their workers drove these positive outcomes. Workers who could provide manager feedback reported that their manager encouraged them more, criticized them less, and was less likely to lack empathy toward them. Managers who received worker feedback were also more likely to socialize with workers outside of work and respond to WeChat messages from their workers.
Managers’ monthly bonuses, which rely heavily on team-level productivity, increased 22.8 percent. Workers’ monthly bonuses, which reflect individual-level productivity, did not increase.
Improvements persisted both after the end of the financial incentives for managers and after the end of the formal system of worker feedback. Additionally, managers did not seem to know or place significant emphasis on the impact of worker evaluations on their monthly bonuses. New employees reported more encouragement and greater empathy from managers whose employees had previously been able to provide feedback, providing further evidence that effects persisted.
Given the positive impacts for both workers and managers, an open question remains as to why managers weren’t more encouraging and empathetic in the first place. Researchers note two potential mechanisms to be evaluated in future research—managers may not understand the importance of retention, or they may understand the importance of retention but not know how to improve it.
Cai, Jing, and Shing-Yi Wang. 2020. “Improving Management through Worker Evaluations: Evidence from Auto Manufacturing.” Quarterly Journal of Economics, revise and resubmit. (August 2020). https://www.nber.org/papers/w27680