Monitoring Harassment in Organizations

Partnering with a large garment manufacturer in Bangladesh, researchers from Columbia University, Princeton University, Ben Gurion University, and the University of Washington tested different methods of surveying employees about workplace harassment (PDF) and found that secure survey designs that ensure the plausible deniability of responses to sensitive questions can help uncover harassment that would otherwise go unreported.

The findings from the survey are especially relevant for factory workers in developing countries, but the methods developed by the researchers can be used in organizations much more generally.

At the Bangladeshi apparel firm, specifically, automatically garbling survey responses – what researchers refer to as  “hard garbling’’ – increased reporting of physical harassment by 288%, sexual harassment by 269%, and threatening behavior by 46%. The new method revealed that harassment is widespread, that the problem is not restricted to a minority of managers, and that victims are often isolated in different teams. Harassment against women by managers who are men is common, but harassment by men against subordinate men is also substantial. 

How hard garbling works

Existing evidence from similar contexts suggests that employees underreport harassment: many report having witnessed harassment, but almost no one reports being a victim. Key reasons for underreporting are fears of retaliation and reputational damages. The researchers seek to ease reporting by ensuring participants’ plausible deniability in the event they complain. 

To provide plausible deniability about reporting harassment, the researchers automatically record a random subset of reports as complaints (this is hard garbling). They can then apply statistical formulas to recover policy-relevant statistics of harassment from garbled reports. This includes the share of workers who have been harassed, how widespread harassment is across teams, and how isolated victims are.

Under hard garbling, reports of harassment are always recorded, but reports of no harassment are sometimes switched to reports of harassment. Garbling was explained to survey participants as follows: 

“We are now going to ask you several questions about the way your manager treats you and other employees. For instance: ‘Has your manager shouted at you in the last month? Yes or No?’ 

Each of the questions has a Yes or No answer. Our system is set up so that it’s safe to report an issue. If you choose to respond YES (there is an issue), our system will record it as a YES for sure. Importantly, if someone responds NO, the system will sometimes record the response as YES. 

This means that if you respond YES, we can guarantee that you won’t be the only person saying YES. For every 5 responses from workers, at least 1 will be recorded as YES.”

The methodology builds on decades of research that aims to develop new ways to elicit true responses to sensitive survey questions or to overcome the reality that collusion–for example working together to punish informants–corruption, and other forms of misbehavior among employees can prevent those conducting surveys from collecting accurate data on harassment, and other sensitive or controversial issues. 

Dating back to work published as early as 1965, social scientists have shown that the prevalence of sensitive behavior can be accurately estimated using garbled data. However, prior methods have focused on encouraging survey respondents to voluntarily randomize their answers, a method the authors call “soft garbling.” Although soft garbling can be helpful, recent evidence shows that respondents often decline to garble their own reports, and that the effectiveness of soft garbling in liberating voice can be short-lived in stable organizations. The hard garbling methodology developed in this paper addresses this issue by automating the addition of noise. In addition, tracking garbling frequencies greatly improves the accuracy of the estimates, which matters when dealing with fairly rare reports (as in the case of harassment).

Ultimately, the hard garbling method gives employees the ability, when fearing retaliation, to claim that they did not report harassment, while also helping organizations uncover the true scope of harassment occurring.

Testing three types of survey methods

The authors tested three types of phone-based survey methods, each of which carries a different cost that organizations should consider.

The first method, in which responses were hard garbled, was much more successful at uncovering the true scope of harassment, but comes at the cost of limiting the severity of organizations’ interventions following reports, since some innocent actors will be the target of a realized noisy complaint. This means that a manager could not be fired because of such a report, but an internal evaluation could be launched, or they could be assigned to additional training. Alternatively the worker associated with a recorded complaint could be reassigned to a different line.

The second method, referred to as rapport building, aims to build trust with the survey respondent by having the person conducting the survey engage the respondent in guided small-talk. It requires extra training for those conducting the survey and is more time-consuming to complete. 

The third method, in which the person conducting the survey collects less personally identifying information about the respondent, aims to reduce the fear that leaked responses could be traced back to the respondent. Ultimately, however, it provides organizations with less data about the nature of an organization’s harassment problem. In this implementation, the organization no longer learns the name of the manager responsible for the harassment.

Directly asking survey participants about sensitive questions was the control method. Researchers then randomly assigned participants to different combinations of treatment conditions, varying whether the survey method garbled respondents’ intended reports, the extent to which the surveyor built rapport with the surveyed individual, and the identifiability of a respondents’ team and manager.

Quantifying harassment in the garment industry

The experiment helped researchers understand the scope of harassment at large garment factories that, by many measures, are representative of other factories throughout the industry.

In the control group, 9.9% of workers reported experiencing threatening behavior, 1.52% reported being physically harassed, and 1.78% reported being sexually harassed by their supervisor. 

Hard garbling at times more than doubled the share of workers experiencing harassment, estimating that 13.6% experienced threatening behavior, 5.7% experienced physical harassment, and 7.7% experienced sexual harassment. 

They authors also found that:

• Removing questions about respondents’ supervisor increased the reporting of physical harassment by a marginally statistically significant degree, but had no detectable effect on the reporting of threatening behavior or sexual harassment.
• Building rapport had a positive, but not statistically significant, effect on the reporting of threatening behavior and sexual harassment, but no detectable effect on the reporting of physical harassment. 
• In the control group, women were less than half as likely as men to report threatening behavior or physical harassment, but slightly more likely to report sexual harassment.
• Hard garbling significantly increased rates of reporting for women. Perhaps surprisingly, the effects were the same or larger for men. 
• There is some evidence that while building rapport with respondents increased reporting among women it may have backfired for men. The authors hypothesize that this may be because the individuals conducting the surveys were all women, and being forced into small talk with an unknown woman may have made men less comfortable over the course of the survey.

How organizations can use the survey information

For the anonymous apparel firm where this survey was conducted, the primary takeaway is that harassment is meaningfully more widespread than standard surveys or the firm’s internal reporting channels would suggest, and that addressing the harassment would have much larger benefits for both male and female employees than prior evidence would conclude.

A secondary finding has big implications for anti-harassment policies: Supervisors who do not harass any workers are a minority, ranging from 5 to 27% of all supervisors depending on the type of harassment. About half of supervisors are likely to harass workers at an intermediate rate, and roughly a third of supervisors are estimated to harass workers at a high rate. This one third of supervisors is responsible for about 56% of harassment that takes place.

What does this mean for anti-harassment workplace policy? According to the authors, the fact that harassment is widespread across teams means that “firing a few bad apples cannot be the sole policy option.”  

Instead, policy should prioritize addressing the worst offenders while also focusing on changing the behavior of existing managers. 

The authors also highlight that the extent to which victims are isolated in teams varies substantially by type of harassment. When victims are more isolated, policy will need to outline actions for addressing harassment when only one victim comes forward. 

Finally, the authors argue that the fact that reports are noisy signals may in fact be an advantage when it comes to adoption of the survey method by organizations. Because the consequences of harassment cases are often drastic, organizations sometimes prefer not to find out about internal misbehavior, forcing victims to either remain silent, or take their grief to the court of public opinion. The methodology used by the researchers offers a different paradigm. By making complaints easier and more frequent, and by reducing the strength of follow up actions firms can take, the approach lets organizations get early signals of misbehavior, allowing them to correct behavior before it gets out-of-control.