What Good is a Time Sheet?

Four examples of how human nature gets in the way of meaningful analytics.

What do these instances have in common?

1. Police leadership covertly encouraging officers to under-report violent incidents.
2. Teachers helping students cheat on standardized tests.
3. Project managers shifting expenses between projects to make actual costs match estimated costs.
4. Hourly agency employees over-reporting billable hours to look busier than they are.

In these examples, those entering the data have different goals than those analyzing the data. While organizations want an objective view of their operation, individuals act in their own interests to show data that serves their goals. That's just people being people.

Metrics break down when the reporter applies a judgement to the numbers they collect.

Consider the situation closest to home for us. If I ask staff to report time spent on a project, I may do a few things with that information. I could ask project managers to use it to refine our operations. I could have department heads use it to evaluate employee performance. But If I try to do both, then I create an incentive for project managers and employees to start gaming the system according to their own interests.

When two parties perceive different numbers as good, the data going into your system gets twisted. Count on it. And when data going in is bad, data coming out is even less reliable.

I think about the psychology of reporting as I watch the media and businesses become increasingly reliant on numbers to create narratives and drive strategy. Smart software and beautiful infographics make data driven decision-making very appealing. But so often, we're only analyzing indexes approximated from the estimations of human beings.

When input data is bad:

1. We act on lousy insight created by skewed reports.
2. We focus too much on indexes that are less relevant but easier to measure.
3. We enact harsh honesty incentives that skew numbers even more.

Are there any psychology wonks with good ideas out there? I know plenty of agencies would love to hear your thoughts. I suspect there are some clever game-inspired tactics that could encourage people to report more honestly.

Here's an example. Zappos pays its newest new hires to quit. That's how they create a positive incentive for anyone faking team spirit to self-identify. Genius.