Measurement and evidence

How to measure software work without misleading yourself: developer productivity metrics, AI ROI, cycle time, causal reasoning, and psychometrics.
28
Jul
A white fluffy dog runs joyfully through gentle ocean surf on a sunny day

We need more than a metaphor: here are testable diagnostics for comprehension debt

Do we understand our code? I think we need to get past symptoms and into causes. Here's a field-tested measure of one organizational condition that can erode comprehension (Overproduction Pressure) you can use to find out which world your team is living in.
26 min read
21
May
A photograph showing a truly absurd helmet that entirely covers the head of a person

Software research must become more reusable

This month I gave two invited talks that as paired pieces form a kind of bracket to hold together several
9 min read
12
Feb
A photo of white snow shows two medium-sized pawprints and the four fingers of a gloved hand

How Not to Measure the ROI from AI in your Software Organization

You did your best, but it happened: somebody told you that you are the one who has to measure the
16 min read
14
Nov
An image with a yellow background and two headshots of Cat Hicks and Ariana Mirian on it. Text reads Empirical Security Now! With Dr. Ariana Mirian

Can we make Security Empirical, and why might we want to?

There's a paper I recently came across by Mohammad Tahaei and colleagues called Privacy Champions in Sofware Teams:
7 min read
13
Jul
a corkboard has blue and yellow postit notes on it with handwritten notes. The blue postit note in focus reads Part 1: Developers are People.

Are developers slowed down by AI? Evaluating an RCT (?) and what it tells us about developer productivity

Seven different people texted or otherwise messaged me about this study which claims to measure “the impact of early-2025
24 min read
09
Jun

Unfortunately, you have to think about cause

So somebody came to you (maybe it was your own mind, that traitor!) and said you need to think about
20 min read
04
May

On Taking Down Your Local Meritocracy | Research Dive

18 min read