How Outlier decides something is unusual

This page is the whole method. It is public on purpose. The product claims that its numbers are checkable. A method nobody can read is indistinguishable from a model nobody can audit.

What stays private is the implementation and the edge cases. The arithmetic below is textbook. Stating it plainly is worth more to us than keeping it.

What is read#

Three fields per ticket, and its changelog:

created,updated,status

From the changelog the pipeline keeps two things: when a status changed, and the name of the field that changed. A ticket appears on the card as its key, never its title.

Nothing leaves your site. The app makes no request to any host but your own Jira. There is no model anywhere in the pipeline. In short, we read only how tickets moved. What else arrives, and what is dropped at the boundary, is set out in full in the privacy policy.

Ask Rovo why is the one control outside that sentence: a button that hands a single question to your site's own Rovo, described in the privacy policy.

More is read in two cases only:

• While a gadget is being configured, and only then: labels and issuetype, over up to 500 tickets. The form uses them to offer the labels and the kinds of work that board actually carries. Epics and subtasks are left out to begin with. The ones chosen are stored on the board's own row.
• Only by a board whose card is framed by its sprints: the board's open sprints. What is kept is the sprint's id. Its name is printed on the card and not stored.

A title is content, and that field is not among the ones asked for above.

One kind of value is kept: the status names themselves. A column has to be called something on the card. Nothing else: not a summary, not a description, not a comment, and not who made a change.

The norm is the board's own history#

A column's norm is what that column has done. There is no model of a healthy team, no industry benchmark, and no target.

Measured in working days. Monday to Friday by default, in the board's own timezone. A team that works other days says so in the configuration. A ticket that sits over a weekend has not been sitting for two days.

Built from the ages a board carried, not the durations it finished. This is the one place where the obvious method is wrong.

A completed stay of L working days is not one observation. It is the observations a daily glance at that board would have seen: L, L−1, L−2 … down to one. That is the population an open ticket belongs to: ages a board was carrying on ordinary mornings, mostly resolved without anybody noticing.

Two edge cases:

• A stay shorter than a working day contributes one observation.
• A zero-length stay, a bulk move through a column, was never open at any glance. It contributes none.

The threshold#

Over those ages, two rules produce a number. A third is a floor the result is never allowed below. The two rules do not race: the second runs only where the first cannot.

A quantile of the carried ages is the rule that normally decides. The threshold is a high percentile of what that column has actually carried. If a board has been carrying tickets at this age on ordinary mornings, that age is ordinary.

Which percentile is chosen per board, at install, and then held. The walk that reads a board's history replays its last four months as they stood on the install day. The product asks that history a question rather than asking you: at what setting would a median past morning of this board have flagged about three tickets? The loudest setting inside that budget wins.

That is a bar on how sensitive the board is, not a quota on what a morning prints. Many mornings print nothing; see When the card speaks. A board where no setting fits keeps the quietest one and says so on the card.

It is calibrated once, not continuously. Recalibration is rare and explicit, and always says so.

The band, when a quantile cannot be taken. The band is median ± 1.5 × IQR. The threshold is its upper edge, median + 1.5 × IQR. It is the fallback, not a competitor.

A floor, applied to whichever of those two ran. The threshold is never lower than the median plus one working day. Without it, a team whose review always takes exactly one day has zero spread, and every ordinary wobble becomes an anomaly. On such a column the floor is the threshold.

Each norm records which rule decided it. It is not left to be inferred. A quantile that happens to land on the band's own edge and the band itself are not the same answer. An explanation that guesses between them describes arithmetic nobody ran.

This page does not print one number: the percentile a given board ended up with. It is a fact about that board, produced by that board's own history. It is kept with that board's norms and stays off the card: a published bar reads as a target, and this product does not set one.

When a column gets no norm at all#

Four guards apply, in this order:

A fifth thing can still refuse a column a norm: a borrowed norm that does not describe the column.

The window, and how far it stretches#

Without a window, a board is judged against its whole recorded past.

If 180 days do not hold 15 stays, the window slides back, to at most 360. Past that the column gets no norm. The card says so rather than comparing against fiction.

A stretched window has to agree with itself#

When the window slid back to find enough stays, the recent ones are checked against the older ones. If the usual stay here has changed, the older history no longer describes this column, and the norm is refused.

The card says which of the two is happening:

• a column short of history counts toward its first norm;
• a column whose wait changed counts the recent stays a new one needs.

Borrowing a norm by role#

A column with no norm of its own may borrow the norm of columns sharing its role: to do, in progress, review, done. Only those four. A status the product cannot classify has nothing to pool with. Borrowing from the wrong role is worse than saying nothing. A column judged this way is marked as such on the card.

A borrowed norm has to describe the column that borrows it. The pooled norm is checked against the column's own history. If the column's usual stay sits outside it, the borrowing is refused and the column keeps no norm.

Measured on the bench corpus, this denies exactly one column on one board. It is rare. It is the difference between "no norm" and "somebody else's norm", which is the whole point of having it.

Two medians, and why the card quotes the other one#

Both are computed. They answer different questions.

The card is careful about which one it prints.

When the card speaks#

A ticket is raised when its age in the current status is past that column's threshold. Beyond that:

• At most three findings a day. What does not fit is counted and placed: "…and 4 more items past the usual wait across 2 more columns, not shown". A reader is never told three findings were all of them.
• Several tickets waiting in one column collapse into one finding, with its denominator. Nine lines about nine tickets in the same queue is nine times the same fact.
• A ticket returning from done is an event, not a threshold. The comparison is its own history: it was finished, and it is not.
• Silence is separate from waiting. How long since anything happened to a ticket is read from the changelog by field name. Work events are a commit or merge request linked, a file attached, time logged, the description or title rewritten. Administrative ones are rank, sprint, fix version, story points, assignee. Two working days of nothing is where the card starts saying so.

Silence improves the sentence, not the ranking. "Six days, and nobody has touched it" is what makes a number worth acting on.

Three is a ceiling, not a promise#

At most three, and always with a denominator. That is a ceiling on what one morning may print, not a floor.

Three is also what calibration aims at: the loudest setting under which a median past morning of that board would have flagged about three tickets. Those are two different numbers that happen to share a value. Neither says the card speaks every morning.

How often it speaks#

How often it speaks depends on the board, not on the setting. It was measured through the path an install actually takes: the walk, the hourly collection, the card. The measurement covers 255 mornings across six public boards, anchored where each board was still moving:

• On a board running two-week sprints, the card has something to say on a good half of mornings.
• On a long-running backlog, it is quiet for weeks at a time. The median morning on the long-running boards flags nothing at all.
• The ceiling of three is reached - once in those 255 mornings, and not on the sprint board. The ceiling is not decoration.

A quiet morning is trustworthy because of the arithmetic beside it. The card reports:

• how many tickets it looked at;
• how many it could judge;
• how many sit in columns with no norm yet.

A quiet morning is a result with a denominator, not an absence.

What a sprint has to do with any of this#

Nothing. The norm is the same arithmetic on every board, so a team running two-week sprints reasonably asks whether it is about them.

The norm is age in a column. Not velocity, not carryover, not whether the sprint commitment will land.

The product reads a board's open sprints for exactly two purposes:

• to print the sprint's name on the card;
• to say how much work in flight sits outside it.

It keeps only the id. The name goes onto the card and is not stored, and everything else Jira's answer carries about a sprint, its dates included, is dropped where it is read. It does not read closed sprints. It discards the Sprint field from a ticket's changelog. There is no sprint anywhere in the arithmetic that decides what is unusual.

What it means for a two-week team#

It was measured rather than assumed, on three public boards running a verified fourteen-day cadence.

The norms are short. 84% of column-days print "under a day", against 40% on the long-running boards the other numbers on this page come from.

Short norms do not make the card louder. The threshold is never lower than the median plus one working day. On those boards that floor decides the threshold a quarter of the time. So the bar a ticket has to clear is a median 3.2 working days. The median wait behind a finding is 9.6 working days, most of a sprint. The printed multiples went down, not up: a median of 9x, against 78x on the long boards.

A control group settles what causes the short norms, and it is not the sprint. The same workflow at the same company without sprints prints "under a day" on every column with a norm. It is twice as loud as any of the sprint boards. Short norms come from workflows with pass-through statuses: Needs Review, Needs Merge, Needs Verification. A board that runs sprints is, if anything, quieter than the same team's work that does not.

So a ticket sitting six days in code review inside a ten-day sprint is exactly what this method finds. Whether the sprint commitment will land is not, and this page will not pretend otherwise.

What it would have said before it was installed#

A card installed this morning has no record of its own yet. A board where nothing is stuck produces the correct answer, and an empty one. So on the first mornings, the ones with nothing new to raise, the card also says what it would have said over the board's last quarter.

This is not a record of what the card said: there was no card. It is the board's own history, measured by the norms in use today. Every sentence built from it says so, in those words.

• It comes from the rows already written when the board's history was walked. The reconstruction needs no new storage.
• It stops being shown once the card has ten mornings of its own to point at. Those are evidence; this is a reconstruction.
• A board with no norms yet cannot be reconstructed either. The card says that, rather than reporting a quiet quarter it cannot see.

What the method cannot see#

This list is short and it is not an apology. Each of these is a real limit. Knowing them is the difference between reading the card and trusting it blindly.

Work done outside Jira is invisible#

A ticket somebody is actively working on without touching the ticket looks exactly like an abandoned one. They may be discussing it in a call and building it in a branch. The card can say "nobody has touched it in six days". It cannot say nobody worked on it.

A column used as a container is not distinguished from a queue#

Some teams keep a long-lived ticket in a working column deliberately. It will be raised, repeatedly. It is not wrong to raise it, but it is not news.

A board that changes its workflow starts over#

Renamed and re-created statuses have no history under their new identity.

A team with no variation gets few signals#

That is the floor doing its job. The product is quieter on very regular boards than on chaotic ones.

A column's norm learns from the delays it exists to catch#

A stay that ran long eventually finishes. Then it is history like any other: it enters the window and lifts that column's bar. This is not a flaw of one construction. Any threshold estimated from the population it judges absorbs what it judges.

The absorption is real. A stay the product flagged lifts its column's bar by a median 4.1% when it finishes. An ordinary one lifts it by nothing at all.

The consequence a reader could see has not been observed at all. That consequence is a finding raised yesterday, still standing and a working day older today, and no longer raised because the bar moved under it. It happened on zero of 750 such mornings on one public board, zero of 2560 on another, zero of 266 on a third. The bar never climbs fast enough to overtake a ticket that ages one working day every day.

Both fixes that were tried cost more than that:

• trimming out the stays the product itself called unusual truncates the right tail of its own reference;
• giving each stay one vote, instead of one per day it was open, estimates a quantity nobody asked about.

What the card prints is unaffected either way. The multiple is measured against how long stays take, not against the bar.

A first install has norms only where the board's own history has them#

The history is read at install, so most columns are usable on day one. A column that has passed fewer than fifteen tickets in a year will not be.

A board carrying streams of different speeds shows the slowest one#

The norm is one number per column. A column that three kinds of work pass through at three speeds gets one that fits none of them.

It was measured on a simulated board of 536 tickets over ten months, carrying payments, support and migration behind three labels. The norm for In Progress came out as follows:

A payments ticket standing a week is past its own stream's norm and nowhere near the board's, so it is never raised. Over the same 25 mornings, the whole board raised 2 findings and carried 84 lines: the same slow work, named once and then correctly reported as old. The payments gadget raised 40 and carried 2.

For a stream that finishes work inside a day, the norm is the floor rather than a measurement#

On the same board, support's threshold came out at exactly 2.0 working days in all four columns: the median of 1 plus the minimum half-width of 1. That is the floor doing the job it exists for. For fast streams this stops being "compared against this board's own history" and becomes "tell me when something has sat for two days". Both sentences are true. They are not the same product.

Narrowing to a stream costs you its history, and a thin stream may not have enough#

The advice to run one gadget per stream assumes the stream has passed enough work to be measured.

It was measured on the same three streams over a quarter rather than ten months. Migration, 20 tickets in 90 days, spent 43 of 49 mornings still learning and never raised anything at all. The unnarrowed board raised 27 findings over the same mornings. One of them was a migration ticket parked in its own column, which the migration gadget never named.

Over ten months the same comparison runs the other way. Which is better is a question about how much work the stream finishes, not about narrowing.

How much this card says was decided on the day it was set up#

The norms above keep updating. The bar for speaking at all does not. It is chosen once, by replaying the board's last four months as they stood on the install day, and held. So a board set up during a crunch week starts with a different bar from the same board set up a quiet week later.

It was measured by installing the same gadget on each of 49 consecutive mornings of one simulated board, changing nothing else. 43 of the arrival days settled on one bar, and 6 on a bar half of it.

These counts are comparable across mornings on this board. They are not comparable across boards. A board with three findings and a board with none do not necessarily differ in trouble.

Nothing here is a measure of a person#

The pipeline derives nothing about a person and stores no field about one. No output is per-person.

It is not that no name ever arrives. Jira's changelog hands over whoever made each change, and that name is dropped where the response is read. The rule is enforced by a check that runs on every record before it is stored, and by tests over that check, rather than by intention.

When the arithmetic changes#

The method is versioned. A change that would move a number on a board where nobody touched anything bumps that version, and two things follow: stored norms are rebuilt, so every installation measures the same thing, and the next card a reader sees carries one line saying the measurement moved and in which direction. Not a changelog, not a permanent banner — one sentence, once.

Current method version: 3.