How Outlier decides something is unusual

This page is the whole method. It is public on purpose: the product's claim is that its numbers are checkable, and a method nobody can read is indistinguishable from a model nobody can audit.

What stays private is the implementation and the edge cases. There is nothing to protect in the arithmetic below — it is textbook, and stating it plainly is worth more to us than keeping it.

What is read#

Three fields per ticket, and its changelog:

created,updated,status

A fifth is read when a gadget is being configured, and only then: labels, over up to 500 tickets, so the form can offer the labels that board actually uses. The ones chosen are stored on the board's own row.

A sixth is read only by a board whose card is framed by its sprints: the board's open sprints. What is kept is the sprint's name, which the card prints, and how much of the work in flight falls outside it.

From the changelog the pipeline keeps two things: when a status changed, and the name of the field that changed.

A ticket appears on the card as its key, never its title — a title is content, and that field is not among the ones asked for above.

One kind of value is kept and it is worth naming: the status names themselves, because a column has to be called something on the card. Nothing else. Not a summary, not a description, not a comment, and not who made a change.

Stated precisely, because this is the claim a security review will test: Jira's changelog endpoint has no field filter, so the response carries the text of summary and description edits and the name of whoever made each change, and all of it is discarded at the boundary. The privacy policy sets out what that means for what is stored; what matters here is that none of it enters the arithmetic.

The honest sentence is "we read only how tickets moved", not "content is never sent to us". The second one would be easy to disprove; the first one survives being checked.

Nothing leaves your site. The app makes no request to any host but your own Jira. There is no analytics endpoint, no vendor backend and no model anywhere in the pipeline — the arithmetic below runs inside Atlassian's infrastructure, on your data, and the results are stored there. This is checkable rather than promised: the manifest declares no external domain, no remote and no outbound fetch, and Atlassian shows it to you before you install. The app meets the conditions of Atlassian's Runs on Atlassian programme on that basis; the badge itself is Atlassian's to grant, so treat its presence on the listing — not this sentence — as the confirmation.

One control is outside that sentence, and it belongs here rather than only in the privacy policy: Ask Rovo why hands a single question to your site's own Rovo, on your credits, if your site has Rovo at all. It is a button somebody presses, not a step in the pipeline, and the answer goes to them rather than to us — but a page that says nothing leaves your site should be the page that names it.

One further thing, because it is the one place a person's name could reach this app: the query it sends is assembled from a board's own saved filter, and a saved filter quite ordinarily names a person. That query is not stored. What is stored is what it is composed from — the board and filter ids, and the labels a gadget is narrowed by — and the query is built from those on each pass and kept nowhere.

The norm is the board's own history#

There is no model of a healthy team, no industry benchmark, and no target. A column's norm is what that column has done.

Measured in working days. Monday to Friday by default, in the board's own timezone; a team that works other days says so in the configuration. A ticket that sits over a weekend has not been sitting for two days.

Built from the ages a board carried, not the durations it finished. This is the one place where the obvious method is wrong, so it is worth spelling out.

The obvious version compares an open ticket's age against how long finished stays took. That comparison is biased and the bias is not small: at any moment the open work is, by selection, the work that has not finished yet. Judged against completed durations, a healthy board flags most of its own work in progress.

So a completed stay of L working days is not one observation. It is the observations a daily glance at that board would have seen: L, L−1, L−2 … down to one. That is the population an open ticket belongs to — ages a board was carrying, on ordinary mornings, and mostly resolved without anybody noticing.

That population is not a reasonable choice among several. Counting the carried ages at or above a gives the sum of (L − a + 1) over finished stays, so their density follows the board's own survival curve — which is exactly the distribution the age of a randomly chosen open ticket is drawn from. Asking about a ticket standing in a column this morning is asking a question this population answers and the other candidates do not.

A stay shorter than a working day contributes one observation. A zero-length stay — a bulk move through a column — was never open at any glance and contributes none.

The threshold#

Over those ages, two rules produce a number and a third is a floor the result is never allowed below. The two do not race: the second runs only where the first cannot.

A quantile of the carried ages, and this is the rule that normally decides. The threshold is a high percentile of what that column has actually carried: if a board has been carrying tickets at this age on ordinary mornings, that age is ordinary.

Which percentile is chosen per board, at install, and then held. The walk that reads a board's history is also the moment its whole past can be replayed, so the product asks that history a question rather than asking you: at what setting would a median past morning of this board have flagged about three tickets? The loudest setting inside that budget wins — which is a bar on how sensitive the board is, not a quota on what a morning prints. Most mornings print nothing; see below. A board where none fits keeps the quietest one and says so on the card.

It is calibrated once rather than continuously, and that is deliberate. The norm is already adaptive. An adaptive threshold on top of an adaptive norm lets a team that slowly gets worse drift into silence while the product congratulates it — and it makes "why was there a signal yesterday and not today?" unanswerable, because the bar moved rather than the board. Recalibration is rare and explicit, and always says so.

The band, when a quantile cannot be taken. The band is median ± 1.5 × IQR, and the threshold is its upper edge — median + 1.5 × IQR. This is the older rule, kept as the fallback rather than as a competitor.

Median and interquartile range rather than mean and standard deviation, because one abandoned ticket should not move a team's norm, and on samples this size robust statistics are simply the correct ones — not a simplification of something better. It is not Tukey's fence, which is taken from the upper quartile rather than from the median; the median is the centre this product already quotes everywhere else, and keeping one centre is worth more here than matching a convention.

A floor, applied to whichever of those two ran. The threshold is never lower than the median plus one working day. Without it, a team whose review always takes exactly one day has zero spread, and every ordinary wobble becomes an anomaly. On such a column the floor is the threshold.

The card records which rule decided each number, rather than leaving it to be inferred. A quantile that happens to land on the band's own edge and the band itself are not the same answer, and an explanation that guesses between them describes arithmetic nobody ran.

The one number this page does not print is the percentile a given board ended up with. It is a fact about that board, produced by that board's own history, and it is on the card for the team that owns it.

When a column gets no norm at all#

Four guards, in the order they apply, and a fifth thing that can still refuse a column a norm.

Minimum sample: 15 completed stays. Below that, a threshold is precision that is really noise.

A window of 180 calendar days. Without one, a board is judged against its whole recorded past. On a public backlog corpus the median for one status came out at sixteen years, built partly from stays that ended in 2011 — a team that got three times faster would be measured against the team it used to be.

The window stretches once, to twice its length. If 180 days do not hold 15 stays, the window slides back — to at most 360. Past that the column gets no norm, and the block says so rather than comparing against fiction.

That cap was chosen by measurement. On a board that had stalled, taking "the most recent fifteen stays, however old" picked up exactly the abandoned tickets somebody eventually bulk-closed, and the usual stay in one column jumped from about a day to forty-seven. A norm built from the last gasp of a dead period describes nothing, and describes it confidently.

A stretched window has to agree with itself. When the window slid back to find enough stays, the recent ones are checked against the older ones: if the usual stay here has changed, the older history no longer describes this column and the norm is refused. A public board measured at the end of its recorded history had 90 and 117 completed stays in its two working columns and no norm for either — the wait in both had gone from about two days to weeks, and only twelve stays had finished since. The card says which of the two is happening: a column short of history counts toward its first norm, and a column whose wait changed counts the recent stays a new one needs.

A borrowed norm has to describe the column that borrows it. Where the sample is too thin to build a norm and a pooled one is available, the pooled one is still checked against the column's own history: if the column's usual stay sits outside it, the borrowing is refused and the column keeps no norm. Measured on the bench corpus, this denies exactly one column on one board - it is rare, and it is the difference between "no norm" and "somebody else's norm", which is the whole point of having it.

When a column has no norm of its own, it may borrow the norm of columns sharing its role — to do, in progress, review, done. Only those four: a status the product cannot classify has nothing to pool with, and borrowing from the wrong role is worse than saying nothing. A column judged this way is marked as such on the card.

Two medians, and why the card quotes the other one#

The median the threshold is built from is the median of carried ages. It is the right number for the threshold and the wrong number to say out loud.

So the sentence a reader sees — "usually under a day here" — quotes the median duration of completed stays, which is what a person means by "usually takes". Both are computed; they answer different questions, and the card is careful about which one it prints.

When the block speaks#

A ticket is raised when its age in the current status is past that column's threshold. Beyond that:

• At most three findings a day. What does not fit is counted and placed ("…and 4 more past the usual wait across 2 more columns"), so a reader is never told three findings were all of them.
• Several tickets waiting in one column collapse into one finding, with its denominator. Nine lines about nine tickets in the same queue is nine times the same fact.
• A ticket returning from done is an event, not a threshold. The comparison is its own history: it was finished, and it is not.
• Silence is separate from waiting. How long since anything happened to a ticket is read from the changelog by field name, and split into work events — a commit or merge request linked, a file attached, time logged, the description or title rewritten — and administrative ones: rank, sprint, fix version, story points, assignee. Two working days of nothing is where the card starts saying so. This improves the sentence rather than the ranking: "six days, and nobody has touched it" is what makes a number worth acting on.

At most three, and always with a denominator. That is a ceiling on what one morning may print, not a floor. Three is also what calibration aims at when it picks a board's sensitivity — the loudest setting under which a median past morning of that board would have flagged about three tickets — and those are two different numbers that happen to share a value. A reader took the second for a promise about behaviour and concluded the card speaks every morning.

It does not, and how often it speaks depends on the board rather than on the setting. Measured through the path an install actually takes — the walk, the hourly collection, the card — over 255 mornings across six public boards, anchored where each board was still moving:

So on a board running two-week sprints the block has something to say on a good half of mornings, and on a long-running backlog it is quiet for weeks at a time. The median morning on the long-running boards flags nothing at all. The ceiling of three is reached — once in those 255 mornings, and not on the sprint board.

That last number matters more than it looks: the ceiling is not decoration, and a page saying it had never been reached would be a claim a reader can disprove in half an hour. It was, and this is the correction.

Why calibrate to three, then, if a morning rarely holds one? Because a queue collapses into a line: eight tickets stuck in one column print one finding, so the number of lines says how many columns are odd rather than how loud the board is. Three is measured over flagged tickets for that reason, and a board whose history offers no setting quiet enough keeps the quietest one and says so on the card.

What makes that trustworthy is not the silence but the arithmetic beside it. The block reports how many tickets it looked at, how many it could judge, and how many sit in columns with no norm yet. A quiet morning is a result with a denominator, not an absence.

What a sprint has to do with any of this#

Nothing, and that is worth saying plainly, because the examples on this site are drawn from long-running boards and a reader running two-week sprints reasonably asks whether the arithmetic is about them.

The norm is age in a column. Not velocity, not carryover, not whether the sprint commitment will land. The product reads a board's open sprints for exactly two purposes — to print the sprint's name on the card, and to say how much work in flight sits outside it — and it reads only the name and the id. It does not read a sprint's start or end date, it does not read closed sprints, and it discards the Sprint field from a ticket's changelog. There is no sprint anywhere in the arithmetic that decides what is unusual.

What that means for a two-week team is narrower than it sounds, and it was measured rather than assumed. On three public boards running a verified fourteen-day cadence, the norms are short: 84% of column-days print "under a day", against 40% on the long-running boards this page's other numbers come from. But short norms do not make the card louder. The threshold is never lower than the median plus one working day, and on those boards that floor decides the threshold a quarter of the time — so the bar a ticket has to clear is a median 3.2 working days, and the median wait behind a finding is 9.6 working days. Most of a sprint. The printed multiples went down, not up: a median of 9x against 78x on the long boards.

A control group settles what causes the short norms, and it is not the sprint. The same workflow at the same company without sprints prints "under a day" on every column with a norm, and is twice as loud as any of the sprint boards. Short norms come from workflows with pass-through statuses — Needs Review, Needs Merge, Needs Verification — and a board that runs sprints is, if anything, quieter than the same team's work that does not.

So: a ticket sitting six days in code review inside a ten-day sprint is exactly what this method finds. A question about whether the sprint commitment will land is not, and this page will not pretend otherwise.

What it would have said before it was installed#

A card installed this morning has nothing to show yet, and a board where nothing is stuck produces the correct answer and an empty one. So on the first mornings — the ones with nothing new to raise, where there is room to read it — the block also says what it would have said over the board's last quarter.

This is not a record of what the card said — there was no card. It is the board's own history, measured by the norms in use today: a stay of nine working days that ended on a Thursday was open on each of the mornings before it, and the same arithmetic that chooses the threshold counts how many items would have been past the line on each of them. Every sentence built from it says so, in those words.

It comes from the rows already written when the board's history was walked, so it needs no new storage and nothing is kept about any past morning. It stops being shown once the card has ten mornings of its own to point at: those are evidence, and this is a reconstruction.

A board with no norms yet cannot be reconstructed either, and the block says that rather than reporting a quiet quarter it cannot see.

What the method cannot see#

This list is short and it is not an apology. Each of these is a real limit, and knowing them is the difference between reading the card and trusting it blindly.

• Work done outside Jira is invisible. A ticket somebody is actively working on, discussing in a call and building in a branch, without touching the ticket, looks exactly like an abandoned one. The card can say "nobody has touched it in six days"; it cannot say nobody worked on it.
• A column used as a container is not distinguished from a queue. Some teams keep a long-lived ticket in a working column deliberately. It will be raised, repeatedly, and it is not wrong to raise it — but it is not news.
• A board that changes its workflow starts over. Renamed and re-created statuses have no history under their new identity.
• A team with no variation gets few signals. That is the floor doing its job, and it means the product is quieter on very regular boards than on chaotic ones.
• A column's norm learns from the delays it exists to catch. A stay that ran long eventually finishes, and then it is history like any other: it enters the window and lifts that column's bar. This is not a flaw of one construction — any threshold estimated from the population it judges absorbs what it judges — and the consequence a reader could see has not been observed at all. A finding raised yesterday, still standing and a working day older today, and no longer raised because the bar moved under it: zero of 750 such mornings on one public board, zero of 2560 on another, zero of 266 on a third. The absorption is real — a stay the product flagged lifts its column's bar by a median 4.1% when it finishes, against nothing at all for an ordinary one — it simply never climbs fast enough to overtake a ticket that is aging a day a day. Both fixes that were tried cost more than that: trimming out the stays the product itself called unusual truncates the right tail of its own reference, and giving each stay one vote instead of one per day it was open estimates a quantity nobody asked about. What the card prints is unaffected either way — the multiple is measured against how long stays take, not against the bar.
• A first install has norms only where the board's own history has them. The history is read at install, so most columns are usable on day one — but a column that has passed fewer than fifteen tickets in a year will not be.
• A board carrying streams of different speeds shows the slowest one. The norm is one number per column, and a column that three kinds of work pass through at three speeds gets one that fits none of them. Measured on a simulated board of 536 tickets over ten months, carrying payments, support and migration behind three labels: the whole board's norm for In Progress came out at 16.5 working days, the same board narrowed to payments at 5.0, and to support at 2.0. A payments ticket standing a week is past its own stream's norm and nowhere near the board's, so it is never raised. Over the same 25 mornings the whole board raised 2 findings and carried 84 lines — the same slow work, named once and then correctly reported as old — while the payments gadget raised 40 and carried 2.
• For a stream that finishes work inside a day, the norm is the floor rather than a measurement. On the same board, support's threshold came out at exactly 2.0 working days in all four columns: the median of 1 plus the minimum half-width of 1. That is the floor doing the job it exists for, and it means that for fast streams this stops being "compared against this board's own history" and becomes "tell me when something has sat for two days". Both sentences are true; they are not the same product.
• Narrowing to a stream costs you its history, and a thin stream may not have enough. The advice to run one gadget per stream assumes the stream has passed enough work to be measured. Measured on the same three streams over a quarter rather than ten months: migration, 20 tickets in 90 days, spent 43 of 49 mornings still learning and never raised anything at all, while the unnarrowed board raised 27 findings over the same mornings — including a migration ticket parked in its own column, which the migration gadget never named. Over ten months the same comparison runs the other way. Which is better is a question about how much work the stream finishes, not about narrowing.
• How much this card says was decided on the day it was set up. The norms above keep updating; the bar for speaking at all does not. It is chosen once, by a sweep over the work in flight on the install day, and held — so a board set up during a crunch week starts with a different bar from the same board set up a quiet week later. Measured by installing the same gadget on each of 49 consecutive mornings of one simulated board and changing nothing else: 43 of the arrival days settled on one bar and 6 on a bar half of it. Which means these counts are comparable across mornings on this board, and not comparable across boards — two teams reading three signals and none are not two teams with different amounts of trouble.
• Nothing here is a measure of a person. The pipeline derives nothing about one and stores no field about one, and no output is per-person. It is not that no name ever arrives: Jira's changelog hands over whoever made each change, and that name is dropped where the response is read. The rule is enforced by a check that runs on every record before it is stored, and by tests over that check, rather than by intention.

When the arithmetic changes#

The method is versioned. A change that would move a number on a board where nobody touched anything bumps that version, and two things follow: stored norms are rebuilt, so every installation measures the same thing, and the next block a reader sees carries one line saying the measurement moved and in which direction. Not a changelog, not a permanent banner — one sentence, once.

Current method version: 3.