AI feature prioritisation: five frameworks and where they break

1-minute summary

The prioritisation frameworks most product teams already use were built for deterministic software, where a feature either ships or it does not. AI features ship at a quality level, cost money every time someone uses them, and can fail in ways that are worse than not shipping at all. This post runs five standard frameworks against the same four AI features and shows exactly where each one stops giving you useful answers. The short version: use Opportunity Scoring to pick the problem, RICE to sequence the work, Kano to set the quality bar, and MoSCoW only if you attach an eval threshold to every "must have".

What changes when the feature is AI

Before any framework, it is worth being precise about what is actually different. Four properties break the assumptions these frameworks were built on.

Output quality is a distribution, not a state. A checkout button either works or it does not. A summarisation feature works 94% of the time, and the shape of the remaining 6% matters more than the number. Every framework below asks you to score a feature as a single thing. AI features are a family of possible features, one per quality level.

Effort is unknowable until you have run evals. You cannot size an AI feature the way you size a CRUD screen. The honest estimate is not "five weeks", it is "two weeks to find out whether this is five weeks or five months". Any framework with an effort term is consuming a number you invented.

Cost is per inference, not per build. Traditional features have a build cost and then approximately zero marginal cost. AI features have a build cost and a bill that scales with success. A feature that gets more popular gets more expensive, which inverts the usual relationship between adoption and margin.

Failure has a blast radius. Shipping a broken sort order annoys people. Shipping a confidently wrong answer to a billing question costs you the account. The downside is not symmetric with the upside, and none of these frameworks have a term for it.

The example we will score

Throughout, I will use the same product: an AI layer on top of a B2B helpdesk. Roughly 40,000 tickets a quarter across 200 support agents. Four candidate features on the table:

1. Auto-tagging: classify every incoming ticket by topic and urgency.
2. Suggested replies: draft a response for the agent, who edits and sends it.
3. Auto-resolution: answer and close simple tickets with no human involved.
4. Confidence dashboard: show where the model is unsure and route those tickets to a human.

Same four features, five frameworks, five different answers.

1. RICE

Created by Sean McBride on the product team at Intercom, RICE scores each initiative as Reach times Impact times Confidence, divided by Effort. Reach is how many people or events it touches per period, Impact is a fixed scale (0.25 to 3), Confidence is a percentage, Effort is person-months.

Scoring our four:

1. Auto-tagging: 40,000 tickets, impact 0.5, confidence 90%, effort 2. Score 9,000.
2. Suggested replies: 40,000 tickets, impact 2, confidence 70%, effort 5. Score 11,200.
3. Auto-resolution: 12,000 tickets (only the simple ones), impact 3, confidence 40%, effort 12. Score 1,200.
4. Confidence dashboard: 200 agents, impact 1, confidence 80%, effort 3. Score 53.

Where it breaks. Look at the confidence dashboard. It scores 53 against 11,200, so RICE says never build it. But the dashboard is the thing that makes auto-resolution safe to ship at all. RICE has no term for enabling value, and because Reach is measured in whatever unit you picked, a feature measured in agents will always lose to a feature measured in tickets. The Effort term is the other problem: the 12 person-months for auto-resolution is a number I made up, because nobody knows what it costs to get a model from 78% to 95% on your data until they have tried.

The fix is cheap. Run RICE twice: once now to decide what to spend two weeks investigating, and again after the eval spike, when Effort and Confidence are real numbers instead of vibes.

2. ICE

ICE is the same idea with the Reach term removed, developed by Sean Ellis for scoring growth experiments quickly. Impact times Confidence times Ease, each scored 1 to 10.

1. Auto-tagging: 5 x 9 x 8 = 360.
2. Suggested replies: 8 x 7 x 5 = 280.
3. Auto-resolution: 10 x 4 x 2 = 80.
4. Confidence dashboard: 4 x 8 x 7 = 224.

Where it breaks. The Confidence term is doing two incompatible jobs. For AI work there are two separate uncertainties: will anyone want this, and can the model actually do it reliably. They are independent, and critically, one of them is measurable and the other is not. You can answer "can the model do this" with a benchmark in a fortnight. You cannot answer "will agents trust it" without shipping. Collapsing both into a single 1-to-10 guess throws away the fact that half your uncertainty is cheaply resolvable.

If you use ICE on AI work, split Confidence into two scores and treat anything with low model confidence as an eval task rather than a build task.

3. Kano

The Kano model comes from a 1984 paper by Noriaki Kano and colleagues in the Journal of the Japanese Society for Quality Control. It sorts features by how they affect satisfaction: must-be features cause misery when absent and nothing when present, one-dimensional features scale satisfaction linearly with quality, and attractive features delight when present but are not missed when absent.

1. Auto-tagging: one-dimensional today, rapidly becoming must-be as competitors ship it.
2. Suggested replies: one-dimensional. Better drafts, happier agents, in a straight line.
3. Auto-resolution: attractive at 95% accuracy. Actively negative at 80%.
4. Confidence dashboard: indifferent to end customers, must-be to the operations lead who signs the contract.

Where it breaks. Kano assumes a feature has one category. For AI features, the category is a function of model quality. Auto-resolution is a delighter, an irrelevance or a reverse feature depending entirely on where it lands on an accuracy curve, and you do not know where it lands until you build it. That is a circular dependency the model was never designed to handle.

This is also the framework that most usefully survives contact with AI, provided you use it the right way round. Do not ask which category a feature is in. Ask what accuracy this feature needs to reach before it moves from reverse to attractive, and make that number the acceptance criterion.

4. MoSCoW

MoSCoW came out of Dai Clegg's work at Oracle in 1994 and was later donated to the DSDM Consortium. It sorts a timeboxed release into Must have, Should have, Could have and Won't have this time.

For our next quarter:

1. Must have: auto-tagging and the confidence dashboard.
2. Should have: suggested replies.
3. Could have: auto-resolution, behind a flag, for three pilot accounts.
4. Won't have this time: auto-resolution on billing and refund tickets.

Where it breaks. "Must have suggested replies" is not a requirement, it is a wish. Delivered at what quality, measured how? MoSCoW's entire value comes from the categories being unambiguous, and for AI features they are ambiguous by default, because the thing being categorised is binary while the thing being delivered is continuous.

The repair is the most practical thing in this post: every Must gets an eval threshold. Not "must have suggested replies" but "must have suggested replies at 85% agent-accept rate on the 500-ticket benchmark, with no unsafe advice in the refund category". Now the category means something, and a feature that misses its threshold drops to Should have automatically instead of shipping because the quarter ended.

Note also that the Won't have row is where AI prioritisation actually happens. Deciding which categories the model is not allowed to touch is a higher-leverage decision than anything in the Must row.

5. Opportunity Scoring

From Anthony Ulwick's outcome-driven innovation. You survey users on outcomes rather than features, rating each for importance and current satisfaction on a 1 to 10 scale, then score Importance plus max(Importance minus Satisfaction, 0). High importance and low satisfaction means underserved.

Surveying our 200 agents on outcomes:

1. "Route a ticket to the right person first time": importance 9, satisfaction 4. Score 14.
2. "Answer a routine question without retyping it": importance 8, satisfaction 5. Score 11.
3. "Know when to trust an automated answer": importance 9, satisfaction 2. Score 16.
4. "Close a simple ticket without reading it": importance 6, satisfaction 3. Score 9.

Where it breaks. Two things. Users cannot rate their satisfaction with an outcome they have never had delivered reliably, so satisfaction scores for anything AI-shaped are noisy. And right now importance ratings for anything with AI attached are inflated by novelty, which will correct in a way your roadmap will not.

But look at the result. The top-scoring outcome is "know when to trust an automated answer", which maps directly to the confidence dashboard, the feature RICE ranked dead last at 53 points. That disagreement is not a flaw in either framework. It is the most useful output of the whole exercise, and it happens because Opportunity Scoring is the only one of the five that asks about the job rather than the technology.

Putting it together

What I would actually do

None of these is a winner. They answer different questions, and the mistake is picking one and running your roadmap off it.

1. Opportunity Scoring first, to choose the outcome worth attacking. It is the only framework here that is technology-agnostic, which matters when the technology is the thing everyone is excited about.
2. Kano second, to turn that outcome into an accuracy target. Decide what quality level moves the feature from irritating to valuable before anyone writes a prompt.
3. A two-week eval spike, which is the step none of the frameworks include and all of them depend on.
4. RICE after the spike, with real effort and confidence numbers, to sequence the work.
5. MoSCoW last, to commit to a release, with every Must carrying the threshold from step 2.

The frameworks are not wrong. They were built for a world where the question was what to build, and AI has added a second question of equal weight: how good does it have to be before building it is a good idea. Any prioritisation process that cannot express a quality threshold is going to keep producing confident scores for features that should not ship.

Further reading

1. Intercom on the RICE scoring model
2. Kano, Seraku, Takahashi and Tsuji, "Attractive Quality and Must-Be Quality" (1984)
3. The MoSCoW method and its DSDM origins
4. Ulwick on outcome-driven innovation
5. Opportunity Scoring, explained by Scrum.org