From brief to rubric: How to set judging criteria your panel will actually use

There is something satisfying about the moment in an awards cycle when the submission deadline has passed, the entries are in and the judges begin scoring. But then the scores might start to diverge: One judge may find an entry especially creative while others place more weight on impact. Even within the same scoresheet, a panel can reach very different conclusions.

Particularly when judges score independently and online, clear judging criteria are essential. This makes scores more comparable and turns different areas of expertise into a transparent assessment. For that to work, the criteria must be well chosen, clearly worded and applied consistently throughout the process.

What judging criteria do

Judging criteria give the panel a benchmark. They make clear which aspects matter most and what outstanding work looks like. Even an expert panel with excellent judgement needs a shared framework to reach a fair and comparable result. 

Judging often serves several goals at once, as shown by this 2026 study published in Public Choice, which looks at jury decisions in international piano competitions. Fair judging should reduce bias, make results more comparable and still draw on the judges’ expertise. These goals do not always pull in the same direction. Rigid rules can improve comparability, but they can also limit room for different expert perspectives. 

For your awards program, this means deliberately defining what is being judged while still leaving space for well-reasoned differences of opinion. The aim is a shared understanding of what a strong entry looks like, without needing identical scores.

Why even experienced panels judge with bias

Humans can’t judge completely objectively and even experienced judges can be influenced unconsciously by a range of effects. Three patterns are particularly relevant for awards:

• Halo effect: One strong aspect, such as a compelling video or a well-known brand, can colour the perception of an entire entry and can lead to a more favourable evaluation.
• In-group bias: Familiar names, industries or regions of origin can influence judgement without anyone noticing. A 2025 review in Cogent Social Sciences by Patrick Alexander Braeunig shows this for sports in which judges award points, such as figure skating. Among other things, it documents favouritism towards fellow nationals, the marking down of rivals and vote trading between judges. This cannot be transferred to awards one-to-one. Still, the same principle applies: personal connections and group loyalties can shape scores.
• Order effects: The position of an entry can also play a role. The first or last entry to be evaluated may be perceived differently from those in between.

These effects arise unconsciously and have nothing to do with bad intent. Good judging criteria direct attention to the aspects that are meant to be assessed and help judges separate personal impressions from defined criteria.

Two further levers are a calibration session before judging begins and a clear approach to conflicts of interest. Anyone with a personal connection to an entrant, or who helped prepare an entry, should step back from scoring it. In early rounds, anonymised judging is worth considering, since it reduces any personal influences. Keep in mind it also withholds information that may be relevant to the judgement.

What makes judging criteria good

Good criteria strike a balance between guidance and openness. They are specific enough to limit room for interpretation but leave enough space for exceptional work. Four features help:

• Four to six criteria: More criteria may look more precise at first glance, but it can quickly make scoring needlessly complex.
• Clear weighting: Not every aspect is equally important. Weighting shows judges which criteria count most towards the overall score.
• Short descriptions: A precise explanation makes clear what each criterion means and what judges should look for.
• Anchor descriptions: Examples of strong, average and weak performance make abstract standards tangible and help judges score more consistently.

A criterion such as ‘overall impression, 1 to 10’, by contrast, is of little help. It leaves too much room for interpretation and makes it hard to compare different scores. A scoring rubric for an innovation award might look like this:

It is also worth adding a free-text field for observations that the defined criteria do not cover. That way the rubric provides guidance without losing sight of what makes an entry exceptional.

Six steps to a scoring rubric

1. Clarify the goal.
What should your award recognise? Innovation, quality of implementation, social impact, originality? The answer shapes everything that follows.

2. Develop the criteria together.
Involve your judges in the process rather than presenting them with finished rules. This helps you see early on where criteria are understood differently and lets you sharpen the rubric from the perspective of the experts who will use it.

3. Make the language concrete.
Replace abstract buzzwords with observable statements. Instead of ‘relevance’, for example: ‘The entry solves a specific problem faced by the target audience.’ The more precise the wording, the easier it is for judges to apply it to individual entries.

4. Weight and explain.
Why does originality count for more than presentation? Make this decision deliberately and explain it.

5. Plan a calibration session.
Before scoring starts, the panel scores one or two sample entries together. Imagine two judges give the same entry 9 and 4 points. In conversation it emerges that one understands ‘implementation’ as a finished product and the other as a realistic plan. Clearing this up takes ten minutes and saves a great deal of discussion later.

6. Include comment fields.
Ask judges to add a short rationale to their scores. This documents decisions, helps when questions arise and can give entrants valuable feedback.

What entrants expect from judging criteria

Clear judging criteria helps the judging panel. It also helps. entrants, who should understand the standards by which their entry will be assessed. This makes decisions easier to follow, especially when the result is disappointing.

That is why you should publish your criteria with the call for entries. Entrants can then tailor their submissions and better gauge what matters in the judging. At the same time, transparent communication creates a reliable framework for the whole program.

Individual scores deserve a closer look, however. Research on jury decisions suggests that publicly visible individual judgements can lead judges to align more closely with the expected consensus. Making criteria and weighting public creates transparency. Whether you also publish individual scores is something to decide deliberately.

When digital support makes sense

Paper forms and spreadsheets can quickly reach their limits in larger programs. Digital judging systems let you configure scoresheets flexibly, combine scores automatically and build in comment fields. A complete record of the judging steps keeps the process transparent after the event, too.

Internal Award Force data shows how differently awards design their judging systems: 77% of programs use a single overall score rather than weighted individual criteria. Only 2.4% make comments mandatory when scoring, and a further 11% offer them as optional.

Above all, the figures show that programs take very different approaches. In practice, there is no single perfect judging model. An overall score can work well for smaller programs or when it comes with clearly described anchors. What matters is that the system suits the award. An innovation award with several distinct criteria may need a different rubric from a program in which the panel judges an entry on one overall score.

Award Force offers a configurable environment for this. Criteria, weightings and scoring scales can be adapted to your program rather than tied to rigid templates. You decide whether comments are required and can shape the judging process to your requirements. Security and data protection are a high priority, because entrants and judges entrust you with sensitive information.

A good criteria system is a promise of trust

Awards rely on the belief that excellence can be recognised and fairly rewarded. A well-designed judging system makes that promise visible. It gives the panel direction, entrants clarity and your program substance. This is exactly where Award Force helps: with a flexible platform that lets you adapt judging criteria, weightings and judging processes to your awards program.