by Lindsay Nash | May 4, 2021 | Articles, Blog
There’s more than one way to pick a winner.
In fact, in the Award Force judging suite the options for setting up and managing an effective and successful assessment process are practically endless.
While the options are aplenty, judging configuration can sometimes feel like a daunting task. Where should you start? How can you decide which judging mode (or combination of modes) is right for your program?
We’ll break it down for you in this article with an overview of the Award Force judging suite, some helpful tips, use cases and best practices on how to set up your awards program judging process for ultimate success.
There is no one way to judge an awards entry. That’s why Award Force provides an entire suite of sophisticated judging tools to help you assess each submission. All our judging modes are designed to be quick, friendly and easy to use for judges.
We offer the following four judging modes:
Qualifying is used to make simple pass/fail decisions. This is an ideal method for cleaning up a large volume of awards program entries before moving them on to a more in-depth judging round.
The Top Pick judging mode employs the STV method (Single Transferable Vote – widely used in Western elections)) of calculating results. It’s designed to achieve proportional representation through ranked preferences. Perfect for multi-winner outcomes or to draw out consensus amongst a large judging panel.
With this method, you can decide how many preferential picks each judge gets, and they have a live progress bar to help them keep track of their choices.
The Voting judging mode is a “people’s choice” option and is great for soliciting votes from the public. It’s also useful for garnering votes from a closed audience, like a specific organisation, industry body or other members-only groups.
VIP Judging is a highly configurable score-based assessment method and is best for in-depth reviews performed by industry, organisational or expert judges. You have complete control over the judging criteria, which your VIP judges can see and refer to throughout the judging process.
In Award Force, you can tailor your judging configuration to suit your program by using one or more judging modes to support your judging process. You can use multiple judging modes in tandem or set them up to run consecutively, moving entries through several different rounds of qualitative and quantitative assessment for the best judging outcomes.
To help you visualise what’s possible, we’ve compiled a few common use cases to illustrate just some of the ideal judging configurations.
What: A highly visual arts contest that solicits a high volume of photographic entries.
Suggested judging mode(s): Qualifying followed by VIP judging with an online gallery for sharing (shown above in the Award Force platform)
How it works: Using the Qualifying judging mode, program managers (or the judges themselves) can quickly pass/fail entries based on a benchmarked quality standard. Once the entry has passed the qualification round, it can be automatically sent to expert judges using the VIP judging mode. These experts then assess and score based on specified criteria with the highest scored entry being declared the winner.
Other tips: Create categories to help organise the entries as they are submitted. Consider setting up an online gallery (also possible in Award Force) to share shortlisted entries with the public.
What: An association-sponsored awards program to recognise the best in industry, using a large community of board members to serve as judges.
Suggested judging mode(s): Top Pick followed by VIP judging
How it works: The association wants all the board members involved in the awards selection process but finding consensus amongst a large judging panel can be challenging, so we suggest using Top Pick to help find the top 10 preferred entries and then follow that up with a round of expert review using VIP judging. Initially, using Top Pick, each judge would simply pick 10 of their preferred entries. These individual picks will then be calculated using the Single Transferable Vote method and the top 10 preferred entries will move forward in the judging process. Then, the association’s expert judges will assess these 10 entries against a set of criteria and apply a score for each criteria set. The winner is declared based on the highest scoring entry.
Other tips: You can optionally allow judges to discuss and collaborate with each other during the judging process by enabling judges to comment and/or view comments from other judges on a per entry basis.
What: A university portfolio assessment program for graduate students requiring extensive research and written portfolio assessment.
Suggested judging mode(s): Qualifying round with plagiarism detection followed by VIP Judging
How it works: During the qualified round, a plagiarism detection scan can be run on all submitted entries to search for any plagiarised portfolios. If plagiarism is found, these portfolios can be quickly discarded. Once applications have passed the plagiarism scan, they can be automatically sent for expert review using the score-based VIP judging.
The score-based judging can involve multiple rounds, with different criteria and judges for each round to help assess each portfolio in-depth.Scores can be given to the portfolio as a whole or you can create individual scoring criteria for each section of the portfolio. Different scoring criteria can be given different weightings.
It’s also possible to provide access to others to participate in a peer review process. Automations can be configured to set up a workflow where the student is notified when the portfolio has been submitted, qualified, scored or whether they will be required to perform further actions.
Other tips: Use entry categories to organise portfolios up by topic, subject, grade, year or level.
What: A nationwide podcast awards program which accepts mp3 files of podcast samples, plus images of podcast art and advertisements There are several categories in this awards program, including “Best presenter” and “Best topical podcast”.
Suggested judging mode(s): Qualifying round followed by a Public Voting round
How it works: Every entry goes through a quick pass/fail qualifying round to ensure entry quality. Once the qualifying round is complete, all qualifying entries will be made available for public voting, where members of the public can easily vote for their favourites.
Each member of the public is allowed to vote once per category. And the winner in each category is determined by the highest number of public votes.
Other tips: Require voters to register before they can participate. This cuts down on any abusive voting practices such as syndicate voting schemes or bot voting. Get more tips on creating a buzzworthy public voting program.
What: A staff excellence program where employees are nominated and then approved by their manager before assessment by a review panel.
Suggested judging mode(s): Review flow and VIP judging
How it works: A nominator submits an entry, the nominee is notified and decides whether they would like to enter by accepting or declining their nomination. If they accept, the entry is sent to their manager who either approves the nomination for judging, or requests a new submission from the original nominator. Manager approved submissions then move forward into the next round of VIP judging and a winner is selected.
Other tips: You can also optionally allow the nominee to edit their submission before it is sent to their manager for approval. This makes it easy for nominees to edit or correct the submission and ensure it is both factual and compelling.
Now that you have an idea of what’s possible when it comes to actual judging configuration, what else can you do to make sure the process runs as smoothly as possible? We’re glad you asked. Providing a delightful experience is often in the details. Here are some tips from our judging software experts at Award Force.
Your judges are likely very busy people. It’s important to make sure the judging experience is both easy and fast! The Award Force platform is intuitive from the start. But there are extra steps you can take to make sure your judges have a seamless experience, and that starts with providing all the information they need, right where they need it.
In Award Force, you can add contextual information, which we call content blocks, throughout the judging view. In these locations you can provide judging instructions, images, or even videos to convey your message or content. In the scoring section, it’s easy to add help text right where they’ll add scores, making it easy for them to score without any confusion or questions.
In addition to providing helpful content for your judges, it’s also important to communicate regularly with them. This could include general updates about your program, important dates for judging, reminders on when their judging will start, whether there are any new entries for them to assess and other important program announcements.
In Award Force, you can send personalised emails to your judges directly from the platform, complete with your branding. No jumping between systems, software or other websites required.
It’s important to create an entry form that your judges will love—from the very beginning. This means only asking for the information that judges need to make informed decisions about each entry. Consider applying word or character limits to entry fields to keep the entries in check and require consistency in any required attachment files. This will save your judges time and help them easily view and assess each entry efficiently.
Manage potential conflicts of interest or bias with options for you to recuse and judges to abstain from judging.
In Award Force’s judging software, you can easily recuse a judge from assessing specific entries and re-assign any relevant entries in the Assignments view and Panels. And, if a judge identifies a potential conflict of interest, they can also abstain from assessing it with a simple click on the relevant entry.
It’s also possible to anonymise entries to remove potential judging bias. Use the field visibility tool to hide personal entrant information on entries (includes downloads and attachments), to ensure your judges won’t know who submitted what.
The judging round of an awards program is an important part of every program. So, it’s vital to provide a simple and efficient experience for your judges.
The Award Force judging suite gives you ultimate control to configure your judging quickly and easily and with flexible options you can create a judging workflow as unique as your awards.
Articles
Feature focus
How-to-guides
Press releases
Product updates