Ordinally Rank Submissions

Chris Quiroz -

The ordinally rank submissions task allows reviewers to rank submissions using a "Top 10" system (administrators can change the number to suit whatever ranking scheme they prefer). When using this system, the scoring can be set to rank from 1st (most preferred submission) to last as well as other ranking options to meet the needs of the specific review round. To read more about this customization, refer to the Eligibilty Tab options described below.  


Creating and Editing an Ordinal Ranking Review Task

To create the task select Add a new task from the appropriate stage in the workflow designer. Select the review group who will be completing this task from the list of groups in the drop-down then select Ordinally rank submissions as the task type. Confirming your selection by choosing Create Task will redirect to a new page where settings can be adjusted to meet the needs of the review round.


Basic Information Tab

The Basic Information tab provides a quick overview of the elementary descriptions (Title and Description) associated with the task, and can be formatted using the Visual Editor.


Name: The name of the task - how it will appear in the Competition Design page, and referred to in every other instance within your site. 

Set a Description: This description is shown to applicants who fill out this form. Specifying a description is optional. When checked, the Visual Editor is displayed, which allows the content to be formatted accordingly.


Round & Group Tab 

The Round and Group tab allows the administrator to select which stage the task will be in, as well as the group the task should be applicable to.


Stage: The specific stage the task should be affiliated with can be selected from this drop down menu. 

Group: The Group this task should be assigned to can be selected from this drop down menu. 


Behavior Tab 

The Behaviour tab presents critical options that affect the visibility of the task, as well as its importance.


Required: This option makes the task required to progress further in the contest. It is not possible to submit the application if a required task has not been completed. 

Viewable upon completion:  This option will determine whether the user can view their task after completing it. 

Set a deadline: Deadlines dictate when a user must complete this task by.  


Eligibility Tab 

The Eligibility tab determines which reviewer group should have access to this task. This is useful if your site has multiple reviewer groups, each with separate ranking forms.


This task should only be available for submissions in group: Select from the drop down which group submissions this ranking application should apply to. 

This task should be available to: Select from the drop-down which reviewers should have access to this task. 

Restrict to members of review team: If review teams are present on the site you can restrict the ranking task to only members of a specific team.

Submissions should be ranked by top: Numerically define what the scale should be for ranking submissions. 

Submissions should be handled as: Select how the overall ranking hierarchy should appear.
- First place gets highest last place gets lowest
- All submissions chosen get a score of 1 
- Only the first chosen submission receives a score of 1 

Make this task available to submissions in any round: Selecting this option allows the task to be completed regardless of which round the submission is in. 


Visibility Tab 

The Visibility tab allows the administrator to specify which groups this task will be visible to. 


Modify visibility: This option will allow you to hide this task from users in specific groups.


Completing an Ordinal Ranking Review Task

A reviewer will be able to rank submissions using an ordinal ranking system once the review task has been created and the submissions have been assigned to the reviewer. The reviewers will be able to review assigned submissions in their entirety within the review portal by selecting the submission titles. Once reviewed, the user can then select the Edit option of the review task which will navigate them to a new page where a list of the assigned submissions will be displayed and can be ranked in order from most favourable (1st of x) to least favourable.



Have more questions? Submit a request


Please sign in to leave a comment.