AI scored ranked recommendations

AI scored recommendations is an AI scored feature that uses artificial intelligence (AI) with predefined AI scored question sets to evaluate and score interview responses submitted by candidates for on-demand video interviews. AI scored question sets are predefined sets of questions that measure competencies critical to success for a wide variety of jobs. AI scored question sets can be used alone or paired with non-AI questions within an on-demand video interview or Virtual Job Tryout^® (VJT) workflow.

Prior to use, AI scored must be licensed at the application level. Once licensed, you can create new jobs from AI scored job templates that include questions associated with the specific job competencies you selected for that job. Once the job is created, you can add AI scored or standard questions to the job, configure additional stage settings, and invite candidates to submit on-demand video interviews for that stage.

Candidates are given the choice to consent or opt out of the use of AI to evaluate their interview. If candidates consent, the interview responses they submit are evaluated and scored by the AI model and the resulting score is presented to recruiters and hiring managers on the candidate review page.

Creating a job using AI scored question sets

You can build on-demand video interview stages using job templates that include AI scored question sets. AI scored question sets contain questions designed to evaluate candidates for specific job competencies and are broken down by job category and job level. When you create a job using an AI scored job template, the job is built with an on-demand video stage containing a Hirevue assessment for that question set.

Working with the stage

Jobs created using AI scored question sets contain a Hirevue assessment for the question set within the on-demand video stage. To view the assessment, open the Job Summary for the stage, and then select the Assessment link within the Interview Settings. The job template that you selected to create the job appears as the name in the Assessment drop-down box (see below). To view the questions that candidates will be asked as part of this assessment, refer to the AI scored question sets page, and look for the assessment name identified in the Assessment field.

Note: As with any Hirevue assessment, you are not able to make changes to the assessment options shown. For more information, contact your client administrator.

Once you finish defining the job stage that contains the AI scored questions, you can invite candidates to the interview. Candidates are given the option to decline the use of AI to evaluate their interview. If candidates decline, they will still be required to answer the AI scored questions. The questions will not be scored, however, nor will a recommendation be provided during the Review phase. Recruiters and hiring managers will need to evaluate and provide an overall score for candidate responses on the question set.

The candidate experience

When candidates are invited to complete an on-demand video interview, they receive an email invitation with a link to log in to Hirevue and begin their interview. After they log in, candidates who have been invited to interview stages that contain AI scored questions are presented with a consent page that describes how Hirevue uses AI. Candidates are asked to provide their consent to being evaluated using AI before they continue.

If candidates select:

Yes, I consent, they begin the interview process. When they've completed and submitted their interview, AI scored evaluates their responses and provides recommendations for recruiters and hiring managers to review.
I do not consent, they begin the interview process. The candidate has the same experience as the candidate who opts in to the AI evaluation. However, the AI model will not evaluate their responses nor will it provide recommendations when recruiters and hiring managers review the interview.

See example of consent page

Reviewing AI scored recommendations by rank

When candidates complete a ranked recommendations interview, Hirevue automatically scores the assessment and displays the resulting score on the candidate card within the interview stage. To help direct your attention first to those candidates that require attention, and then second to the candidates who received the highest assessment score, Hirevue prioritizes candidate cards as follows (default):

Candidates who do not have an assessment score and require a manual review by the recruiter.
Candidates with the highest assessment score first, followed in sequential order by candidates with lower scores. The candidate with the lowest assessment score appears at the bottom of the candidate card list.

If you want to change the order in which candidate cards are displayed in the interview stage you can either:

Select the Status or Assessment sort options in the top right of the page. Hirevue will sort and display candidates according to the option you select.
Use the filter options in the left pane of the page to display only those candidates that meet the specified criteria.

The following image displays an interview stage that uses AI scored ranked recommendations, with the assessment sort order selected. Because the default view has been changed to assessment, the candidates with the highest score appear at the top of the list.

1 - AI scored ranked recommendations score

Automated score generated by Hirevue AI models after evaluating the candidate's response. Candidates with the highest score will be positioned at the top of the list by default (or when you sort the list by assessment).

2 - Reviewer rating and recommendations

Average reviewer response score. This score is generated solely by reviewer scoring and is unaffected by AI scoring algorithms.

3 - Unable to score

Indicates that the candidate response was unable to be scored using AI scored. This can happen for one of the following reasons:

Transcription accuracy did not meet the minimum threshold: If the interview transcription is projected to be less than 70% accurate, it will not be used to score the interview.
Response was too short: If the candidate response was less than the minimum threshold, it will not be transcribed and used to score the interview.
Candidate did not consent: Candidates opted-out of the use of AI on the Consent page they reached after log in.

4 - Star rating and recommendation

Running average of the ratings and stars and thumbs up/ thumbs down reviews submitted for the candidate. Thumbs up/ thumbs down ratings are rounded to the positive. For example, if there is one "thumbs up" and one "thumbs down" rating for an interview, then the average rating will be "thumbs up".

Reviewing candidate interviews

Recruiters and hiring managers are notified by email when the candidate's interview is ready for review. Reviewers click a link in the email to access the Review page, where they can review the candidate response and provide their feedback.

On the Review page, information is divided into tabs on the left side of the page. Select the links below to view the tabs and the information each contains.

Summary

Provides an overall assessment score, along with specific competency scores and definitions.

If Hirevue is unable to score the candidate's response, the Summary page shows an explanation under the Candidate profile that it was unable to be scored. if this occurs, the candidate is limited to the stars rating assigned by reviewers.

Assessment responses

Displays the questions to which the candidate responded, along with their response. The response varies depending on the assessment response type. For example, if the candidate responded via a text response, the text is displayed. If the candidate responded with video or audio, those files are available for review.

Evaluating the candidate

Lets you select a star rating and recommendation for the candidate after reviewing their responses. These ratings will be presented on the interview stage candidate card.