AI scored recommendations

Recommendations is an AI scored feature that uses artificial intelligence (AI) to evaluate and score interview responses submitted by candidates for on-demand video interviews, and then presents the resulting ratings and recommendations to recruiters and hiring managers.

Prior to use, AI scored must be licensed at the application level. Once licensed, you can add AI scored questions associated with specific job competencies to on-demand video interview stages and invite candidates to submit on-demand video interviews for that stage.

Candidates are given the choice to consent or opt out of the use of AI in their interview. If candidates consent, the interview responses they submit are evaluated and scored by the AI model and the resulting indicators and recommendations are presented to recruiters and hiring managers on the candidate review page.

Adding AI scored questions to a stage

Once AI scored is licensed, you can add AI scored questions from the AI scored question library to interview stages. The AI scored question library provides a set of questions that will be scored by machine learning and used to produce a recommendation at the review level. This does not replace the reviewer's input; it is a recommendation engine that provides reviewers with the information and insight they need to make good hiring decisions.

Video: "Matching the right AI scored questions to the right role" with Chief Science Officer Mike Hudy (1:46)

To add AI scored questions to an on-demand video interview stage

1. Create the job to which you want to add AI scored questions. 
2. Define an on-demand video interview stage for the job.

3. Within the on-demand video interview stage, select Interview Questions, and choose Add AI scored Question.

   

4. Use the Narrow By filter options to filter questions by Roles, Competency, or Response Type. For more information about these options, see AI scored question library. Select the question(s) that you want to add to the stage and then click Add Questions to Stage.

   Note: Filter options are subject to change as gathered data is analyzed. Different categorizations or filtering schemes may be identified and therefore look different over time. In addition, the question set may grow and potentially change over time.

5. Repeat this process until the question set is complete within the interview stage.

Working with AI scored questions in the stage

AI scored questions within an interview stage are marked with an icon and accompanying text that reads "Automatically Scored." When you are working in the stage you can reorder or remove these questions, but you cannot edit them. If desired, you can also add new automatically scored questions to the stage. The following image depicts the Interview Questions (Questions & Answers) panel, with the second question in the list identified as an AI scored question.

Once you finish defining the job stage that contains the AI scored questions, you can invite candidates to the interview. Candidates are given the option to decline the use of AI in their interview. If candidates decline, they will still be required to answer the AI scored questions. However, the questions will not be scored, nor will a recommendation be provided during the Review phase.

The candidate experience

When candidates are invited to complete an on-demand video interview, they receive an email invitation with a link to log in to Hirevue and begin their interview. After they log in, candidates who have been invited to interview stages that contain AI scored questions are presented with a consent page that describes how Hirevue uses AI. Candidates are asked to provide their consent to being evaluated using AI before they continue.

If candidates select:

• Yes, I consent, they begin the interview process. When they've completed and submitted their interview, AI scored evaluates their responses and provides recommendations for recruiters and hiring managers to review.
• I do not consent, they begin the interview process. The candidate has the same experience as the candidate who opts in to the AI evaluation. However, the AI model will not evaluate their responses nor will it provide recommendations when recruiters and hiring managers review the interview.

See example of consent page

Reviewing AI scored recommendations

The review process for interview stages that contain AI scored questions is the same as for those that contain non-scored questions. Recruiters and hiring managers are notified by email when the candidate's interview is ready for review. Reviewers click a link in the email to access the Review page, where they can review the candidate response and provide their feedback.

Once they are on the Review page, reviewers can select an interview question. For non-scored questions, reviewers can provide feedback as required. For AI scored questions, the recommendation is provided. They can either keep the suggested rating (if they agree) or provide a new one. The recommendation generated by the AI scored model does not take the place of the reviewer's response; however it does provide a benchmark for reviewers to refer to when evaluating the candidate and determining their rating.

If there are no suggested ratings, one of the following has occurred:

• Transcription accuracy did not meet the minimum threshold: If the interview transcription is projected to be less than 70% accurate, it will not be used to score the interview.
• Response was too short: If the candidate response was less than the minimum threshold of 150 characters, it will not be transcribed and used to score the interview.
• Candidate did not consent: Candidates opted out of the use of AI on the Consent page they reached after login.

Note: If licensed, AI scored-Recommendations and AI scored-Ranked Recommendations now support multiple languages. For more information, see AI scored language support.