On this page you will find detailed instructions for scoring applications. Watch the Reviewer 101 Video to walk through scoring an application and helpful tips. If you are looking for information on how to use the portal and tech features, please review the Site User Instructions.
Thank you for helping to review the NCWIT Collegiate Award Finalist Round applications! The applicants are women in any year of college who are majoring in computing and related fields. We are looking for applicants with outstanding technical accomplishments, demonstrated by projects that showcase their skill, mastery of the subject and expertise. Higher scores should be awarded to more innovative, impactful and difficult technical projects where the applicant was able to define her role clearly and confidently.
Please keep the following important guidelines in mind when scoring each application:
- Applicants are broken into three categories:
- freshman, sophomore or two year degree program
- junior or senior
- graduate student
- Applicants will, and should, be at different levels of technical experience and expertise. Please take the “grade level” into consideration when viewing the presentation, scoring the technical questions and reviewing the applicant as a whole. For example, try to avoid scoring a college freshman low just because her project doesn’t yet exhibit the technical savvy of a Ph.D. student’s project. Score them against their peer-group expectations, to the extent possible.
- Applicants may not have English as their first or native language. Please do not penalize such applicants for minor syntax errors.
- Applicants have been asked to explain why their technical project is innovative or impactful. We urge you to put evidence of technical ability at the heart of your scoring. Please attempt to use your best understanding of the technology addressed to assess “how hard” it was to attain their accomplishment - rather than how well they presented it, or how “cool” it is. A project contribution that uses technology to solve a problem in a whole new way ought to be scored higher than something socially-positive or cute, but not that difficult to accomplish (technically).
- If you feel like an application contains a level of technical depth that exceeds your knowledge in the subject area (remember, some are Ph.D. candidates), please feel free to abandon your review and leave it to others. To do so, simply leave the Review page without saving the review (i.e., go “Back” in your browser), do not select the “Discard Review” button.
- You may be assigned an application for a student you personally know, or who attends an institution to which you have personal connection. If you think that your relationship with the applicant or her college/university might influence your review, you should consider it a conflict of interest. In that situation, please do not complete your review of the application. Simply leave the Review page without saving or submitting the review (i.e., go “Back” in your browser), do not select the “Discard Review” button.
- We encourage you to review comparatively. As you start to review, save your first few applications as drafts before submitting the reviews. This will allow you to have a better understanding of how each application compares to the overall pool of applicants. Do not forget to go back and submit all of your draft reviews!
PRELIMINARY ROUND APPLICATION REVIEW QUESTIONS
Applicant Prompt: Please provide a thorough description of the project. Explain the problem the project solved, and how, without requiring the reader to be an expert with deep background knowledge.
Reviewer Prompt: Please utilize the point descriptors below to score the Project Description.
- The applicant DOES NOT PROVIDE a description of the project. The problem solved, and how it is solved, is not explained. The reader, without deep background knowledge, would not understand the project.
- The description provides an ADEQUATE description of the project. The problem solved, and how it is solved, is explained adequately. The reader would need a level of expertise and some background knowledge to understand the project.
- The description provides a THOROUGH description of the project. The problem solved, and how it is solved, is explained thoroughly. The reader would not need to be an expert or any background knowledge to understand the project.
Applicant prompt: Whether you chose the project or were assigned to it, please describe the aspects that were innovative and interesting.
Reviewer Prompt: Please utilize the point descriptors below to score this essay.
- The project lacks any aspects that are innovative or interesting.
- The project has a LIMITED amount of innovation and interest.
- The project has a MODERATE amount of innovation and some interesting aspects.
- The project is CLEARLY innovative and interesting.
- The project has REAL INNOVATION and has a high number of interesting aspects.
Applicant prompt: In your opinion, why is this project important and what impact will it have?
Reviewer Prompt: Please utilize the point descriptors below to score this essay.
- The project has NO real potential to make an impact in the field.
- The project has the potential to make a SMALL impact in the field.
- The project has the potential to be MODERATELY impactful in the field.
- The project has the potential to, or CLEARLY does, impact the field.
- The project has the potential to SIGNIFICANTLY impact the field.
Applicant Prompt: Clearly describe each of the technical skills that you utilized in the creation of this project and explain why each skill was vital to the success of the project.
Reviewer Prompt: Rate the technical complexity of the project based on technical skills used, along with overall description of the project. Please utilize the point descriptors below to score this essay.
- The project has NO CLEAR technical complexity or difficulty. The skills described are very basic in nature.
- The project has a relatively LOW LEVEL of technical complexity or difficulty. The skills described are a mix of basic skills and intermediate skills.
- The project has a MODERATE LEVEL of technical complexity or difficulty. The skills described are intermediate skills.
- The project has a HIGH LEVEL of technical complexity or difficulty. The skills described are a mix of intermediate and advanced skills.
- The project has an EXTREMELY HIGH LEVEL of technical complexity or difficulty. The skills described are highly advanced skills.
Applicant Prompt: Referencing your answers above (in the multiple choice section), please describe your individual contribution to this project.
You may refer to these items to view the student’s self-assessed contribution:
Reviewer Prompt: The suggested score for project contribution is provided above and is based on the multiple choice questions under "Project Contribution"".
Endorser Prompt: Please provide us with your relationship to this student and a frank assessment of the project and her technical contribution to it.
Reviewer Prompt: The Project Endorsement is a mandatory component of the application. Please keep in mind that Endorser statements vary greatly in length and content. Please utilize the point descriptors below to score the Project Endorsement.
- Endorsement DOES NOT PROVIDE information on the project and her technical contribution.
- Endorsement provides ADEQUATE information on endorser relationship, the project, and the applicant's technical contribution.
- Endorsement provides THOROUGH information and STRONG SUPPORT for the applicant's project and her contribution to it.
Preliminary Summarization of Student
Reviewer Prompt: Considering this application in its entirety, please indicate your overall recommendation for this applicant. Recommended applicants may be invited to participate in the Finalist round for the NCWIT Collegiate Award.
- Application is VERY WEAK and SHOULD NOT BE CONSIDERED AT ALL for the Finalist round of the NCWIT Collegiate Award.
- Application is WEAK and SHOULD NOT MOVE FORWARD to the Finalist round of the NCWIT Collegiate Award.
- Application is ADEQUATE and SHOULD MOVE FORWARD WITH RESERVATIONS to the Finalist round of the NCWIT Collegiate Award.
- Application is STRONG and SHOULD MOVE FORWARD to the Finalist round of the NCWIT Collegiate Award.
- Application is EXTREMELY STRONG and SHOULD MOVE FORWARD WITHOUT RESERVATION to the Finalist round of the NCWIT Collegiate Award.
*Reviewer Comments: Reviewer comments are required. Your comments are invaluable and make a big difference during the selection phase. Please note comments and scores will not be shared with applicants, they are solely to be used for award selection.