Project Overview

Renaissance Philanthropy, in partnership with The Learning Agency,  is leading a project to develop three state-of-the-art benchmark datasets, leveraging CareerVillage.org, a platform that has crowdsourced career advice, answering over 60,000 questions for more than 3.5 million learners. This project – CareerNet – aims to enhance artificial intelligence's (AI) ability to guide users in navigating careers and accessing social benefits. The datasets will serve as benchmarks for AI models in the domains of reskilling, computer science occupations, and allied health occupations. 

CareerNet seeks graduate students and professionals in the fields of healthcare, computer science, or career advising who will serve as annotators to review the question and answer data, scoring it based on quality, and ensuring that the final data set can be deployed across platforms to deliver more accurate, personalized, and equitable career guidance to those who need it most.

Annotator Role and Responsibilities

Annotators will:

• Participate in 3-4 training and norming sessions on Zoom. 
• Read large volumes of questions and answers in their area of expertise (healthcare, computer science, reskilling in an area).
• Rate the completeness and correctness of the question and answer data to curate a strong dataset and ensure the data is useful for machine learning models. 

Annotator Qualifications

Annotators must:

• Have expertise in career advising or career pathways, which includes an understanding of the credentials required to obtain careers in these fields and the common pathways that career navigators may take, in at least one of the following domains:
1. reskilling and general career advising; 
2. computer science occupations; or 
3. allied health occupations. 
• Feel comfortable with independent work and working with set deadlines.
• Be authorized to work in the United States. 

Annotators should:

• Have an understanding of or experience with research methods, data analytics or data annotation. However, no understanding of machine learning is needed for this role.

Timeline & Commitments

• August 2025: Required initial training period; annotators will be expected to attend training calls to orient them to this work.
• The time commitment will be 10 hours per week or less during the training period
• August through November 2025:  Independent work period 
• The time commitment will be dependent upon each annotator’s availability during the week. We expect a minimum contribution of 10 hours per week.

Compensation

Annotators will be compensated commensurate with experience. 

Next Steps

If you are interested in being considered for the role of annotator, please complete this form.  If you have questions, please contact Jules King (jules@the-learning-agency.com).

Annotators must:

• Have expertise in career advising or career pathways, which includes an understanding of the credentials required to obtain careers in these fields and the common pathways that career navigators may take, in at least one of the following domains:
1. reskilling and general career advising; 
2. computer science occupations; or 
3. allied health occupations. 
• Feel comfortable with independent work and working with set deadlines.
• Be authorized to work in the United States.