About the role
At Workera, our assessments follow a rigorous assembly process and our computerized adaptive test (CAT) is developed to orient learners throughout their careers. We're seeking a Psychometrician to handle tasks like psychometric analysis, competency modeling, item bank maintenance, scoring, and reporting. The role demands flexibility, initiative, and comfort in a fast-paced setting. You will collaborate closely with assessment, learning, and product teams to create valid assessments, enhance our platform, and ensure a superior user experience. You'll use your psychometrics expertise to influence UX design, integrate IRT models with deep knowledge tracing, and build our platform's psychometric infrastructure. This position offers the chance to lead internal projects and significantly contribute to improving our product by integrating engaging, formative assessment with the learning experience.
About your team
The Assessments team is composed of psychometricians, assessment program managers, and assessment developers who are subject matter experts in areas such as AI, data science, and software engineering. We own the development, administration, analyses, and maintenance of Workera’s catalog of assessments.
What You’ll Do:
Within 3 months, you will:
- Conduct common psychometric analyses, such as classical item analysis, IRT item calibration, and evaluating assessments for item fit, person fit, and dimensionality.
- Maintain technical documentation and dashboards related to assessment development and psychometric analyses for internal and external stakeholders.
Within 6 months, you will:
- Perform psychometric research, such as analyses to support and improve our CAT, and designing studies to establish validity evidence for our assessments.
- Support product teams by serving as an internal psychometric consultant to improve Workera’s assessment quality and user experience. You will balance assessment development and psychometric best practices with UX and product design considerations to support new product features, including the implementation of alternative item types and automated psychometric analyses.
- Manage item banks, including monitoring and evaluating item banks, performing gap analyses, developing crosswalks between assessment versions, creating item writing assignments, and monitoring item exposure rates and statistical anomalies.
Within 9 months, you will:
- Support client custom domain development, including providing guidance on assessment design, and performing statistical analyses, calibration, scoring, standard setting, and reporting.
- Lead projects to improve our assessments and product, including different measurement models, item types, and assessment design.
*Please note as we are a dynamic and quickly growing scale-up, things are always subject to change*
What You’ll Bring
- Command of assessment and measurement best practices. Knowledge of the Standards for Educational and Psychological Testing (AERA, APA, & NCME, 2014) and/or NCCA standards is required.
- Comprehensive knowledge of psychometrics. Leverage experience in item response theory (IRT) and classical test theory (CTT) measurement models, including item analyses, calibration, scaling, and equating. Experience with computational psychometrics preferred
- Experience with the development and scoring of alternative item types. This includes matching, short answer, essay, coding, scenario, and/or situational judgment tasks
- Experience conducting validation studies. This includes establishing evidence of content, construct, and criterion-related validity
- Experience with computer adaptive testing (CAT). Experience with CAT and automated test assembly methods required
- Strong programming skills. Command of Python or R (particularly IRT packages) required. Experience with SQL and Git preferred.
- Strong communication and meeting facilitation skills. Experience guiding and facilitating meetings of SMEs, and ability to communicate technical information to non-technical audiences
- Experience working with product teams is a plus, not required.
- Experience in talent assessment is a plus, not required
- Master’s degree or PhD in I/O Psychology, Educational Measurement, Psychometrics, and/or related-fields.
- 2-3 years of applied psychometrics experience
- Proficiency in utilizing AI tools such as ChatGPT or equivalent digital assistants, applying them effectively within your work context.
Workera is a fast-growing, Series B Silicon Valley start-up redefining how enterprises understand, develop, and mobilize talent. Workera’s skills intelligence platform empowers leaders to make better, more informed talent development decisions. Utilizing computational psychometrics, machine learning, and AI technologies, Workera delivers best-in-class computer adaptive assessments with hyper-personalized learning plans to global companies across all industries. Our clients include Samsung, Siemens Energy, and the US Air Force.
Our founder is Kian Katanforoosh, an award-winning Stanford Computer Science Lecturer who has taught AI to over 1 million people, and our Chairman is Dr. Andrew Ng, co-founder of Coursera (NYSE: COUR), CEO of DeepLearning.AI, and founding lead of the Google Brain project.
We’re learners, dreamers, and game-changers. Join us.
At Workera we are committed to providing an environment of mutual respect where equal employment opportunities are available to all applicants without regard to race, color, religion, sex, pregnancy (including childbirth, lactation, and related medical conditions), national origin, age, physical and mental disability, marital status, sexual orientation, gender identity, gender expression, genetic information (including characteristics and testing), military and veteran status, and any other characteristic protected by applicable law. Workera believes that diversity and inclusion among out employees is critical to our success as a company, and we seek to recruit, develop and retain the best and most talented people from a diverse candidate pool. Selection for employment is decided on the basis of qualifications, merit, and business need.