Senior IAM Engineer

Who We Are:   CompTIA is a leading voice and advocate for the $5 trillion global information technology ecosystem; and the estimated 75 million industry and tech professionals who design, implement, manage, and safeguard the technology that powers the world’s economy.   Through education, training, certifications, philanthropy and market research, CompTIA promotes industry growth; the development of a highly skilled workforce and a commitment to creating an environment where innovation happens, and the opportunities and benefits made possible through technology are available to all.     Scope Summary;   The Senior Psychometrician plays a vital role by conducting timely psychometric analyses across the entire certification product portfolio, leveraging psychological and industry best practices while adhering to ISO 17024 standards. The analysis and reporting deliverables provide critical inputs to the entire high-stakes development process, program security posture, and overall validity and reliability of exam forms and all exam results. The role will need to communicate well, prioritize tasks to ensure deadlines are met, think outside the box, provide innovative solutions to complex problems, including new types of analyses, and participate in the development of technology solutions that will automate basic psychometric analyses. This person will collaborate with Exam Services, IT, Certification Operations and Security teams.   Essential Duties and Responsibilities;   * Responsible for the analysis of all exam results, the defensible construction of all exam forms and/or exams delivered leveraging CAT methodologies, and the identification of fraudulent activities via data forensic analyses. * Work with the Exam Services team members to confirm clear timelines for all analyses needed to launch new and updated certification products. * Produce timely JTA survey analyses and JTA reports, establishing draft blueprints for Governance Committee (GC) review and approval. * Conduct item and simulation analysis leveraging a combination of the most appropriate methodologies for the given product, based on available data and project needs: * Classical Test Theory (CTT) * Item Response Theory (IRT) * Rasch * Conduct analyses in support of standard setting, using SME Angoff ratings and providing Hofstee analyses during standard setting workshops. * Develop standard setting reports that help inform the cut score selection process and GC committee approval. * Conduct beta exam rescores leveraging relevant software tools and produce critical outcomes for comparison (audit) against test delivery provider results. * Build multiple equated forms (based on project criteria) and coordinate clear deliverables and reports to the relevant Exam Services team members to inform exam publishing. * Include a specific number of simulations. * Include a specific number of unscored items (for ongoing beta testing). * Generate scaled scoring tables to manage score equating in a confidential manner. * Generate reports in support of ISO 17024 standards. * Conduct various routine analyses across the entire portfolio based on quarterly volume, identify potential issues, and recommend solutions or process adjustments based on results: * General item statistics (exam health check), including distractor analysis, forms analysis, etc. * DPF analysis * DIF analysis * SSI analysis * RSI analysis * Others * Conduct proctoring modality comparisons across ISO 17024 accredited products, ensuring score validity and equivalent meaning of scores in both modalities. * Conduct large-scale analyses mid-lifecycle to produce larger numbers of forms in support of frequent forms publishing and ongoing product maintenance. * Conduct special data forensic analyses towards the invalidation of scores when applicable, protecting the brand while ensuring the validity of the overall program. * Conduct analysis of test center performance, producing z-score reports in support of test center policies and audit practices. * Conduct ad hoc or custom analyses as needed to identify problems and/or review fairness and flag potential bias or other issues that might be cause for concern. * Conduct analyses of translated exam forms, performing comparisons and making recommendations as needed. * Consult with the Exam Services team as needed to set up special research projects, and provide other consultative inputs to development processes, reports, etc. * Consult with Innovation team regarding the validity of emerging assessment types, assessment methods, learning offerings and new measurement systems/platforms.   Key Performance Indicators:   * Conduct analyses and produce reports consistently on time in relation to core certification development project timelines. * Provide clear, documented input and new requirements for platform certification delivery enhancements and/or certification database enhancements, leveraging automated statistical analysis. * Conduct scheduled data forensic analyses, providing recommendations for detection and remediation. * Engage during ANSI audits to address auditor’s psychometric questions to ensure CompTIA certification programs remain accredited. * Remain abreast of new industry trends, new statistical analyses and considers implementation of new methods to improve validity and reliability throughout the assessment process.   Additional Expectations:   * Employees are expected to model CompTIA's values and reflect them in how they work and collaborate. * Participate in required and role relevant training to maintain skills, ensure compliance, and support ongoing professional development. * May be required to travel based on role responsibilities and business needs. * Please note that this job description is not intended to provide an exhaustive listing of tasks, activities, or responsibilities required of the employee in this role. Tasks, responsibilities, and duties may change without notice.   Skills and Qualifications:   * Proficiency with a combination of preferred software tools (e.g. SPSS, R, SAS, Qualtrics, Tucker Equating, IRTPro, flexMIRT, etc) in support of all required analyses conducted throughout the development lifecycle. * Proficient with data visualization tools (e.g. Tableau, ggplot2, etc). * Excellent written and verbal communication skills * Attention to detail and accuracy in all work * Project management skills with the ability to manage multiple tasks, priorities, and deadlines * Analytical thinking and complex problem-solving skills * Ability to innovate and think creatively to build and deploy new processes and solutions * Advanced understanding of certification exam delivery platforms and various methods of delivery (e.g., fixed forms, LOFT, CAT, etc). * Experience participating in software development projects is a plus * Willingness to participate at industry conferences, presenting best practices and new methodologies, maintaining CompTIA’s leadership in the certification industry * Effectively communicate complex statistical concepts and results to non-technical stakeholders. * Demonstrate ethical judgment and integrity when using, manipulating and storing sensitive data, PII and other confidential information.   Education and/or Experience:   * Ph.D.in Psychometrics, Educational Measurement, Statistics or a related field. * 10+ years of direct experience in related field   Why Join Us:   We're a team of driven, creative problem-solvers who are passionate about empowering people to realize their full potential. We foster a collaborative and inclusive culture where ideas are welcomed, growth is encouraged, and success is celebrated. We're not just looking for employees, we're looking for partners who share our vision and are eager to contribute to our purpose. If you're ready to unlock your potential and reach new heights, join us today.    Our team works hard and we recognize the importance of taking care of our own. We offer our employees a comprehensive suite of benefit offerings including:   * Health, Dental, and Vision Insurance & FSA/HSA Plans * Performance bonus up to 20% of base salary * Unlimited PTO & 15 Paid Holidays * Flexible Schedules & Summer Hours * 12 weeks of Paid Parental Leave * Sponsored Costco or Sam’s Membership * 401K Retirement Plan with 6% company match * Spot Bonuses for going above & beyond * Tuition Reimbursement * Home Office Allowance * Wellness Reimbursement * Student Loan Repayment  * Broadband Stipend * Expected compensation based on experience and qualifications – $200,000 - $225,000

POSITION SUMMARY: As a Senior Software Engineer, AI Experience at Penn Foster Group, you will build AI-native products end-to-end — owning features that shape how millions of learners discover, enroll in, and progress through our programs. This is a hands-on full-stack role for an engineer who treats LLMs as core product infrastructure, designing retrieval pipelines, building agentic workflows, writing evals, and pushing features from prototype to production. You will collaborate closely with product, platform, and other engineering teams, mentor peers on agentic development practices, and help set the bar for what good looks like in an AI-first engineering org. This position is fully remote. ESSENTIAL JOB FUNCTIONS: * Design, build, and ship full-stack features end-to-end using TypeScript, Vue.js, and Node, deployed on Azure. * Write evals and observability for LLM features. * Use agentic coding tools (Claude Code, Cursor, or equivalent) as your primary development environment for planning, executing, and reviewing AI-generated work to ship faster while maintaining clean code that is ready for change. * Take ownership of AI-powered content authoring and management features. * Integrate model APIs (Anthropic, OpenAI, others) with attention to tokenization, latency, cost control, prompt-injection risk, and graceful degradation when models fail. * Collaborate closely with product, platform, and sibling engineering teams to translate ambiguous problems into shippable systems. * Lead code reviews, set technical standards for AI-assisted development, and mentor engineers on prompting, context engineering, eval design, and agentic workflows. * Ensure scalability, security, observability, and unit economics of the systems you own. KNOWLEDGE, SKILLS, ABILITIES: * 5+ years of experience building customer-facing products at a high-quality software company. * Strong UI and TypeScript fundamentals, with experience across the full stack (Browser technologies, Node, PostgreSQL). * Track record of driving complex, end-to-end features (not just incremental improvements and refactors) with visible impact on the product. * Product sensibility: you care about UX, speed, and polish, and you push back when something doesn't feel right. * Comfortable working without heavy PM overhead; you can help to shape the problem, not just implement a solution. * High ownership mentality: self-directed, full-lifecycle builder who takes a feature from idea to shipped. * Thrives in a remote, async-first environment with a lean, high-output team. * Experience at a startup or a company with a genuinely high engineering bar. WORKING CONDITIONS & HIRING PROCESS:  * U.S. Work Location Requirement: All employees must reside in and perform work within the United States. We do not support international remote work arrangements for this role. * Travel Expectations: While this is a remote position, occasional travel may be required for team meetings, company events, conferences, summits, etc. Candidates must be willing and able to travel as needed. * On-Camera Work Environment: We operate in a highly collaborative, remote, on-camera culture. Employees are expected to be on camera during meetings unless extenuating circumstances apply. * Recorded Video Interviews: All video interviews conducted during the hiring process will be recorded to support internal hiring consistency and process improvement. * Background Checks: Employment with Penn Foster Group is contingent upon successfully completing applicable pre-employment screening requirements, which may include verification of employment history, education, and criminal background, where permitted by law. * Identity Verification (Form I-9): As part of our onboarding process, all hires must complete employment eligibility verification in compliance with federal law. This includes remote Form I-9 verification, which may require an in-person identity verification step with an authorized representative.   About Us: At Penn Foster Group, we are transforming online learning to help learners by uniting Penn Foster, CareerStep, Ashworth College, James Madison High School, the New York Institute of Photography, the New York Institute of Art and Design, and other education platforms. Together, we create an accelerated path to greater economic mobility through real-world skills and knowledge that enable them to achieve long-term success in the workplaces of the future. Our history dates back to 1890 when our founder, Thomas Foster, pioneered distance education by offering training by mail for coal miners to get the necessary skills for safer jobs. Today, with the partners who use our education and training programs, we continue that mission of providing accessible training and education for in-demand skills and are building a workforce that’s prepared for the future job market. Equal Employment Opportunity: We strive toward Diversity, Equity, and Inclusion at Penn Foster Group by intentionally building teams that are diverse – in identities, lived experiences, and ideas to create a culture where people feel connected to each other and have a sense of belonging. We value diversity, equity, and inclusion because it is the foundation that enables us to achieve what we set out to do as an organization – from maximizing the number of learners who can reach their goals while giving them the kinds of experiences we want them to have, to becoming the type of company we want to work in. What We Offer: We offer a robust benefits package that includes medical, dental, vision, flexible spending, generous paid time off, sponsored volunteer opportunities, a 401K with a company match, plus free access to all of our online programs.