Senior QA Engineer

ABOUT BEAMERY With the rapid expansion of AI and automation, the future of work has never been more challenging for organizations. Beamery’s unique jobs, skills and tasks data platform helps organizations navigate these challenges and make more informed decisions across Talent Lifecycle Management. Our solutions power recruitment, mobility, upskilling, diversity, work architecture and workforce planning for some of the world’s most forward-thinking companies. We believe that where you work is much more than just a job. Millions of people are being left behind every day in their careers, and we’re on a mission to fix this by creating equal access to meaningful work, skills and careers for all. We are an equal opportunity employer committed to building a representative Beamery, creating an equitable, inclusive and engaging environment for our people. So, what are you waiting for? Join us and help us transform the future of work once and for all. What’s ahead — and why it’s an exciting time to join the team:  Deepening our native integrations with SAP, Workday, Microsoft, and LinkedIn to seamlessly embed our skills intelligence into the platforms where critical workforce decisions are made. Embedding our agentic AI to help customers plan smarter for the future—powering workforce strategies, internal mobility, and skills forecasting. Advancing our use of proprietary LLMs and knowledge graph technology to help organizations unlock broader talent pools, make fairer decisions, and expand access to opportunity at scale. But it’s not all about creating high-quality products, we also very much value the company culture we have worked hard to create; built on trust, empathy & honesty ensuring our workforce is able to bring their full selves to work. ABOUT THE ROLE Beamery’s Job Architecture product uses AI to generate structured role profiles, including skills, tasks, proficiency levels, human agency, and priorities, at scale for some of the world’s largest enterprises. These outputs go directly into our customers’ talent platforms, so accuracy and consistency matter in everything we deliver. As a Senior QA Engineer, you’ll be the person who checks and improves the quality of job architecture outputs we produce. You’ll work closely with our Data Science and Professional Services teams to review AI-generated data, spot issues, fix errors, and make sure each project is delivered accurately and on time. This is a great role for someone who enjoys working with data,using AI agentic tools (e.g. Claude), has a sharp eye for detail. You’ll also help build and document the QA processes we use, so you’ll have a real say in how things are done. WHAT WILL YOU DO AS A SENIOR QA ENGINEER AT BEAMERY? REVIEW AND IMPROVE AI-GENERATED OUTPUTS * Check AI-generated role profiles against our quality standards, looking at skills, proficiency levels, and priority assignments to make sure they’re accurate and consistent. * Spot errors, gaps, or inconsistencies in the data and fix them, making sure the overall structure stays intact. * Work with the Professional Services team to manage the QA stage of each project, keeping things on track and within agreed timelines. * Review Beamery’s Taxonomies and ensure the accuracy of Skills, Tasks and Role titles. * Use AI agentic tools like Claude to review the output and implement deterministic agentic flow to accelerate and scale Beamery’s QA activities. * Flag anything that needs a decision before sign-off and communicate clearly about what changes you’ve made and why. COLLABORATE WITH DATA SCIENCE * Work with the Data Science team on defining, implementing and tracking QA metrics to enhance the AI output quality. * Keep a clear record of the issues you find during QA, what went wrong, how often it happens, and what’s likely causing it, so the Data Science team can improve the AI pipeline. * Help test pipeline updates between projects to check that fixes work as expected and don’t introduce new problems. * Turn your QA findings into straightforward recommendations that the team can act on. HELP BUILD OUR QA PROCESSES * Write and maintain documentation for our QA workflows so that new team members can get up to speed quickly. * Spot gaps in our current quality checks and suggest improvements, whether that’s adding new review steps, refining how we measure quality, or updating what “good” looks like. * Help shape our quality standards as the product and customer needs evolve. WHO ARE WE LOOKING FOR? * Experience (3+ years) in QA, data analysis, or a similar role where attention to detail and data accuracy mattered. Internships and placement years count. * Experience (1+ years) in using Agentic tools for QA. * Basic knowledge in Agentic technologies. * A sharp eye for detail, you’re the kind of person who spots the inconsistency everyone else missed. * An analytical mindset, when something looks wrong, you don’t just fix it; you want to understand why it happened so it doesn’t happen again. * Good organisational skills, you can manage your own work across a few projects at once without things falling through the cracks. * Clear communication, you can explain what you found and what you changed to both technical and non-technical people, in writing and in conversation. * Comfort with ambiguity, some of our processes are still being built out, and you’re happy to help shape them rather than waiting for a playbook. This is not a checklist of requirements, don’t worry if you don’t tick every box. We’d still love to hear from you. NICE TO HAVE * Any exposure to AI-generated content, NLP outputs, or taxonomy/classification systems. * An interest in HR technology, talent management, or how organisations think about skills and roles. * Experience defining what “good” looks like for data, for example, writing acceptance criteria or quality metrics. * You’ve helped set up a process or workflow from scratch, even informally. WHY THIS ROLE? This is an early hire in a growing team. As the first QA Analyst focused on job architecture at Beamery, you’ll have a real impact on how quality is defined and maintained across the product. You’ll sit at the intersection of Data Science and customer delivery, and your work will directly shape the outputs that enterprise clients rely on. If you care about getting the details right, want to work with AI-powered products, and are looking for a role where you can grow and make an early mark, we’d love to talk. Beamery is for Everybody. Diversity and open expression are fundamental to us. We acknowledge the challenges in our industry and strive to develop an inclusive culture where everybody can contribute. We are dedicated to creating an inclusive environment for everyone, regardless of ethnicity, religion, color, sexual orientation, gender identity, race, national origin, age, disability status, or caregiver status. If, for whatever reason, you need us to make reasonable adjustments and adaptations to our recruitment process, please email accommodations@beamery.com Visit our Diversity, Equality and Inclusion page to learn more about progress and commitments.

YOUR MISSION We're a fast-moving, lean EU team looking for a high-agency QA engineer based in Kosovo, with zero middle management. You'll work alongside our founder and our Kosovo engineering team, report directly to the Head of Engineering, and own quality end-to-end. We already have a Playwright/TypeScript E2E suite covering our core flows, but it needs a real owner. Your job is to own quality across the whole product: our Next.js web app, our backend APIs, and the AI document-extraction engine underneath. You'll broaden coverage across the app, make the suite trustworthy (fast, stable, and run on every change), and extend it into an AI-driven evaluation pipeline, so neither an everyday feature change nor a prompt or model change can silently break what we ship. Note: CVs filled with LLM slop will be immediately disqualified. We want to see your actual experience in your own words. WHAT YOU'LL DO * Own quality across the product: Full E2E and regression coverage of our Next.js frontend and backend APIs, from critical user journeys to the edge cases, so every release ships with confidence. * Make the suite trustworthy: Take ownership of our existing Playwright/TypeScript suite, drive down flakiness, and get it green and trusted as a CI gate the team relies on. * Harden the email → extraction → downstream pipeline: We have early coverage of our email-to-document flow. You'll turn it into the rock-solid regression backbone that protects extraction accuracy and downstream processing. * Build AI evaluation, not just pass/fail: Use Langfuse to build eval harnesses for extraction quality, golden datasets, scoring against ground truth, and drift detection that gates on quality, not just green checks. * Use AI to test faster: Lean on AI to generate cases and synthetic documents, triage failures, and review diffs, wherever it makes the suite broader and faster than hand-written tests alone. * Make testability a first-class concern: Partner with engineers so features ship testable by design, with clear, actionable failure reporting. YOUR PROFILE * Full-stack app testing: You test real web apps end to end, UI flows, API contracts, and data integrity, and you know what makes a suite flaky and how to fix it. Strong with Playwright and comfortable in TypeScript and/or Python. * 3+ years in QA / test engineering: We hire experienced QA only. What matters is depth of ownership: you've built and run automation strategy and frameworks yourself, not just executed someone else's test plans. A sharp engineer who's shipped real automation beats a coaster with twice the tenure. * High Agency: You don't wait for perfectly scoped tickets. You unblock yourself, make pragmatic decisions, and take extreme ownership of quality. * AI-first instincts: You actively use AI in your testing workflow, and you're keen to take on the harder problem of testing non-deterministic AI output, or can show real aptitude for it. NICE TO HAVE * Experience testing non-deterministic systems: evaluating LLM/ML outputs, building eval sets, reasoning about quality metrics over pass/fail. * Data/ML pipeline testing or MLOps background. * Document processing, OCR, or data-extraction domain experience. * Familiarity with Prisma, async/queue-based systems, or complex event patterns. WHY US? * High Ownership & Influence: You'll define how Holocene does QA. Your test strategy directly protects the product and the roadmap. * Elite, Lean Team: Work with highly technical peers in an environment where experimentation is expected and red tape is nonexistent. * Fast Decisions: We respect your time. Just 2 interviews, with a final decision within 48 hours.