Washington State Licensed Agentic Judge

To support a tech real estate company, the part-time remote Washington State Licensed Agentic Judge will evaluate AI/ML processes related to verbal interactions in real estate, ensuring quality standards and providing feedback to optimize model performance. Key responsibilities Review and evaluate annotated data to ensure it meets quality standards and project requirements Provide feedback and guidance to engineers to improve accuracy and optimize model performance Develop and maintain annotation guidelines and best practices for training data Required qualifications Minimum of 3+ years of active experience as a licensed real estate agent or broker Demonstrated interest and experience in reviewing or evaluating AI-generated content Bachelor's degree in English, Linguistics, or a related field Experience with Natural Language Processing projects, focusing on transcriptions or localization Familiarity with tools and software used for Natural Language Processing annotation is a plus