Fully Remote - Part Time - AI Domain - Software Engineering Positon

This is 10 week part time job with 5 hours per day and 2 days a week.

Job description

Company is partnering with an AI innovation initiative seeking Professional Software Engineers to support the evaluation and improvement of software engineering-focused AI tasks.

In this role, you ll use your engineering and systems-level reasoning to assess model competency in real-world technical domains, debugging, architecture decisions, and system behavior analysis.

What You ll Do

• Create challenging SWE-related prompts based on visuals, datasets, or model outputs.
• Evaluate AI-generated responses using provided rubrics.
• Develop or improve rubrics to ensure they reflect accurate reasoning for complex tasks.
• Review and stress-test model outputs for accuracy and logical consistency within your domain.
• Provide clear, structured feedback based on your subject-matter knowledge.

Ideal candidate profile

Experience using an LLM is ideal. They want someone who has been working in their actual field most recently, rather than AI training, but experience working with LLMs is a plus.

Daily tasks

Design problems and evaluate AI prompts based on the domain expertise. Provide thorough feedback.

Required skills

Need a bachelor s degree or higher, experience in related field.