Jobs›Automation Product, Bangalore

Automation and AI product testing

Nasugroupcom · Bangalore
Pay₹9L–10La year, as listed
WhereBangaloreKarnataka
TypeFull time3-6 years
Posted5 Nov13 days ago, via the company site
Kaam checked
No fee, deposit or pay-to-apply signs
Day work
No vehicle or licence needed
Skills they list5 named
PlaywrightevalsgoldenLarge Language Models (LLM)AI product testing
About this job

Job Summary

We are looking for an experienced AI Automation Tester with hands-on experience in testing AI-powered products and applications. The ideal candidate should have strong expertise in Playwright automation, along with practical experience in LLM evaluations (Evals), Golden Datasets (Goldens), and AI Observability.

Key Responsibilities

Design, develop and execute automated test scripts using Playwright.

Test AI-powered applications, LLM-based products and Generative AI solutions.

Work on Evals to assess AI response accuracy, relevance, quality and consistency.

Create, maintain and validate Golden Datasets (Goldens) for benchmarking AI outputs.

Implement and work with AI Observability to monitor model performance, response quality, latency and failures.

Perform functional, regression, API and end-to-end testing.

Identify, report and track defects to ensure product quality and reliability.

Never pay to get work. If a listing asks for a fee, it is a scam. The ten signs →

Apply on the company site
Opens cutshort.io in a new tab