All Jobs
Senior AI Quality Engineer | Horizons Team

Senior AI Quality Engineer | Horizons Team

Posted 4 days ago

Job Description

Join Hostinger, and we’ll grow fast! 🚀

We’re shaping the future of online success - powered by AI and driven by people. With 900+ talented professionals and over 4 million clients in 150 countries, we help creators and entrepreneurs bring their ideas to life faster and easier than ever before.

Our mission: To provide tools that help individuals and small businesses succeed online faster and easier.

Our culture: Guided by 10 company principles.

Our formula for success: Customer obsession, innovative products, and talented teams.

Your role at Hostinger

Join Horizons - the team exploring and building AI-powered products that don’t have industry playbooks yet.

As a Horizons Quality Engineer, you won’t just test features - you’ll define what “correct” means for AI systems that didn’t exist three months ago. This isn’t a traditional QE role with predefined test cases or stable requirements.

You’ll operate in high ambiguity, where the product evolves weekly and the rules are still being written.

Your impact will go beyond finding bugs - you’ll shape how we evaluate AI at Hostinger, define best practices, influence product decisions, and raise the quality bar for intelligent tools used by millions.

This role is for someone who sees ambiguity as opportunity, not chaos - someone who doesn’t wait for documentation, but creates it, and who builds confidence in the future of our AI products.

Curious to learn more? Connect with your team: Aurimas Karlonas, QA Lead

Your day-to-day

Define success criteria for AI-driven features where no industry standards exist, and clarify what “correct” truly means.

Design and execute testing strategies for AI tools, including exploratory and adversarial testing to uncover failure modes.

Develop evaluation frameworks to measure AI behavior - accuracy, consistency, hallucination rate, edge-case handling, and stability.

Build and maintain testing datasets, validation pipelines, and experimentation frameworks.

Identify patterns in AI behavior, developing intuition around its strengths, weaknesses, and risk areas.

Act as the voice of quality strategy, bridging the gap between what we build and how we know it’s truly good.

Your skills and experience

4+ years of experience in software quality engineering (experience with AI-powered products is a strong plus).

Strong critical thinking - you reason from first principles, not templates.

Experience in web testing and automation (Playwright, Cypress, Selenium, or similar), and confidence working with APIs, logs, debugging tools, and data.

Ability to define and measure quality beyond “it works,” including meaningful metrics and evaluation criteria.

Strong interest in how AI systems work under the hood, with the curiosity to quickly learn new tools and technologies.

Comfortable operating in ambiguity - no documentation, no prior art, no one to ask.

Proactive, analytical, and persistent, with strong communication skills in English (Lithuanian is a plus).

What makes you stand out

Hands-on experience testing LLM-based or AI-driven systems, including prompt engineering and working with evaluation datasets or benchmarking methods.

Strong understanding of AI failure modes (hallucinations, bias, drift, edge cases) and experience designing evaluation frameworks or internal quality standards.

Experience working in fast-paced, experimental environments where requirements evolve quickly.

Benefits for you

🚀 360 Growth: We provide limitless learning opportunities: access to platforms like Reforge and CoachHub, global conferences, physical and digital libraries, feedback culture, and mentoring through TesoXchange. Advance your career with internal mobility and grow with a team eager to share knowledge and support your success.

🎯 Freedom & responsibility: Work on your terms: from modern offices in Kaunas and Vilnius, the comfort of home, or anywhere in the world. Enjoy flexibility in managing your schedule and bring your ideas to life in a fast-paced, dynamic environment.

💪Wellness simplified: Your health comes first with insurance from Day 1, gym memberships, recharge leave, Headspace subscriptions, and regular health checks. Join our wellbeing-dedicated events, sports, arts, and hobby clubs, or simply enjoy the balance of a lifestyle that prioritizes wellness.

🎉 Work hard – play hard: Celebrate your achievements with company events like Summerfest & Winterfest, Town Hall, Meet the Client initiatives, team-buildings, and workations. Enjoy access to the Žalgiris Arena VIP Lounge and celebrate life’s big moments with milestone gifts for weddings, new parenthood, and graduations.