Senior AI Engieer - Voice & Agentic Systems

Xebia is a global AI-first, digital transformation, and engineering partner. With over 25 years of experience and a team of 5,000 professionals across 16 countries, we help organizations design and build scalable products, platforms, and data-driven solutions. 

We specialize in Artificial Intelligence, Data and Cloud, Intelligent Automation, and Digital Products, combining deep technical expertise with a strong focus on engineering excellence and a people-first culture.  

In the CEE region, we’re a team of nearly 1,000 experts delivering modern applications, data platforms, and AI solutions for clients such as Millennium, ING, Play, Edenred, Arabian Drilling, FedEx, Leroy Merlin, Truecaller, Volotea, Schmitz Cargobull, and many, many more. We work with leading technologies including AWS, Azure, GCP, Databricks, and Snowflake, and combine strong engineering culture with a consulting mindset and a continuous focus on growth and knowledge sharing. 

About project:

We are looking for a Senior AI Engineer to join a product team building a voice-first AI agent that allows users to interact through natural spoken conversations. The agent reasons, uses tools, and takes actions in real time, with the product currently in active development and its architecture and product direction still evolving.

This is a hands-on engineering role covering the full voice-agent stack, from real-time communication and speech processing to agent orchestration, tool use, memory, and backend integrations. You will have significant influence over both the technical architecture and product decisions, working closely with product, design, and a small engineering team.

You will be:

Agent engineering

• designing and building agent workflows using LangGraph, including state management, tool and function calling, multi-step reasoning, memory, error recovery, and human handover,
• defining how the agent makes decisions during live conversations where latency and interruptions are critical,
• building and maintaining the backend integrations that allow the agent to take actions,
• designing reliable agent workflows that can operate in real-world production environments.

Voice pipeline

• building and tuning real-time voice pipelines using technologies such as Pipecat, LiveKit Agents, or similar,
• working with streaming STT, TTS, voice activity detection, turn-taking, and interruption handling,
• improving conversation quality through end-of-turn detection, barge-in handling, filler and back-channeling, and graceful error recovery,
• measuring and reducing end-to-end latency across the voice and agent pipeline,
• ensuring reliable voice interactions under real-world network conditions.

Architecture and quality

• shaping the overall system architecture and making build-vs-buy decisions for models, speech technologies, and other components,
• building evaluation into the development workflow through conversation-level test sets, regression suites, LLM-as-judge approaches, and latency and quality metrics,
• implementing observability and tracing using tools such as Langfuse, LangSmith, or similar solutions,
• diagnosing production issues through traces, metrics, and real user sessions,
• applying strong engineering practices including code reviews, automated testing, CI/CD, documentation, and maintainable design.

Product

• translating user needs and product goals into practical technical proposals,
• challenging assumptions and suggesting alternative approaches when they can improve the user experience,
• prototyping ideas quickly and hardening successful approaches for production,
• communicating technical trade-offs clearly to both technical and non-technical stakeholders,
• contributing to regular in-person team sessions for planning, design, and technical reviews.

Your profile:

• proven track record of shipping GenAI and agentic systems to production that are used by real users,
• hands-on experience with LangGraph, including designing and running production workflows,
• strong Python skills, including asynchronous programming and building production services such as FastAPI applications,
• strong software engineering and system architecture skills, with the ability to design systems end to end and make informed technical trade-offs,
• senior or lead-level engineering experience with ownership of significant technical decisions,
• strong product sense and evidence of influencing what gets built, not only how it is implemented,
• strong understanding of production-quality software development, testing, observability, and reliability,
• fluent English with the ability to communicate clearly with both technical and non-technical stakeholders,
• willingness to participate in regular in-person team sessions.

• Practical experience using AI-powered assistants (e.g. Claude Code, GitHub Copilot, Cursor) to improve productivity, quality, or decision-making in software delivery.

• Work from the European Union region and a work permit are required.

  Strongly preferred:

  • hands-on experience building voice agents or real-time voice pipelines,
  • experience with Pipecat, LiveKit Agents, or similar voice-agent frameworks,
  • practical experience with streaming STT/TTS technologies such as Deepgram, Soniox, ElevenLabs, or Cartesia,
  • experience with voice activity detection, turn-taking, interruption handling, and voice latency optimization,
  • production experience with WebRTC and/or LiveKit,
  • experience with telephony technologies such as Twilio, Telnyx, or SIP.

Nice to have:

• experience with LLM evaluation and observability tooling such as Langfuse, LangSmith, or custom evaluation frameworks,
• experience working with multiple model providers such as OpenAI, Anthropic, and Gemini,
• experience routing between models based on cost, latency, and capability,
• experience with MCP or similar tool-integration patterns,
• enough TypeScript and React experience to contribute to the client-side voice interface,
• experience working in a startup or 0-to-1 product environment.

• Experience applying GenAI in a more structured way within the SDLC, including defined workflows, prompt patterns, or tool integrations embedded into daily work.

• Interest in and familiarity with emerging AI-driven practices (e.g. agent-based workflows, automation patterns, AI-augmented development), with a willingness to explore and experiment beyond standard approaches.

Work mode:

• working remotely from Europe, ideally within 1–2 hours of CET,
• participating in in-person team meetings every 4–6 weeks,
• being open to collocating in Berlin, with Madrid or Barcelona also possible alternatives.

Recruitment Process:

CV review – HR call – Interview – Client Interview – Decision

Job Description

The team is building a voice-first AI agent: users talk to it in natural spoken conversation, and it reasons, uses tools and takes actions for them in real time. The product is in active development. The architecture is still taking shape and many product decisions are still open, so the engineer in this role will have real influence on both.

The work covers the whole voice-agent stack:

Real-time transport: WebRTC/LiveKit, telephony
Speech: streaming STT, TTS, voice activity detection, turn-taking, interruptions
The agent: LangGraph orchestration, tool use, state, memory
Integrations: the back-end systems the agent acts on
About the role

We're looking for a senior engineer who can design and build across that stack and own important technical decisions. Just as important is strong product sense. We want someone who thinks about how the product feels to the person talking to it, anticipates what users will need, and turns that into sound technical choices, instead of waiting for a detailed spec.

You'll work closely with product, design and a small engineering team. You'll be expected to challenge assumptions, suggest alternatives and explain trade-offs clearly to both technical and non-technical colleagues.

What you'll do

Agent engineering

Design and build agent workflows in LangGraph: state management, tool and function calling, multi-step reasoning, memory, error recovery and handing over to a human
Define how the agent decides what to do in a live conversation, where latency and interruptions matter
Build and maintain the integrations the agent uses to take actions

Voice pipeline

Build and tune the real-time voice pipeline (e.g. Pipecat, LiveKit Agents or similar): STT → agent → TTS, streaming end to end
Improve the parts that make a conversation feel natural: end-of-turn detection, barge-in and interruption handling, filler and back-channelling, and recovering from errors
Measure and reduce end-to-end latency, and make the system reliable under real network conditions

Architecture and quality

Help shape the overall system architecture and make build-vs-buy decisions for models and speech vendors
Build evaluation into the workflow: conversation-level test sets, regression suites, LLM-as-judge where it's appropriate, and latency and quality metrics
Add observability and tracing (e.g. Langfuse or LangSmith) so production issues can be diagnosed from real sessions
Bring solid engineering practice: code review, testing, CI/CD, documentation and maintainable design

Product

Turn user needs and product goals into technical proposals, and push back when a requirement won't give a good experience
Prototype quickly to test ideas, then harden what works for production
Take part in regular in-person team sessions for planning, design and review