Prompt Engineer & LLMOps Specialist

We’re seeking a candidate with a solid grasp of English and Python fundamentals, expertise in LLMOps, familiarity with current prompting trends, and enthusiasm for AI advancements. This role will focus on developing innovative solutions for Voice AI and automation projects, making technology helpful for our clients while ensuring robust, scalable, and monitored LLM implementations. Required Qualifications:

Extensive experience with LLM prompting techniques (e.g., RAG, CoT, function calls, few-shot, chaining)
Strong proficiency in LLMOps: evaluating and monitoring accuracy, latency, costs, and relevant metrics
Familiarity with LLMOps tools for systematic tracking and evaluation of prompts and fine-tuning
Expertise in designing and implementing robust evaluation frameworks for LLMs in production
Strong proficiency in English
Ability to write clean, readable Python code, and capability to understand and modify existing scripts
Proficiency with APIs, SQL databases, and Git
Solid understanding of data processing concepts, cloud platforms, and machine learning fundamentals Nice to Have:
Experience developing custom LLM utilities and tools
Conceptual understanding of neural network and LLM internals
Hands-on experience with LLM fine-tuning and hyperparameter optimization
Familiarity with A/B testing and experimentation frameworks for LLMs
Client management and communication skills
Experience with CI/CD pipelines for LLM-based applications Key Responsibilities:
Develop, test, and continuously optimize prompts tailored to specific client needs
Design and implement robust systems for evaluating and monitoring LLM performance in production environments
Utilize LLMOps tools to ensure systematic tracking, versioning, and evaluation of prompts and fine-tuning experiments
Implement logging and monitoring solutions to track LLM performance, usage patterns, and potential drift
Develop and maintain dashboards for real-time monitoring of LLM metrics and KPIs
Collaborate with data scientists and ML engineers to integrate LLMs into larger ML pipelines and systems
Actively participate in project reviews and creative brainstorming sessions, contributing both technical expertise and innovative ideas
Engage directly with clients to understand their requirements, align expectations, and provide technical guidance
Stay current with the latest developments in LLM technology, prompting techniques, and LLMOps best practices
Contribute to the development of internal tools and frameworks for streamlining LLMOps processes LLMOps Focus:
Implement version control for prompts, model weights, and datasets
Set up automated testing pipelines for prompt evaluation and model performance
Develop strategies for prompt optimization and iterative improvement based on production data
Implement safeguards and fallback mechanisms for LLM-based systems
Collaborate on developing cost-effective strategies for LLM deployment and scaling

The ideal candidate will bring a strong technical background in LLMs and a methodical approach to operationalizing AI systems. They will be passionate about pushing the boundaries of what’s possible with language models while ensuring reliable, scalable, and ethically sound implementations.

graph8

New Jobs

Prompt Engineer (Excellent English)