Generative AI evaluation in Morocco: measure before deployment

Generative AI evaluation in Morocco is not a matter of asking a few colleagues whether a demo “looks good.” A system can sound convincing while citing the wrong source, ignoring an instruction, calling the wrong tool, or exposing information it should not use. Before production, business expectations must become test cases, criteria, and release thresholds.

This discipline belongs in an industrialised Artificial Intelligence project. It covers model choice, prompts, retrieval, tools, permissions, and user experience. The goal is not a universal score; it is evidence that the system is reliable enough for a defined use.

Start from the business decision

An evaluation should answer an actionable question. Can the system summarise a case without omitting mandatory facts? Can it route a request to the right team? Can it answer from an authorised corpus with a verifiable citation? Each question requires different criteria.

Describe the user, task, accessible data, permitted action, and impact of failure. This extends an AI audit: the audit selects the right use case, while evaluation verifies that a specific solution meets deployment conditions.

Write failure modes before metrics

List what can go wrong: unsupported answer, ignored instruction, stale source, incomplete extraction, unsuitable tone, data disclosure, incorrect tool, agent loop, or excessive refusal. Connect each failure to its impact and response: warn, request human approval, block the action, or fall back safely.

The NIST AI RMF Generative AI Profile provides a risk lens that includes confabulation, privacy, information integrity, security, and human oversight. Adapt it to the organisation’s actual context instead of copying it as a generic checklist.

Build a representative test set

The evaluation set should reflect production inputs: simple, ambiguous, incomplete, multilingual, poorly written, and intentionally adversarial requests. For a service used in Morocco, testing may need to cover French, Arabic, English, and Darija according to target users. Performance in one language should not be assumed to transfer to another.

Organise cases by scenario, difficulty, language, user segment, and risk. Store a reference answer when possible, while recognising that a generative task can have several valid wordings. Stable elements should be explicit: required facts, authorised sources, structure, action, and limitations.

Separate development and validation

If a team continuously adjusts prompts against the same questions, it eventually optimises for those examples. Keep a visible development set, a protected validation set, and a critical non-regression set. Version every case, its source, expected behaviour, and risk level.

Production examples should be cleaned, minimised, and anonymised before entering a test suite. A data quality practice helps control provenance, duplicates, and usage rights.

Combine multiple graders

No single grader covers everything. A robust suite usually combines:

• deterministic checks for formats, schemas, fields, citations, and prohibited actions;
• business rules for precise domain constraints;
• reference comparison when expected output is stable;
• human review with an explicit rubric for ambiguous or sensitive cases;
• LLM-as-judge for scalable qualitative assessment after calibration against human decisions.

OpenAI’s official evaluation best-practices guide recommends task-specific evals, a mix of methods, and continuous evaluation rather than a one-time test. A model grader is not ground truth: track disagreements with experts and re-review difficult cases.

Choose metrics tied to the use case

Metrics should drive a decision. For structured extraction, measure schema compliance, field accuracy, and omissions. For a document assistant, measure retrieval relevance, faithfulness to sources, citation quality, and unsupported claims. For an agent, examine task success, trajectory, tool calls, and final state.

Add operational metrics such as latency, cost by scenario, retries, human escalations, and technical errors. Do not collapse everything into one average. A global score can hide a severe failure in a small segment. Report results by language, scenario, risk, and version.

Evaluate RAG by component

A RAG assistant can fail because the correct document was not indexed, retrieval selected the wrong passage, or the model invented an answer despite good context. Evaluate ingestion, retrieval, and generation separately.

• check whether relevant passages are retrieved;
• measure noise and unauthorised sources;
• verify that answers remain grounded in context;
• confirm that citations point to the passages used;
• test questions the system should decline.

The Ragas documentation presents a dataset- and metric-based approach to evaluating LLM and RAG applications. The guide to RAG in Morocco covers the system design that these tests assess.

Test agents and tools

For an AI agent, the final answer is not enough. Analyse the trajectory: selected tool, arguments, permissions, error handling, number of steps, and stop condition. Two trajectories may reach the same result, but one may be costly or unsafe.

Prepare scenarios where a tool is unavailable, data is missing, the user changes objective, or an irreversible action requires confirmation. The system should fail safely and hand a clear case to a person.

Include security and adversarial tests

A functional suite does not cover prompt injection, data exfiltration, permission bypass, or inputs designed to provoke an action. Add negative tests for behaviours the system must never perform.

Classify findings by severity. An awkward phrase does not carry the same impact as confidential data access or an unauthorised external action. Critical cases should block a release even if the overall average improves.

Keep humans in the loop

Automated evaluation speeds up comparison, but sensitive decisions still need human review. Use a short rubric covering accuracy, completeness, justification, tone, and risk. Train evaluators with annotated examples and measure disagreement.

The same principle applies in production. A human-in-the-loop workflow defines cases that require approval, escalation, or recovery. Human corrections then become candidates for the evaluation set.

Automate regression tests

Changing a prompt, model, index, tool, or rule can improve one scenario and degrade another. Run non-regression tests before every promotion and compare results with a reference version.

This control is part of MLOps: model version, configuration, corpus, test set, scores, and release decision stay connected. Record cost and latency as well, so that a quality gain that is too slow or expensive remains visible.

Monitor after deployment

An offline suite cannot predict every real request. In production, monitor errors, refusals, escalations, user feedback, distribution changes, and incidents. Regularly sample interactions for qualitative review while respecting privacy rules.

An observability practice links each interaction to the model, prompt, corpus, and tool versions. A discovered failure can then be reproduced and added to the non-regression set.

Release checklist

• business objective and impact of failure defined;
• failure modes and sensitive cases documented;
• representative, versioned, and multilingual test set when required;
• metrics segmented by scenario and risk;
• automated graders calibrated against humans;
• RAG, agent, and security tests separated when applicable;
• blocking thresholds and release owner named;
• non-regression and production monitoring active.

Generative AI evaluation in Morocco turns an impression into a documented decision. It cannot guarantee zero error, but it makes limits visible, versions comparable, and deployment governable. To design an evaluation suite for your use case, contact Kanteek.