How I built an edge AI translation system for multilingual operations

Most translation products are optimized for a simple user story: enter a sentence, get a translated sentence back.

MoYi started from a different problem. In factories, logistics teams, and remote operations, translation is part of a workflow. A phrase may include a machine name, a safety warning, an internal process, or a shorthand that only makes sense inside the company. If the translation loses those details, the team does not just get awkward language. They can lose time, training quality, or operational safety.

That is why I designed MoYi as an edge AI translation system, not just a chat UI around a translation API.

The product problem

The practical requirement was clear: translation should work close to the user, preserve sensitive operational context, and reduce dependency on cloud calls.

The system needed to support:

• domain glossary control for product names, machine names, and internal terms;
• safety-aware handling for warnings, instructions, and high-priority messages;
• local or edge execution where possible;
• a backend-agnostic runtime so the model can change without rewriting the product;
• integration paths for desktop, Python tooling, and mobile applications.

This shaped the architecture more than the choice of model did.

The architecture

The core idea is to separate the workflow from the inference backend.

Input message
    -> language and context normalization
    -> glossary lookup
    -> safety phrase detection
    -> translation request plan
    -> model/backend adapter
    -> validation and repair
    -> final response

The runtime can then target ONNX Runtime, llama.cpp, mobile runtimes, or hardware-specific acceleration later. The product does not depend on one provider. It depends on a stable pipeline.

That pipeline also makes the system easier to test. I can evaluate latency, glossary accuracy, safety phrase recall, and memory usage independently instead of judging the product only by whether a single response “sounds good.”

Why edge-first matters

For a normal consumer app, sending text to the cloud may be acceptable. For internal operations, that is not always true.

A local-first translation layer gives the organization more control over:

• private operational messages;
• terminology that should not leak outside the company;
• latency during meetings or field work;
• offline or weak-network environments;
• model/runtime choices per device.

This is especially useful when the system is deployed on desktops, Android devices, embedded Linux machines, or Qualcomm/Intel edge hardware.

What I learned

The main lesson is that translation quality is not only a model problem. It is a systems problem.

If the glossary is missing, the output may be fluent but wrong. If safety phrases are not detected, the system may soften an urgent warning. If latency is high, the team stops using it in real workflows. If the runtime is locked to one provider, the product becomes fragile.

MoYi is still evolving, but its direction is clear: build a practical translation runtime for multilingual operations, with the product workflow and safety constraints treated as first-class parts of the system.

Related project: MoYi Edge Translation.