Structured Document Translation via Format Reinforcement Learning
Haiyue Song, Johannes Eschbach-Dymanus, Hour Kaing, Sumire Honda, Hideki Tanaka, Bianka Buschbeck, Masao Utiyama
2025-12-09
Summary
This paper focuses on improving how computers translate structured text, like code documentation written in XML or HTML, from one language to another.
What's the problem?
Current translation tools are good at translating individual sentences, but they struggle with the overall structure of documents that use tags and hierarchies like XML or HTML. They don't understand how the different parts of the document relate to each other, leading to messy or incorrect translations of the entire file.
What's the solution?
The researchers developed a new technique called Format Reinforcement Learning, or FormatRL. This method builds on existing translation models and teaches them to pay attention to the document's structure. It does this by giving the model rewards based on how well the translated document's structure matches the original, using two new measures: TreeSim, which checks if the overall tree-like structure is similar, and Node-chrF, which checks the quality of the translation at each individual tag. They also used a metric called StrucAUC to pinpoint exactly where the model was making structural mistakes.
Why it matters?
This work is important because it makes machine translation much better for technical documentation and other structured content. Better translations mean developers and users can more easily understand software and other complex systems, and it could automate the process of updating documentation when software changes.
Abstract
Recent works on structured text translation remain limited to the sentence level, as they struggle to effectively handle the complex document-level XML or HTML structures. To address this, we propose Format Reinforcement Learning (FormatRL), which employs Group Relative Policy Optimization on top of a supervised fine-tuning model to directly optimize novel structure-aware rewards: 1) TreeSim, which measures structural similarity between predicted and reference XML trees and 2) Node-chrF, which measures translation quality at the level of XML nodes. Additionally, we apply StrucAUC, a fine-grained metric distinguishing between minor errors and major structural failures. Experiments on the SAP software-documentation benchmark demonstrate improvements across six metrics and an analysis further shows how different reward functions contribute to improvements in both structural and translation quality.