Evaluation metrics for a translation memory system
โ Scribed by Edward K. Whyman; Harold L. Somers
- Publisher
- John Wiley and Sons
- Year
- 1999
- Tongue
- English
- Weight
- 161 KB
- Volume
- 29
- Category
- Article
- ISSN
- 0038-0644
No coin nor oath required. For personal study only.
โฆ Synopsis
This paper concerns the methods of evaluation of a computational translator's aid known as a Translation Memory (TM). A typical TM system selects candidate translations into a target language by obtaining matches to an input source-language sentence from a database comprising source-target pairs of previously translated sentences. We describe a metric for evaluating TMs and illustrate it with a case study based on a particular commercial TM system and a database of French-English sentence pairs specifically created from a previously translated text. This metric builds on definitions of the familiar parameters of precision and recall, and involves a measure of the 'usefulness' to a translator of the selections made by the system. Several variations on these parameters and suggested weighting schemes are discussed in the search for a suitable metric, resulting in a 'weighted precision' measure based on the number of keystrokes needed to correct the proposed text. In combination with appropriate measures of recall, parameters which could serve both as numerical optimisation criteria to be employed by the translator in seeking the most efficient use of a TM tool, and as means of carrying out quantitative evaluations of such systems are defined.
๐ SIMILAR VOLUMES
## Abstract In this article, we introduce a new information system evaluation method and report on its application to a collaborative information seeking system, AntWorld. The key innovation of the new method is to use precisely the same group of users who work with the system as judges, a system w
This article describes the process of developing a system for translating VDM executable specifications into Lazy ML prototypes. The system was specified in VDM and a Lazy ML prototype implementation was derived from its specification. This article concentrates on discussing the lessons learned in e
Conventional operating systems employ a kernel-controlled caching strategy that cannot properly serve all access-pattern types used by applications. When running under these systems, many memory-intensive applications with mis-matching access patterns cause excessive page faults and page replacement