๐”– Bobbio Scriptorium
โœฆ   LIBER   โœฆ

Evaluation metrics for a translation memory system

โœ Scribed by Edward K. Whyman; Harold L. Somers


Publisher
John Wiley and Sons
Year
1999
Tongue
English
Weight
161 KB
Volume
29
Category
Article
ISSN
0038-0644

No coin nor oath required. For personal study only.

โœฆ Synopsis


This paper concerns the methods of evaluation of a computational translator's aid known as a Translation Memory (TM). A typical TM system selects candidate translations into a target language by obtaining matches to an input source-language sentence from a database comprising source-target pairs of previously translated sentences. We describe a metric for evaluating TMs and illustrate it with a case study based on a particular commercial TM system and a database of French-English sentence pairs specifically created from a previously translated text. This metric builds on definitions of the familiar parameters of precision and recall, and involves a measure of the 'usefulness' to a translator of the selections made by the system. Several variations on these parameters and suggested weighting schemes are discussed in the search for a suitable metric, resulting in a 'weighted precision' measure based on the number of keystrokes needed to correct the proposed text. In combination with appropriate measures of recall, parameters which could serve both as numerical optimisation criteria to be employed by the translator in seeking the most efficient use of a TM tool, and as means of carrying out quantitative evaluations of such systems are defined.


๐Ÿ“œ SIMILAR VOLUMES


Cross-Evaluation: A new model for inform
โœ Ying Sun; Paul B. Kantor ๐Ÿ“‚ Article ๐Ÿ“… 2006 ๐Ÿ› John Wiley and Sons ๐ŸŒ English โš– 336 KB ๐Ÿ‘ 1 views

## Abstract In this article, we introduce a new information system evaluation method and report on its application to a collaborative information seeking system, AntWorld. The key innovation of the new method is to use precisely the same group of users who work with the system as judges, a system w

A System for Translating Executable VDM
โœ PAULO BORBA; SILVIO MEIRA ๐Ÿ“‚ Article ๐Ÿ“… 1997 ๐Ÿ› John Wiley and Sons ๐ŸŒ English โš– 133 KB

This article describes the process of developing a system for translating VDM executable specifications into Lazy ML prototypes. The system was specified in VDM and a Lazy ML prototype implementation was derived from its specification. This article concentrates on discussing the lessons learned in e

Hipec: A System for Application-customiz
โœ PAUL C. H. LEE; RUEI-CHUAN CHANG; MENG CHANG CHEN ๐Ÿ“‚ Article ๐Ÿ“… 1997 ๐Ÿ› John Wiley and Sons ๐ŸŒ English โš– 226 KB ๐Ÿ‘ 2 views

Conventional operating systems employ a kernel-controlled caching strategy that cannot properly serve all access-pattern types used by applications. When running under these systems, many memory-intensive applications with mis-matching access patterns cause excessive page faults and page replacement