Will translation ever be fully automated?

Article written at the occasion of the XRCE 20^th anniversary celebration .

Since its beginnings in the 1940s, machine translation has been a subject of fascination for computer scientists. It has a unique position among artificial intelligence problems because it is both a challenging intellectual task, like text or scene understanding, but also — in contrast to those tasks — one that produces overt traces, the translated texts, which are relatively transparent to us human judges.

In the early 1990s, the field experienced a revolution. To the astonishment of the machine translation (MT) establishment, a team of researchers at IBM demonstrated that an MT system could be automatically learnt from a large set of text using statistical techniques, instead of requiring the delicate hand-crafting of linguistic rules common at the time.

Since this initial experiment, Statistical Machine Translation (SMT) has contributed to a significant resurgence of interest in machine translation, and led to popular generic SMT services such as Google Translate, which about 200 million people use 1 billion times each day.[1]

Despite the huge progress experienced over the last 20 years, machine translation still falls far short of producing results of comparable quality to human translation. So, will translation ever be fully automated?

Being old enough in the field to have participated in the development of both rule-based and statistical translation systems let me share some brief thoughts about this slightly chimerical question.

Machine translation and AI-completeness

In the field of artificial intelligence, ‘AI-complete’ problems refer to challenges that can only be solved by programs that perform at the level of human intelligence.

It is not difficult to illustrate the AI-completeness of translation. Take for example the French text: “Le policier avait poussé la femme dans le canal.;Sa condamnation fut rapide.” which could translate into English as ;“The policeman had pushed the woman into the canal. His/Her conviction was quick.”Because the agreement of possessive adjectives in French is with the following noun, but in English with the owner, the only way that a translator can determine whether “sa” should be translated by “his” ;or by “her” ;is through a complex process of reconstructing implicit ;facts not directly mentioned in the text: the woman drowned and died, the policeman’s act is a crime, crimes are judged in courts, etc.

The opposition implicit/explicit; represents a major challenge for machine translation. This is because languages are very different in what they mark explicitly and what they leave in the background.[2]

The consequences of this for MT may be illustrated by the following optical analogy:[3]

French and English “light sources”

In the picture, French and English correspond to two different “light sources” that project “shadows” of the same mental representation onto a textual plane. These shadows have many things in common, but also significant differences. These differences correspond to the different aspects of the representation that the two languages make explicit. To translate from English to French, one needs to produce the French shadow from the English one.

The natural translation process requires first reconstructing the common situation using knowledge of the English light source along with commonsense understanding. It then projects this onto the plane using the French light source.

It however often is possible to produce good approximations of this complex process by focussing on regularities that can be observed at the level of the textual plane. This has been the route followed by most rule-based and statistical machine translation systems.

Here, SMT has some decisive advantages over traditional rule-based systems:

• It is unrealistic to hand write rules covering the myriad idiosyncratic constructions that need to be translated case-by-case between different languages. In contrast, SMT systems are able to automatically learn many such constructions from bilingual corpora; thus Google Translate has no difficulty translating Tom is hungry into Tom a faim [Tom has hunger] despite the distinct constructions, and this observation holds for many similar expressions.

• An equally important advantage is that SMT systems embed a powerful mechanism which is a probabilistic language mode;of the target language. Such a model provides an estimate of how likely a certain sentence in the target language is, based on generalisations from observations of a large number of existing target language texts. Such estimates can often provide a “proxy” to simple commonsense inferences, helping to disambiguate between different choices; thus the Google service translates le temps est souvent pluvieux into the weather is often rainy, but also le temps est souvent limité into time is often limited, correctly selecting the right translation for the ambiguous French word temps in both examples.

Current SMT systems are however still limited in their capability to generalize. For example, slightly enlarging these examples tends to produce wrong translations: Tom is extremely hungry –> Tom est extrêmement faim, le temps est souvent désagréablement pluvieux –> time is often unpleasantly wet. Here the target language model is “misled” by the increased distance between semantically clashing words (est with faim, time with wet), and is not able to detect the inconsistencies.

Solutions on the Horizon

Returning to our original question, if we wished to focus on the AI-completeness argument, our answer would have to be something like: “Fully automated translation will probably happen in the future, but around the same time at which our streets will be populated with the kinds of androids depicted in the recent Swedish sci-fi series RealHumans”.[4] However, many other tasks that are already highly automated are, or can be construed to be, AI-complete — e.g., speech recognition or even optical character recognition. While MT is more AI intensive than these tasks, much progress can be made by the MT community without having to wait for a full solution to the general AI problem. This can be done by focussing on solving many specific difficulties of translation which currently are poorly handled but have potential solutions on the horizon.

Let me outline one possible direction for such progress. Professional human translators typically translate from a foreign language into their native language, where they are much more comfortable with syntax and linguistic nuances. In fact, even novice learners of a foreign language can often do a good job of translation with the help of a bilingual dictionary as long as their linguistic competence in their native target language is high. To some extent, the understanding of the source text may itself often be assessed by checking that the translation makes sense. We could say that these people are exploiting their ‘Natural Language Generation’ capabilities in their native language.

These observations of human translators are interesting to relate to the current status of SMT. While it is true that target language models play an important role, most effort is still devoted to acquiring bilingual corpora. This can be difficult, especially for domains which are not well represented by previous translations. Monolingual data in the target language, for most domains of interest, are much easier to find. It might therefore make sense to focus more attention on better exploiting the monolingual data to produce deeper generation models for the target language. Learning richer language models from these data could prevent frequent errors of the kind illustrated above (Tom est extrêmement faim, time is often unpleasantly wet). Other common generation errors may be more difficult to handle based on data only and without injecting prior knowledge of the grammatical rules of the language, including some intricate prescriptive rules that we learnt at school. If translation is ever fully automated it is likely that rich statistical models emphasizing the target language combined with classical linguistic knowledge will be major ingredients, and the good news is that these are dimensions on which we can already make progress today.

About the author:
Marc Dymetman is a principal scientist at XRCE. His main research interests are statistical modelling of natural language, machine translation and machine learning.

[1] http://www.imdb.com/title/tt2180271/

[2] The World Atlas of Language Structures (WALS) http://wals.info/ provides an extended compendium of the many such typological differences; some languages do not mark Tense or Gender, some others obligatorily indicate things like “Evidentiality” (how the speaker came to know what he says), etc.

[3] Figure inspired from http://www.researchgate.net/publication/245023525.

[4] See http://news.cnet.com/8301-1023_3-57585143-93/google-translate-now-serves-200-million-people-daily/, May 2013.