Wednesday, September 14, 2011

The ProZ-TAUS ‘Great Translation Debate’: Not So Great and, Actually, Not Really a Debate

Irmy: I slept with someone for it. Does that make me a whore?
Kleinmann: [Referring to the money he's holding] This?
Irmy: Just one person. Does that make me a whore?
Kleinmann: Well, no, only by the dictionary definition.
---Shadows and Fog

I’m looking at the program for the “Great Translation Debate,” a virtual conference organized by ProZ.com and TAUS. One session will dicuss the proposition that “Translation automation is good for the industry.” That is orthodox debate structure: You float a proposition and two teams argue, one of them for and the other against. That is the ritual structure set up in Oxford debating societies and practiced in universities throughout the world. However, in this case, we have a very curious twist on that venerable model. In the immortal words of Sideshow Bob: “Oh, Cousin Merle, really…”



The panelists are Renato Beninatto, Kirti Vashee, an MT specialist called Mirko Plitt and someone from the Common Sense Advisory. Now, I have nothing against any of these people. I disagree with Beninatto’s entire philosophy but, as a cultural phenomenon, he basically just baffles me. How can someone that lazy, who makes basic math mistakes in his presentations, who knows absolutely nothing about anything, be such an esteemed and ubiquitous pundit? The man truly is the perfect embodiment of the future of hamsterized localization. And Vashee is all right, even though he doesn’t like me anymore. I once made a silly joke on this blog about the keynote speaker at a conference he attended and Mr. Vashee vigorously corrected me in a crushing rebuttal by pointing out that there wasn’t one keynote speaker…. but three keynote speakers! After the Miracle of the Multiplication of the Keynote Speakers, he doesn’t even retweet my rants but rather leaves anonymous comments signed “Tom,” which, for the record, I think is the most unimaginative alias ever.

But frankly, didn’t the organizers think of inviting someone who can, you know, take the other side of this issue? I mean, for the love of Ding Dong… It is like inviting the Virgin Mary, St. John the Baptist, Mother Teresa and some Torquemadaesque, fanatical, Bible-thumping Inquisitor to discuss the pros and cons of the Catholic Church. If the ProZ people don’t own a dictionary, they really really need to look up the word “debate” in a free online version.

The way this “debate” is structured, it will be more along the lines of that old advertising campaign in which two out-of-control mobs of Budweiser addicts fight over what makes the beer great, whether its great taste or the fact that it is less filling. (I’m guessing minus the hot models in skimpy tops, which further diminishes the appeal for me.)

I’m guessing the “Great Debate” will go a little like this:


Henry Dotterer: Let me begin by asking Kirti what he thinks about translation automation.
Kirti Vashee: Well, Henry, I’ll tell you. I think it’s spiffy. Just spiffy.
Dotterer: Your rebuttal, Renato?
Beninatto: Gee, my position is slightly different. I think it’s super-spiffy.
Dotterer: Okay, let’s keep it civil. Mirko, care to present a counterargument?
Plitt: Boy, I sure hate to disagree with both of these fine gentlemen, but everyone knows that translation automation is super-duper-spiffy.
Dotterer: Your summation, Bible-thumping fanatic foaming at the mouth?
Bible-thumping fanatic: (foaming at the mouth) Aaaarrrgghhhh! Repent, sinners! You are all going to Hell!
Dotterer: I’m sorry, could you be more specific?
Bible-thumping fanatic: (thinks about it for a second) Uhm…. spiffy?


If only… Actually, if they invite a rabid Bible-thumping fanatic, that might actually make it worth watching. As it stands, though, a nude pillow fight at the Playboy Mansion will be more adversarial (and far more intellectually stimulating) than this travesty.

And, while we are at it, is there some constitutional requirement that Kirti Vashee and Renato Beninatto have to be at every single localization event? Do they just show up for free at these gigs, or are they paid in hors d’oeuvres and cheese doodles? I am beginning to suspect that Vashee and Beninatto are the same person. Are they joined at the hip? Has anyone actually seen them occupy different portions of the space-time continuum? Has anyone tested whether one speaks while the other takes a drink of water? Is the “Beninashee” (or “Vashinatto”) some kind of monstrous cyborg created by Asia Online to sow confusion among the heathens?

For the record, I don’t actually believe any of those things, but as the ProZ organizers have demonstrated, reality can be several degrees more absurd than the things that are dreamt of in your philosophy.


Miguel Llorens is a freelance financial translator based in Madrid who works from Spanish into English. He is specialized in equity research, economics, accounting, and investment strategy. He has worked as a translator for Goldman Sachs, the US Government's Open Source Center, several small-and-medium-sized brokerages, asset management institutions based in Spain, and H.B.O. International. To contact him, visit his website and write to the address listed there. You can also join his LinkedIn network or follow him on Twitter.

20 comments:

Jordi Balcells said...

Just for once I am going to disagree with you. While I understand that a debate needs someone in favour and someone against, I think that you can only debate in what cases automation is viable and efficient, not whether automation is good.

Making the computer work instead of you is good. As long as the end result is at least the same and the effort is lower, that's positive. Be more efficient and productive, earn more money for less work. Good stuff.

A typewriter is worse than a computer for translation. Manual localisation of numbers formatting from English into Spanish is worse than cooking a few regular expressions (or better yet, a script) that will do the same for you. Remembering that last year you translated the same sentence that you are now facing and you need to find the document and copy-paste the offending sentence into the new translation is worse than just using a translation memory system. And yes, under certain circumstances (format, specialisation, etc), typing translation from scratch is worse than postediting MT output.

And yes, I realise that automation is dangerous. Less human intervention may lead to unexpected and catastrophic results as things spiral out of control. That's why humans are needed to keep machines in check. Quality Assurance and all that.

Miguel E. Llorens Musso said...

I'm actually not against automation as you describe it, but, although not directly tied to the post, I think a related issue is what "translation automation" means. I think people such as the panelists are casting about for a new euphemism because MT has gotten kind of a bad rap. What does "translation automation" mean? Is it MT? Or is it MT plus TM? Or is it just TM? Or is it simply using any sort of software, like an online dictionary? You see, the terms of the "debate" are about as badly defined and muddled as the thinking that went into organizing the sterilized "Great Debate."

bonnjill said...

LOL! This is definitely one debate I will be missing. Like you said, there isn't a single translator or person standing up for the translator in the bunch.

Miguel E. Llorens Musso said...

Yes, that is another interesting point. Not a single translator. I think that the need to keep the debate hyper-bland so as not to offend any corporate sponsors just killed any chance for a meaningful exchange.

Kirti Vashee said...

For the record I am in no way connected to Tom.

Miguel E. Llorens Musso said...

Aha! But can you actually confirm that you and Beninatto are not the same person?

fab said...

Jordi wrote....

>Making the computer work instead >of you is good. As long as the >end result is at least the same >and the effort is lower, that's >positive. Be more efficient and >productive, earn more money for >less work. Good stuff.

That is actually the point, if you think that with MT you are going to make more money as a freelance translator you are mistaken. The MT pricing model (I have been proposed so far) is to pay "new matches" as fuzzy matches. Basically the untold MT sale pitch to translator is "now you translate 3k words and are paid 3k words but with MT you can translate 5k words in a day and get paid for 3k , aren't you excited at the idea my dear translator?".

All of the above assuming that you are actually saving time with MT to produce "premium quality" translation, which is debatable.

As a translator you could become an expert/learn touch-typing/use voice recognition sw and produce 5k and get paid for 5k, but no use machine translation translate 5k, take a gamble assuming that MT makes you save time and get paid for 3k, how cool is that?

What amazes me is that people selling MT are surprised that translator reject MT.

Unfortunately, most translators dismiss MT on a luddytes basis: "I did not study 5 years to edit what a computer does" rather than exposing the fact that "MT is not profitable for freelancers", unless it is paid by the hour (without a fixed hourly output, beacuse that is a masked per word rate).

Fab

Jordi Balcells said...

Fab: You may not have realised, but you just hit the nail on the head.

The problem with MT is not that it is used for professional translation, but rather that it has no proper pricing model. Translation? Easy, pay per word. Revision? Easy, pay per hour (ideally). Postediting? The current model is pay per word, but that does not make sense because of a lack of constant quality.

With SMT engines, anything may come out of the machine. Sometimes they are perfect segments, where nothing requires editing. Sometimes they are absolute crap, and you must delete and translate from scratch. You just do not know until you get to that sentence.

MT (engine) vendors may speak highly of quality metrics to ensure that a specific engine will work efficiently under given parameters (specialisation, client, text type, etc), but at best these metrics are just a very rough indication. Again, you just cannot know until you get to that sentence.

The problem with paying postediting by the hour is that you need properly trained translators. Some may edit too little and the quality will suffer, some may discard everything and costs will rise.

That's why clients do not want to pay postediting by the hour, they are afraid that it will be as expensive as translation.

If postediting is properly paid (by the hour), then there is no need to fear MT. You made X euros per month before, you make the same X euros per month now with the same (or less) effort. You just work differently.

How to convince clients requesting postediting to pay by the hour? No idea. :D

Miguel E. Llorens Musso said...

Two isolated observations:

The idea of payment by hour for post-editing has been widely discussed and I think universally accepted by the MT community. My observation is that the specifics of the pricing mechanism won't be that big an issue because the wages, whether per hour or per word or otherwise, will tend to be low *in absolute terms*. Why? Because the model will be pioneered by companies that always tend towards the lower end of the wage scale for clients who don't care that much about their translations. Just show me a post-editor (a pure post-editor, mind you, not the person who trains the engine) making $80,000 or more a year and I'll shut my mouth.

So that is one thing. The other is that post-editing will have an impact on your writing style. See Jost Zetsche's experiment working as a post-editor for a week. If you hope to distinguish yourself as a language professional, pumping out 10k per day is not advisable in any field. For premium quality, MT doesn't help because, at best, it is just a mediocre first draft. That is the point a lot of people, including translators who say very superficially that "technological development is inevitable," don't seem to get. Non-translators don't get it because they don't know how to work a sentence until it sounds well in the target language. And that is why there is a "dialogue of the deaf."

Jordi Balcells said...

I am not so sure that an hourly rate for postediting is universally accepted. At least not an hourly rate that is not really a disguised per word rate, as Fab mentioned.

I cannot say I know that many translators making more than, say, USD 40,000 a year. Not that I ask how much they earn, but I doubt it's even $40,000 a year after taxes. That's true for people younger than 40 who do no consulting.

Yes, I agree that postediting can hurt your style and turn you into a hamster that can only work in the low quality end of the spectrum. Even as little as a couple of days doing it is bad for you. You tend to let more things slide. If you do a couple of review passes when postediting, just like you would do with translation, it does not get too bad. The important thing is the end result and how much effort you put into getting there.

Miguel Llorens M. said...

Regarding the pay issue: In the absence of unbiased research into what post-editors earn, the discussion is sterile. But, may I repeat, I suspect that the general trend will be towards being located in the shallower end of the spectrum. In the mean time, though, it would be interesting to hear from regular post-editors: what they feel about the practice, what their working day is like, whether they are happy with their earnings and jobs. Or, failing that, companies that advertise how much they pay their fat, contented post-editors. Even that anecdotal evidence is scarce. Instead, once I made a half-joking entry based on a quotation from an MT entrepreneur about how much his freeelancers make and I got an irate response from him saying basically that if the freelancer’s working life sucks, that is just tough noogies. So I think that is pretty revealing.

Regarding the style issue, when I used MT (and I did!), I discovered that there was no productivity boost for me. Why? Because turning the mediocre MT output into a decent translation plus those “two passes” you mentioned siphoned off so much time that there was no visible impact on my output level. And that was a huge revelation, because I realized that the time I spend making the text my own, adding that little extra is basically my value added, and automation can’t help me there. That is when I discovered that what I do is much more interesting than functioning as a complex human algorithm or as a simple word-plunker.

Aurora Humarán said...

"is there some constitutional requirement that Kirti Vashee and Renato Beninatto have to be at every single localization event?"

Indeed! An international requirement based on their world tours. Great post, Miguel!

Lorena said...

You made my day with this, Miguel: "Is the “Beninashee” (or “Vashinatto”) some kind of monstrous cyborg created by Asia Online to sow confusion among the heathens?". A couple of colleagues complained because translators had not been invited and here´s Drew´s reply: "We are happy to include translators on panels and discussions. It's something we've been trying to do without success. Freelance translators that are willing to participate in the debate should contact me directly."

Diane McCartney said...

Hi Miguel,

I attended a conference in Holland on 9 September that was organized by the Association of Translation Agencies (ATA the lesser as it has been coined by a fellow blogger of yours) about MT. The venerable guests included Renato Beninatto and Jaap van der Meer. I'd like to share a few things that I got out of the conference:

There is unbiased research out there, it's just kept quiet. I attended a session on post-editing that was given by Dr. Sharon O'Brien of the School of Language and Intercultural Studies at Dublin City University. I was surprised someone like her got invited to the conference. She had no problem whatsoever saying that MT sucked, had always sucked, and will always suck. According to her, there are two companies in the world that have successfully implemented MT: Symantec and another one the name of which escaped me. The university has been doing extensive research into MT using real text, a real environment, real metrics and real people. A LOT, and I mean a LOT of work has to go into setting up the MT environment, like PRE-editing translation memories. That's the first time I'd ever heard anyone mention anything being done so MT would be usable. She also mentioned that MT has only become usable since the rule-based MT people and the statistical MT people combined the systems. That being the case, cleaning up the TMs created by humans would seem to be the logical first step. Terminology, of course also has to be managed and employees need to be taught to write in controlled language, which is only possible in very specific environments, i.e. environments that have some sort of template. And all this stuff needs to be maintained. The effort is huge in terms of time and money.

Symantec uses MT for virus alerts that really do need to be published in real time and the goal of which is to enable engineers of all nationalities and mother tongues to understand what's going on so the appropriate action can be taken as quickly as possible. Polished English doesn't really seem to be necessary here. I'm not saying that crap will do: Symantec does post-edit the MT stuff. Symantec does, however, not use MT to translate anything else.

Those who are positive about MT are simply incapable of seeing through the nonsense they're being sold by Beninatto and the other self-pronounced gurus who are doing nothing but applying basic sales tactics: Take an old concept and repackage it. Or they have a vested in interest in selling a piece of crap. And here's me thinking that the term "vaporware" was restricted to the software industry....

--

Aurora Humarán said...

«She had no problem whatsoever saying that MT sucked, had always sucked, and will always suck.»

Hi, Diane!
Thanks for your contribution! I couldn't agree more with Prof. O'Brien. The problem is that a lethal joint venture (MT + slave translators) seems to be working very well. This thanks to certain gurus touring around the world under the motto: PE is translators' niche. It surprises nobody the fact that the agencies that support this are the ones that pay lousy rates. (I wonder who can expect the professional community to remain silent.) Regarding your comment "There is unbiased research out there, it's just kept quiet", I'm interested in learning more, and would like to get in touch with you. This is my email aurora.humaran@aipti.org . I'll much appreciate it if you can contact me.

Thanks!

Au

Dan Newland said...

This is an outstanding blog entry, Miguel. Thank you, thank you, thank you for not pussyfooting around with these robotics salesmen. (I note that Vashee's using an avatar instead of a photo now, so maybe that cyborg comment of yours isn't so far off, maybe he's actually morphed into an authentic terminator!)
I'm getting really tired of people telling me I should be more courteous to the very people who are shamelessly dumping on the market (and the profession) where I make a living providing quality translation. If Vashee isn't a "hired gun" (and I admit I have no proof he is, other than the impression his carnal relations with those who would globalize poor quality translation as the standard for the industry) then he is at least an unconscionable zealot whose only interest—for all of his utterly endless claptrap (the guy's everywhere spreading his BS like fresh manure on the fields of growing translation ignorance)—is lowering the standard for quality translation to abysmal bulk commodity levels.
As for ProZ, I'm thinking that if it continues to defend the indefensible corporate wholesale raping of the profession it claims to defend, this may well be the last year that I'm a paying member. If all serious translators did this (and I'm a little ashamed that I've waited so long) Henry would have to rethink his philosophy or turn ProZ into what it's fast becoming: An apologist site for ruthless commodity wholesalers and a sales outlet for software designed to replace genuine translation with a low-quality robotic substitute, not a site by and for real professional translators.

Paula R. said...

Concerning Jordi's comment that he does not know many translators making $40,000: there are many of us who have been making at least twice that (and much more) for years. If post-editing is the future, I'm done. I could make more as a secretary (or cleaning houses) than anyone could hope to make post-editing. And my brain would thank me.

Kevin Lossner said...

@Dan: Go for it, let your ProZ membership lapse and stop supporting a regime that has any interests but translators' as its focus. Anything you can do there you can do far better elsewhere. Drew's comment "We are happy to include translators on panels and discussions. It's something we've been trying to do without success." gave me a good laugh. Really. It's not that hard. I can think of a number of freelancers who wouldn't mind taking on the weasels in the hen house.

The current MT scam and attempts to bludgeon independent translators into supporting these schemes reminds me of "Y2K reloaded". In the late 1990s, I watched with great amusement as tired old COBOL programmers and others squeezed as much cash out of the scare as they could. Now we have the miracle of MT, a scam without a sell by date. The salesmen will bleed this one for all it's worth as long as they can until their victims, that is the customers and the postediting galley slaves, revolt and ignore them as they should.

Judy Jenner and Dagmar Jenner said...

Thanks for this great post, which Chris Durban just told us about. We could comment on the actual smart insight, but for now, we are busy laughing so hard we are (almost) crying. We particularly loved your Playboy Mansion analogy -- too funny. Thanks for the laugh. Saludos.

Anonymous said...

Hi guys,

I realize this comment is really late, but speaking of PE payment, I just finished a 2 year PE project that actually put some decent bucks in my coffers and was so incredibly easy for someone with my experience and education in translation. (10 yrs. experience, MA translation).

At the volume I could do on it (paid at discounted word rate), I earned about 100 bucks per hour. However, I agreed to it only because of the major concession on quality. This stuff was all internal and would not be published, so they didn't care about the quality. They asked for "understandable" and "good enough" quality.

"Good enough" and "understandable" are subjective. I often found many of the garbled, yet decipherable, sentences simply "good enough." ;-)

No complaints from them.

However, I have had agencies approach me with projects that need to be publishing quality--and they want a discount for MT--my response is usually to look at the MT version, find it unusable and needing of complete re-translation and offer them 2 options:

1) inflated word rate by 50% if they insist on use of the MT.
2) regular rate if they want it translated from scratch.

Needless to say, I don't normally get chosen for these kinds of ridiculous proposals. ;-)

MT can be used well on the type of project I described, which I just finished doing. It all depends on the content and the purpose of the target text.

I personally think that MT has no place in publishing quality work--not yet at least. No matter how many strides they make in MT, they are a loooooooooooong way off from replacing humans.

Whether PE becomes a viable market strategy will be up to us translators. I did it for a bunch of easy, repetitive internal garbage that is of interest only to the C suits of a particular company--and made very decent dough. (I also did not lose my style--I can still translate at proper quality too.)

If it becomes the standard for publishing quality work, well, I'll be looking for another profession.

Sorry for the anonymous post, I don't want this client to make a guess at who I am if they happen to stumble on this.

Great blog!