How to test your translation partners?
In last week’s post, we have proposed a full process on how to select good translation partners for your translation projects. In which, the step of testing translation partners before deciding to work with them can be considered as one of the most important stages that has a direct impact on the translations’ quality later. This step, needless to say, is far from easy and requires a great amount of time to handle.
To make it easier for you to test your translation partners we suggest you apply the 7-step process below which was built based on our team’s experience of working with different types of translation partners including in-house translators, freelance translators, and translation agencies.
Table of Contents
- Step 1 – Assign a Review panel
- Step 2 – Prepare a translation guideline
- Step 3 – Create a measurement method
- Step 4 – Prepare a test for your translation partners
- Step 5 – Send the test to your translation partners
- Step 6 – Evaluate the test of your translation partners
- Step 7 – Notify the results to your candidates
Step 1 – Assign a Review panel
You should start off by defining who will be in charge of testing your translation partners. This team’s main responsibility is to review and evaluate the quality of the test translation to find the most suitable partners for your translation projects. Therefore, it should consist of staff from different related departments to guarantee the best testing results:
- A reviewer – who understands the linguistic elements and specific company’s requirements (such as tones of voices, do-not-translate list, etc.). It is ideal to have an experienced internal translator to handle the task because the translator would understand thoroughly your company’s content and common mistakes to meet your expectations. If you don’t have one, then you should hire a senior reviewer or translation agency to help you with this task.
- A subject matter expert – who understands the terms used by your company or use them on a regular basis to review the accuracy of terms used in the test translation.
Step 2 – Prepare a translation guideline
This step is optional. If your test volume is quite small and the content is not too specialized, then you can skip this step. Otherwise, you should create a translation guideline with 3 key documents below to help your translation partners understand better your requirements and preferences:
(1) Translation Style Guide which includes a set of rules about writing style, translation tone, spelling, punctuations, brand voice, and other textual and visual elements to remain a consistent brand voice for your company;
(2) Translation Glossary which is a list of standardized key terminology appeared in documents, their definitions and translations;
(3) Termbase which is a list of key terminology appeared in documents and their definitions.
Please note that the translation guideline in the testing process is mainly used to test translators’ abilities to follow your instructions and requirements rather than testing their ability to perform exactly like the actual project.
Step 3 – Create a measurement method
As the definition of a good translation is subjective, it is of great importance that you create a measurement strategy or KPIs with clear indications before you test your translation partners. These indications will help you evaluate the candidates’ translations accurately, quickly, and fairly.
The metrics for each project vary based on its specific requirements. In general, your measurement strategy should include 3 key metrics as follows:
- LQA report form – The LQA form is used to evaluate the linguistic quality of a translation. The form covers a wide range of error categories such as spelling, grammar, and layout. Each category will be evaluated based on the error severity which is categorized by Minor, Major, and Critical with weighting figures. Below is an example of an LQA form. The left column is the error categories, the three remaining columns are error severity. The weight figures for minor, major, and critical error are 1, 5, and 10 respectively. For instance, if the translation contains 2 major errors in terminology, it will generate a score of 10.
DOWNLOAD A FREE LQA FORM
- Pass Rate – You need to set a number (predefined threshold) in which a test is considered as pass or fail. This number is calculated by adding the scores of all segments from the QA metric above. If the total scores are lower than your threshold, the test is considered “fail” and you can strike this partner out of your list.
- Delivery time – This is an important metric to test your translation partners’ work ethic and attitude. Do they deliver the translation on time? In case they cannot meet the deadline, do they inform you and ask for a deadline extension or they simply send you the translation 2-day-late without any explanation and apologies? Because the delivery time can tell a lot about the working styles and responsibility of your translation partners’ candidates.
Step 4 – Prepare a test for your translation partners
Most translation agencies and freelance translators agree to do a free test of under 300 words. If you want them to perform a test with a higher volume, it is sensible to see the test as a small, paid pilot project to show your respect for their time and effort.
In addition, the test should contain content and terms relating to the actual projects to guarantee that your translation partners can handle that specific type of content well. The format of a test normally consists of 3 columns: (1) The source sentences (filled in by your team); (2) The translations (will be done by translators), and (3) Translator’s notes.
The notes column is used for translators to explain why they choose a specific term or their references. This helps the translation management team understands how translators create their translations, having a thorough view of their abilities, mindset and responsibility, instead of solely focusing on the translation quality.
Step 5 – Send the test to your translation partners
Before the test, you might communicate with your translation partners via instant messages such as Skype, Wechat, or Whatsapps for quicker response. However, in this stage of communication, we recommend you use business emails to send the candidates the prepared test. This formal way of communication shows your professionalism and avoid missing information due to a large number of instant messages.
In the email, you should include the following information and files:
- The translation guideline you have prepared in step 2;
- Detailed instructions for translators;
- Deadline for the test.
Step 6 – Evaluate the test of your translation partners
Besides evaluating their capability to handle your company’s content, the test should also help you test your translation partners in terms of their working attitudes which have a great influence on the cooperation between two parties later on.
- Quality evaluation – Apply the measurement strategy you have created in step 3. This step should be conducted by senior linguists who have experience in translating your company’s documents before or/and insight into your brand’s voice to evaluate the translations better.
- Attitude evaluation – Is the communication style clear? Are they responsive to your requests? If they miss the deadline, do they notify you? Are they willing to improve their translation quality according to your comments? Considering these questions will give you find the answer on whether they are desirable translation partners you should work with.
Step 7 – Notify the results to your candidates
A common mistake a lot of people make in this step is only notifying the results to the passed candidates and ignoring the failed ones. No matter what the result is, it is necessary that you inform all your translation candidates of whether they get the job or not along with your comments on the errors they made in the test or why they failed. The feedback will be beneficial for both the translators and you.
From the translator’s side, they can improve their translation and/or communication skills when communicating with other clients in the future. From your side, translators sometimes do not agree with your feedback and write you back with their comments on the errors you have marked. Both sides will discuss the problems to reach an agreement. This process might give you an insight into the language or improve your translation quality.