With over 30 years in the language services business, ALTA has built a reputation as a trusted provider of valid and reliable language tests. Our clients depend on us to help them create defensible assessment programs whether through the use of our standard language tests, or through the customization of tests specifically for their companies, organizations, or agencies.
Validity and reliability are the two most important aspects of any testing program, because without them, the tests are meaningless. Validity refers to the ability of the test to measure what it purports to measure. Reliability is the ability of the test to be repeated and yield consistent results. For instance, imagine that a driving test asked you about banking and finance terms. Would that test be valid? Similarly, think about stepping on a scale and getting a different reading every time. Would that scale be reliable? In both cases, the results would be useless since they cannot be interpreted in a meaningful way. ALTA’s job is to create valid and reliable tests that deliver meaningful results so that you can defensibly make decisions about your candidates.
ALTA’s Job Analysis is the first step for many companies in determining which type of test is right for them, and the cut score on ALTA’s scale to use to determine who is selected for the job and who is not. The job analysis consists of an in-depth review of documentation and interviews with employees who perform and supervise the position, i.e., your subject matter experts. Using this two-pronged approach, ALTA can identify the tasks in which the language is used and the level of proficiency needed to perform those tasks. Following the interviews and document review process, we provide you with a report detailing our findings and recommendations. This gives you a solid foundation for deciding how you use the test results, and provides the defensibility you need should these decisions ever be challenged.
In most cases, our clients use ALTA’s standard language tests. These exams measure general proficiency in the target language and only assume that candidates will be able to integrate any job-specific terminology into their communication. However, some clients want to verify that their employees can converse or write about topics that arise in their business situations, and can understand their unique terminology. In these cases, ALTA will provide customized tests. Customization may include modifying a standard test to include prompts that mirror the types of scenarios an employee may encounter on the job, or may simply verify knowledge of terminology in English and the target language.
When designing and creating a test, many steps go into making sure that it is valid (i.e., measures what it purports to measure) and reliable (i.e., it can be repeated and will yield consistent results). In total, there are 9 major steps:
It is important to note that many clients will use the standard tests, which have already undergone this process of validation, and simply choose to employ ALTA to set their cut-score through the Job Analysis and administer the test. This is because the definition of a minimally qualified candidate can change from job to job, or from one organization to another.
No matter what kind of testing is used, ALTA pays critical attention to the quality assurance of its tests. One of the ways ALTA ensures reliability of its language test results is through scrutinizing our inter-rater reliability. This is the degree to which two independent raters would provide the same score on the same exam. To collect and monitor this data, ALTA routinely selects recorded assessments at random and distributes them to a second evaluator for scoring. This score is compared with the original evaluator’s score, and any discrepancies are resolved. This process also helps identify any opportunities for retraining or any revisions to the scoring criteria that need to be made. We are proud to say that we consistently maintain inter-rater reliability at exceptional levels.