In the years that I’ve worked at ALTA, I’ve read a lot of literature about test design, validation techniques, and testing standards. Most of this literature comes in the form of dry, statistic-laden academic materials (apologies to all who enjoy statistic-laden academic materials), but one resource in particular stood out as informative as well as entertaining, and really put many aspects of what we deal with as a language company into focus. I refer to the book, The Spirit Catches You and You Fall Down by Anne Fadiman. As the subtitle explains, Fadiman’s wonderful book tells the story of “a Hmong child, her American doctors, and the collision of two cultures.”

I won’t divulge too much because I feel it is a must-read for anyone interested in language and culture, but here are the basics: A child of refugee parents from Laos, Lia Lee, has severe epilepsy and is taken to a small county hospital in California for treatment. Both parties become frustrated when they find that the Western medical practices and Eastern cultural practices don’t mesh. The Hmong parents are deeply connected to their cultural beliefs, which involve spiritual practices to help their daughter, whereas the hospital’s physicians are deeply concerned about Lia receiving the proper dosages of the medications they have provided to calm her seizures. The title of the book underscores the struggle that they both face:

The spirit catches you and you fall down is the closest translation of the word seizure into the Hmong language. How can a physician impress the need to treat a condition upon individuals who don’t believe the body contains organs? How can these individuals impress it upon the physician that they believe their daughter would be better off using healing ceremonies versus watching her suffer the side effects caused by the medication being prescribed?

The reason this book impacted me the way it did is Fadiman’s way of illustrating, not just the occasional difficulty of translating certain words, but that certain concepts are difficult to translate across cultural divides. This is the reason why we use highly skilled translators in our translation division, and why we concern ourselves with cultural equivalence in our language testing. We need to make sure that what we produce not only meets our client’s expectations in terms of the words in black and white, but also that it reflects the layers of meaning across cultures.

As a language testing provider, clients ask us to describe the added value of using an independent language testing organization such as ALTA. By having language tests administered independently, businesses, government agencies and healthcare organizations gain certain advantages:

Independent language testing is unbiased

The testing organization has no vested interest in the outcome of the tests; rather, its interest is in performing the job properly. The independent testing company has no incentive to pass or fail testing candidates because of other factors such as personality, need to fill positions or friendships. The independent testing company instead focuses only on the candidate’s performance in using the language. Since the testing company has a strong interest in maintaining its reputation for quality, the emphasis is on properly rating the candidate’s performance.

Language Testing providers use objective processes

While it is impossible to eliminate subjectivity completely in language testing, the testing provider’s aim is to minimize it. This is done through using pre-determined rating criteria, conducting rater training and norming sessions, and performing quality assurance practices. These practices ensure that raters are evaluating candidates in a standardized way. An independent testing company aims to make the process objective and continually works to improve the process.

Language testing providers can guarantee a test’s validity and reliability

By creating and administering tests that are both valid and reliable, the testing provider can ensure that the client is getting testing that meets industry standards for testing. This is important both to give assurance that testing candidates are being properly scored and to give the client the assurance that the testing measures the skills needed to perform the job or task for which the testing is being done.

Independent language testing uses up-to-date methods and technologies

Language testing is a technology application. Improvements are made from time-to-time in both the testing methodology as well as the supporting processes. An example of a methodology improvement is the use of video to simulate real-life interactions. An example of the supporting processes is the use of systems, such as online or automated telephone recording systems, which enable the testing provider to deliver the test using the most effective means available. Independent language testing companies utilize these technology improvements to provide competitive service to their clients.

Independent language testing is legally defensible

Occasionally, a client will make decisions based on the results of the testing and those decisions are challenged by an individual or group. These challenges can extend to legal challenges. All of the items described above give the client using independent testing a strong position to support the decisions that were made based on the testing results.

Back to Language Testing Home

______________________________________________________________________________________

ALTA Language Services, a national leader in language testing for global businesses, government agencies, and health care organizations, is the official independent language testing provider for several U.S. cities, including Los Angeles, California, where thousands of Spanish tests are administered each year.

We’ve all had the experience of taking a multiple choice test — one is given a question, and then has to choose the correct answer from a group of choices, usually “a,” “b,” “c,” and “d.”

From a test-taker’s perspective, these can sometimes be very intimidating. Does the following line of reasoning sound familiar?

Okay, the correct answer is there somewhere, and “a” looks pretty good — but wait, so does “c.” But “c” was the answer to the last three questions. Would they have made “c” the correct answer to four questions in a row? I doubt it. So maybe it’s “a.” But it could be “c” also. I just don’t know…It’s definitely not “d”, but “b” is also looking like a possibility now. Yeah, “a” could be right…

While test-takers have the difficult job of actually taking the test, the test-developers have a difficult task as well. Certainly, they are concerned with how the test questions are written, but another chief concern is how the answer choices perform. This includes all of them — the right ones and the wrong ones.

Read the rest of this entry »

You may have had an experience where you took a test for a class, and the instructor showed you the class’s results afterwards. The results probably mapped out into the infamous “bell-shaped curve”, with a few people scoring on the low end of the curve, the majority clustering in the middle, and a few people scoring on the high end. What you may not have known is that this is a normal distribution found in norm-referenced testing.

Norm-referenced tests (or NRTs) compare an examinee’s performance to that of other examinees. Standardized examinations such as the SAT are norm-referenced tests. The goal is to rank the set of examinees so that decisions about their opportunity for success (e.g. college entrance) can be made.

Criterion-referenced tests (or CRTs) differ in that each examinee’s performance is compared to a pre-defined set of criteria or a standard. The goal with these tests is to determine whether or not the candidate has the demonstrated mastery of a certain skill or set of skills. These results are usually “pass” or “fail” and are used in making decisions about job entry, certification, or licensure. A national board medical exam is an example of a CRT. Either the examinee has the skills to practice the profession, in which case he or she is licensed, or does not.

ALTA’s exams, which measure the candidate’s performance against pre-defined criteria (the performance levels), are also an example of criterion-referenced tests. Based on the passing level set by ALTA or the client, the results can then be used to determine whether or not the candidate meets the performance requirements of the job from a language standpoint.

Language testing has been all over the news lately, from new immigration policies requiring language testing for working immigrants, to globalization studies on hiring multi-lingual employees to conduct business in emerging markets throughout the world. Some business and government agency leaders may be considering language testing, but may not be sure who to trust to create, administer, and evaluate statistically valid assessments.

Test customization is one of the many services that ALTA provides, and clients often ask whether they should use a standard, general test, or whether they should have one created specifically for their organization. When answering this question, there are several factors to consider, such as how the test will be used, what type of test will be used, and what the goal of customization is.

Read the rest of this entry »

To verify that an employee or potential employee can speak or write another language at the appropriate level, you should use valid language testing.

In most professional circumstances, it takes more than a simple interview: you need to be confident that the language testing methodology you employ is both valid and reliable. Validity requires that the test accurately reflects the skill level required to perform the job. You want to know whether an employee or potential employee can use the language at the level you need, and legitimate testing services will offer protocols to measure performance versus a standard. ALTA’s language testing services address these needs for validity and reliability by providing the following:

• Valid testing methodology supported by statistical analysis
• Native speakers administering the tests
• Security of the test materials and results
• Process for reviewing and appealing test results.

Read the rest of this entry »