According to a new study just released by the University of Chicago, a foreign accent may undermine a person’s credibility. Participants who took part in the study were asked to rate the truthfulness of statements made by both native and non-native speakers of English.

The researchers found that truthfulness ratings were lower for statements made by non-native speakers than those made by native speakers. The severity of a person’s accent also affected the score; statements made by speakers with heavier accents were rated as less truthful than those with milder accents. “Because an accent makes a person harder to understand, listeners are less likely to find what the person says as truthful,” researchers concluded.

To help individuals reduce their accents and be better understood by native English-speakers, ALTA provides a variety of Accent Reduction Training programs, ranging from self-study to instructor-led training for individuals as well as corporate groups. These programs provide maximum efficiency and effectiveness in reducing one’s accent because the learner is able to focus only on the sounds that he or she needs to correct in order to sound more like a native English-speaker. An initial assessment pinpoints the individual’s particular problem areas, and provides a rating according to whether each is a minor, moderate, or a major problem area. The learner can then tackle those issues in an organized sequence to maximize his or her progress. ALTA’s methodology uses a series of steps to help the learner understand the right way to produce a sound, and then assimilate that sound into his or her natural speech. Further support is offered in the form of online materials that can be used to practice in between classes.

Changing one’s accent is by no means an easy task; an accent is essentially a habit of how you speak. The objective of an accent reduction training program like ALTA’s is to break an old habit of speaking and replace it with a new one. However, it is a task that can be accomplished with practice, and it breaks down a significant barrier for non-native English speakers.

                          

A psychometrician is someone who practices the science of educational and psychological measurement, or in other words, testing. Psychometricians measure the validity, reliability, and fairness of an exam program, and are an integral part in the process of creating valid and reliable language tests. They may be involved in all aspects of the development of the program, such as defining the knowledge, skills and abilities (KSAs) that the exam is designed to measure; they may also be involved in the development of the test specifications, the item writing process and the standard setting studies. While practicing or overseeing each of these areas, the psychometrician is highly concerned with ensuring that each part of the process conforms to testing industry standards, such as those in the Standards for Educational and Psychological Testing. These standards are considered the primary source for test publishers, test takers, and test users.

How Do I Become a Psychometrician?

There are currently no certifications for becoming a psychometrician; rather, people practicing in the field will often have an advanced degree from an Educational Measurement or Psychology program. Generally, a psychometrician needs skills and experience in quantitative analyses, critical thinking and statistics, as data analyses is an integral part of psychometrics. Many psychometricians go on to work for testing organizations. The Bureau of Labor Statistics estimates that the job outlook for psychometricians will grow faster than the average rate for all occupations, with excellent salary prospects.

A list of over fifty advanced degree programs in the U.S. and Canada that focus on Educational Measurement was compiled by the National Council on Measurement in Education (NCME). It can be found here.

                          

In the years that I’ve worked at ALTA, I’ve read a lot of literature about test design, validation techniques, and testing standards. Most of this literature comes in the form of dry, statistic-laden academic materials (apologies to all who enjoy statistic-laden academic materials), but one resource in particular stood out as informative as well as entertaining, and really put many aspects of what we deal with as a language company into focus. I refer to the book, The Spirit Catches You and You Fall Down by Anne Fadiman. As the subtitle explains, Fadiman’s wonderful book tells the story of “a Hmong child, her American doctors, and the collision of two cultures.”

I won’t divulge too much because I feel it is a must-read for anyone interested in language and culture, but here are the basics: A child of refugee parents from Laos, Lia Lee, has severe epilepsy and is taken to a small county hospital in California for treatment. Both parties become frustrated when they find that the Western medical practices and Eastern cultural practices don’t mesh. The Hmong parents are deeply connected to their cultural beliefs, which involve spiritual practices to help their daughter, whereas the hospital’s physicians are deeply concerned about Lia receiving the proper dosages of the medications they have provided to calm her seizures. The title of the book underscores the struggle that they both face:

The spirit catches you and you fall down is the closest translation of the word seizure into the Hmong language. How can a physician impress the need to treat a condition upon individuals who don’t believe the body contains organs? How can these individuals impress it upon the physician that they believe their daughter would be better off using healing ceremonies versus watching her suffer the side effects caused by the medication being prescribed?

The reason this book impacted me the way it did is Fadiman’s way of illustrating, not just the occasional difficulty of translating certain words, but that certain concepts are difficult to translate across cultural divides. This is the reason why we use highly skilled translators in our translation division, and why we concern ourselves with cultural equivalence in our language testing. We need to make sure that what we produce not only meets our client’s expectations in terms of the words in black and white, but also that it reflects the layers of meaning across cultures.

                          

Independent language testing is unbiased

The testing organization has no vested interest in the outcome of the tests; rather, its interest is in performing the job properly. The independent testing company has no incentive to pass or fail testing candidates because of other factors such as personality, need to fill positions or friendships. The independent testing company instead focuses only on the candidate’s performance in using the language. Since the testing company has a strong interest in maintaining its reputation for quality, the emphasis is on properly rating the candidate’s performance.

Language Testing providers use objective processes

While it is impossible to eliminate subjectivity completely in language testing, the testing provider’s aim is to minimize it. This is done through using pre-determined rating criteria, conducting rater training and norming sessions, and performing quality assurance practices. These practices ensure that raters are evaluating candidates in a standardized way. An independent testing company aims to make the process objective and continually works to improve the process.

Language testing providers can guarantee a test’s validity and reliability

By creating and administering tests that are both valid and reliable, the testing provider can ensure that the client is getting testing that meets industry standards for testing. This is important both to give assurance that testing candidates are being properly scored and to give the client the assurance that the testing measures the skills needed to perform the job or task for which the testing is being done.

Independent language testing uses up-to-date methods and technologies

Language testing is a technology application. Improvements are made from time-to-time in both the testing methodology as well as the supporting processes. An example of a methodology improvement is the use of video to simulate real-life interactions. An example of the supporting processes is the use of systems, such as online or automated telephone recording systems, which enable the testing provider to deliver the test using the most effective means available. Independent language testing companies utilize these technology improvements to provide competitive service to their clients.

Independent language testing is legally defensible

Occasionally, a client will make decisions based on the results of the testing and those decisions are challenged by an individual or group. These challenges can extend to legal challenges. All of the items described above give the client using independent testing a strong position to support the decisions that were made based on the testing results.

Back to Language Testing Home

______________________________________________________________________________________

ALTA Language Services, a national leader in language testing for global businesses, government agencies, and health care organizations, is the official independent language testing provider for several U.S. cities, including Los Angeles, California, where thousands of Spanish tests are administered each year.

                          

We’ve all had the experience of taking a multiple choice test — one is given a question, and then has to choose the correct answer from a group of choices, usually “a,” “b,” “c,” and “d.”

From a test-taker’s perspective, these can sometimes be very intimidating. Does the following line of reasoning sound familiar?

Okay, the correct answer is there somewhere, and “a” looks pretty good — but wait, so does “c.” But “c” was the answer to the last three questions. Would they have made “c” the correct answer to four questions in a row? I doubt it. So maybe it’s “a.” But it could be “c” also. I just don’t know…It’s definitely not “d”, but “b” is also looking like a possibility now. Yeah, “a” could be right…

While test-takers have the difficult job of actually taking the test, the test-developers have a difficult task as well. Certainly, they are concerned with how the test questions are written, but another chief concern is how the answer choices perform. This includes all of them — the right ones and the wrong ones.

Each test item is made up of three parts: the stem (the question), the key (the correct answer choice), and the distractors (the incorrect answer choices). The key and the distractors are sometimes referred to collectively as the alternatives. The idea behind each item is to present a question that the test-taker, having the requisite level of knowledge about the particular skill area, would answer correctly when given a collection of possible answers. A test-taker who has less than the requisite level of knowledge about what’s being tested might choose the right answer to a question (sometimes by guessing), but there is some likelihood that he or she will select one of the incorrect distractors — if the question and answers are created properly, that is. Simply put, each question has the job of distinguishing between test-takers who have the level of knowledge needed to pass the test and those who do not.

When writing each question, test-developers need to ensure that there is one — and only one — best answer to each question. But they also need to ensure that the distractors are doing their job of, well, distracting the less-skilled test-takers. Each distractor must be a plausible option; otherwise it does not serve this purpose. Take a look at the following example:

1. According to the passage, the Earth’s core contains levels of which elements?

a. nickel, iron, and gold
b. iron, sulfur, and carbon
c. oxygen, lithium, and neon
d. red, white, and blue

In this example, option “d” is clearly ridiculous and could be quickly eliminated, giving less-skilled test-takers a greater probability of selecting the correct option. Therefore, this answer choice is not doing its job of helping to distinguish among candidates. The test-developer would want an alternative that followed the form of the other three options.

So how does the developer know how well the answer choices are performing?

After the administration of a test, the responses can be analyzed to see what percentage of candidates selected each option. The analysis is performed to determine a figure that test developers call Item Facility (IF).

The IF is used to quantify the difficulty presented by each test answer choice. It is equal to the number of correct answers divided by the number of respondents. An IF value of 1.00 means that all respondents got the question correct, and therefore the item may be too easy. Conversely, an IF of 0.00 mean no one got the question correct, and that the item may be too difficult. In norm-referenced testing, items with extreme values would be considered of little value for distinguishing performance and would be removed or modified, as this type of testing is designed to spread out examinees’ scores across a normal distribution.

The ideal IF range for these test items is between a 0.4 and a 0.6, meaning that the majority of test-taker’s scores are clustered under the “bell-shaped curve.” However, criterion-referenced testing has to do with the mastery of the skill or subject matter being tested, and in these cases, it is common to see more extreme IF values.

From this type of analysis, among others, test-developers can go back and improve the quality of the items so that the whole test performs well in determining who has the skills to succeed at whatever the test is measuring.

                          

You may have had an experience where you took a test for a class, and the instructor showed you the class’s results afterwards. The results probably mapped out into the infamous “bell-shaped curve”, with a few people scoring on the low end of the curve, the majority clustering in the middle, and a few people scoring on the high end. What you may not have known is that this is a normal distribution found in norm-referenced testing.

Norm-referenced tests (or NRTs) compare an examinee’s performance to that of other examinees. Standardized examinations such as the SAT are norm-referenced tests. The goal is to rank the set of examinees so that decisions about their opportunity for success (e.g. college entrance) can be made.

Criterion-referenced tests (or CRTs) differ in that each examinee’s performance is compared to a pre-defined set of criteria or a standard. The goal with these tests is to determine whether or not the candidate has the demonstrated mastery of a certain skill or set of skills. These results are usually “pass” or “fail” and are used in making decisions about job entry, certification, or licensure. A national board medical exam is an example of a CRT. Either the examinee has the skills to practice the profession, in which case he or she is licensed, or does not.

ALTA’s exams, which measure the candidate’s performance against pre-defined criteria (the performance levels), are also an example of criterion-referenced tests. Based on the passing level set by ALTA or the client, the results can then be used to determine whether or not the candidate meets the performance requirements of the job from a language standpoint.

                          

Test customization is one of the many services that ALTA provides, and clients often ask whether they should use a standard, general test, or whether they should have one created specifically for their organization. When answering this question, there are several factors to consider, such as how the test will be used, what type of test will be used, and what the goal of customization is.

How will the language test be used?

Many of our clients are using proficiency testing as part of a hiring or selection process. In this case, we typically recommend using a general test, as this test is aimed at measuring global language proficiency, i.e. use of the language that is not dependent on knowing industry-specific terminology or being able to talk about industry-related topics. This puts candidates on an even playing field when testing. It should also be used if this knowledge is acquired on the job or in training programs. Typically, candidates who have a strong command of the language can easily supplement their vocabulary with an organization’s terms, and discuss related topics with ease. However, if candidates are expected to be able to speak about the organization’s topics in the target language – whether upon hiring, or after being on the job – then a customized test can be appropriate.

What type of language test do you need?

In terms of test type, there are some caveats to consider when having a test customized. For instance, an oral or written test can prompt the candidate with your organization’s topics, but does not guarantee that the candidate will respond using industry-specific terminology. Secondly, ALTA’s evaluators are measuring overall use of the language. There are no “right” or “wrong” answers to questions found on these types of tests. Therefore, if the objective is to elicit certain responses, these test types do not satisfy this goal.

What is the goal of language testing for your organization?

If you do want to measure knowledge of certain terms or phrases used in your industry, an online test, which is set up to determine correct versus incorrect responses, might be a better model to use. ALTA can develop such a test for your organization, in addition to tests that measure the overall use of a language. If your testing needs require a more complex solution, or if you are not certain what type of skill measurement would best suit your organization, feel free to contact us at 404.920.3820 for a free consultation and language testing estimate.

                          

In most professional circumstances, it takes more than a simple interview: you need to be confident that the language testing methodology you employ is both valid and reliable. Validity requires that the test accurately reflects the skill level required to perform the job. You want to know whether an employee or potential employee can use the language at the level you need, and legitimate testing services will offer protocols to measure performance versus a standard. ALTA’s language testing services address these needs for validity and reliability by providing the following:

• Valid testing methodology supported by statistical analysis
• Native speakers administering the tests
• Security of the test materials and results
• Process for reviewing and appealing test results.

There are four types of language communication skills: speaking, listening, reading and writing. It’s likely that the positions you need filled have different requirements in the two languages for each skill. In many cases one or more of the skills may not be needed at all.

Defining what language skills are needed in both languages is an important step in defining your need for a bilingual employee. Almost no one operates at the native level in two languages in all four types of language skills. The needed types of language capabilities can be identified by looking at the job functions and how the person will interact with people who speak the first language and with people who speak the second language.Determining the type of language skill needed is important, and ALTA provides assistance as part of its Language Review Service.

DEFINING LANGUAGE CAPABILITY:

It is important to identify the level of capability (e.g. native-fluent, nearly fluent, advanced, beginner, etc.) required to perform a particular job function.

A call-center agent and an online Latin American sales representative both need to have high-level bilingual skills, but their respective jobs require distinct skills. There are no international standards for defining levels of language capability. Several countries have government-authorized definitions of “correct” language usage for the language in that country, but there is no agreement on standards outside specific countries or across languages.

In the U.S. two organizations have set cross-language standards that apply to their specific needs: the Interagency Language Roundtable (ILR) standards for U.S. government use, and the American Council for the Teaching of Foreign Languages (ACTFL) for academic use. Also, there are specialized exams that test one or more types of language communication in one language only, such as the Test of English as a Foreign Language (TOEFL), and the Cambridge exam in the UK.

ALTA’s performance levels and testing protocol were designed to provide language testing standards that are meaningful across languages and cover the full range of performance levels.