What is a Psychometrician?

A psychometrician is someone who practices the science of educational and psychological measurement, or in other words, testing. Psychometricians measure the validity, reliability, and fairness of an exam program, and are an integral part in the process of creating valid and reliable language tests. They may be involved in all aspects of the development of the program, such as defining the knowledge, skills and abilities (KSAs) that the exam is designed to measure; they may also be involved in the development of the test specifications, the item writing process and the standard setting studies. While practicing or overseeing each of these areas, the psychometrician is highly concerned with ensuring that each part of the process conforms to testing industry standards, such as those in the Standards for Educational and Psychological Testing. These standards are considered the primary source for test publishers, test takers, and test users.

How Do I Become a Psychometrician?

There are currently no certifications for becoming a psychometrician; rather, people practicing in the field will often have an advanced degree from an Educational Measurement or Psychology program. Generally, a psychometrician needs skills and experience in quantitative analyses, critical thinking and statistics, as data analyses is an integral part of psychometrics. Many psychometricians go on to work for testing organizations. The Bureau of Labor Statistics estimates that the job outlook for psychometricians will grow faster than the average rate for all occupations, with excellent salary prospects.

A list of over fifty advanced degree programs in the U.S. and Canada that focus on Educational Measurement was compiled by the National Council on Measurement in Education (NCME). It can be found here.

Since earning recognition for Innovation in Multicultural Health Care, Kaiser Permanente’s Clinician Cultural Linguistic Assessment language testing tool, which is administered by ALTA, has been receiving a lot of positive attention from people in the medical community. ALTA receives a mention in the following article from the California Healthfax publication:

Read the rest of this entry »

As our nation’s policy makers and health care providers debate the future of health care reform, ALTA and Kaiser Permanente are working together to improve the quality of care for ethnic minorities and multilingual patients.

Having recently been recognized with an award for Innovation in Multicultural Health Care from The National Committee for Quality Assurance (NCQA), ALTA and Kaiser are pleased that their language testing program is making a positive impact. The award recognizes the Clinician Cultural and Linguistic Assessment for, “effectively and compassionately addressing race, ethnicity, and language proficiency in the delivery of quality, cost-effective services,” (Robert Ross, CEO of The California Endowment).

The Clinician Cultural and Linguistic Assessment (CCLA) is a tool developed by Kaiser Permanente and administered by ALTA to determine the level of target language proficiency of bilingual physicians. The CCLA is designed to assess target language communication (speaking, reading, and listening comprehension) between doctors and patients in a linguistically accurate and culturally sensitive manner without the use of an interpreter. The CCLA is the first and only validated and reliable language assessment tool of its kind, and sets a standard level for effective communication in a medical setting. The results are used to match patients with physicians who are fluent in their language. ALTA, a national leader in language testing, has administered the assessments for Kaiser Permanente since September of 2007.

Recognition of the ALTA-administered language assessment comes as U.S. health care providers increasingly face the challenges of linguistic diversity. A study conducted by The Institute of Medicine recently found that ethnic minorities tend to receive lower quality health care than non-minorities, even when factors such as insurance status and income level are controlled. Kaiser developed its language assessment program to remedy this situation as the number of ethnic minorities in need of health care is likely to increase in the coming decades.

According to NCQA president Margaret O’Kane, “one in five Americans speaks a language other than English at home…it’s imperative that health care reform efforts address the impact of disparities on the quality and cost of U.S. health care.” In addition, a 2005 study conducted by the National Alliance for Hispanic Health found that the number of elderly Spanish speakers in the U.S. will climb to 13 million by 2050, even as the U.S. Census Bereau reports that latinos 65 years and older are projected to outnumber all other elderly minorities by 1 million as soon as 2030. The CCLA is offered in Spanish and 13 other languages.

In the years that I’ve worked at ALTA, I’ve read a lot of literature about test design, validation techniques, and testing standards. Most of this literature comes in the form of dry, statistic-laden academic materials (apologies to all who enjoy statistic-laden academic materials), but one resource in particular stood out as informative as well as entertaining, and really put many aspects of what we deal with as a language company into focus. I refer to the book, The Spirit Catches You and You Fall Down by Anne Fadiman. As the subtitle explains, Fadiman’s wonderful book tells the story of “a Hmong child, her American doctors, and the collision of two cultures.”

I won’t divulge too much because I feel it is a must-read for anyone interested in language and culture, but here are the basics: A child of refugee parents from Laos, Lia Lee, has severe epilepsy and is taken to a small county hospital in California for treatment. Both parties become frustrated when they find that the Western medical practices and Eastern cultural practices don’t mesh. The Hmong parents are deeply connected to their cultural beliefs, which involve spiritual practices to help their daughter, whereas the hospital’s physicians are deeply concerned about Lia receiving the proper dosages of the medications they have provided to calm her seizures. The title of the book underscores the struggle that they both face:

The spirit catches you and you fall down is the closest translation of the word seizure into the Hmong language. How can a physician impress the need to treat a condition upon individuals who don’t believe the body contains organs? How can these individuals impress it upon the physician that they believe their daughter would be better off using healing ceremonies versus watching her suffer the side effects caused by the medication being prescribed?

The reason this book impacted me the way it did is Fadiman’s way of illustrating, not just the occasional difficulty of translating certain words, but that certain concepts are difficult to translate across cultural divides. This is the reason why we use highly skilled translators in our translation division, and why we concern ourselves with cultural equivalence in our language testing. We need to make sure that what we produce not only meets our client’s expectations in terms of the words in black and white, but also that it reflects the layers of meaning across cultures.

(click image to enlarge)
–
To determine if you qualify & to inquire, please visit:
www.altalang.com/beyond-words/help-alta-develop-proficiency-tests/
–
Original Artwork Courtesy: banlon1964

The mastery of foreign languages allows us to broaden national boundaries and blur international horizons. By learning a new language and the culture of a people, we gain the ability to move fluidly in the global marketplace.

Now more than ever, foreign language skills provide employees and employers with a depth and breadth of resources to reach clients around the world. Increasing globalization has changed the market, creating a wealth of careers for which knowledge of a foreign language is crucial. Here are just a few of these fields:

Read the rest of this entry »

As a language testing provider, clients ask us to describe the added value of using an independent language testing organization such as ALTA. By having language tests administered independently, businesses, government agencies and healthcare organizations gain certain advantages:

Independent language testing is unbiased

The testing organization has no vested interest in the outcome of the tests; rather, its interest is in performing the job properly. The independent testing company has no incentive to pass or fail testing candidates because of other factors such as personality, need to fill positions or friendships. The independent testing company instead focuses only on the candidate’s performance in using the language. Since the testing company has a strong interest in maintaining its reputation for quality, the emphasis is on properly rating the candidate’s performance.

Language Testing providers use objective processes

While it is impossible to eliminate subjectivity completely in language testing, the testing provider’s aim is to minimize it. This is done through using pre-determined rating criteria, conducting rater training and norming sessions, and performing quality assurance practices. These practices ensure that raters are evaluating candidates in a standardized way. An independent testing company aims to make the process objective and continually works to improve the process.

Language testing providers can guarantee a test’s validity and reliability

By creating and administering tests that are both valid and reliable, the testing provider can ensure that the client is getting testing that meets industry standards for testing. This is important both to give assurance that testing candidates are being properly scored and to give the client the assurance that the testing measures the skills needed to perform the job or task for which the testing is being done.

Independent language testing uses up-to-date methods and technologies

Language testing is a technology application. Improvements are made from time-to-time in both the testing methodology as well as the supporting processes. An example of a methodology improvement is the use of video to simulate real-life interactions. An example of the supporting processes is the use of systems, such as online or automated telephone recording systems, which enable the testing provider to deliver the test using the most effective means available. Independent language testing companies utilize these technology improvements to provide competitive service to their clients.

Independent language testing is legally defensible

Occasionally, a client will make decisions based on the results of the testing and those decisions are challenged by an individual or group. These challenges can extend to legal challenges. All of the items described above give the client using independent testing a strong position to support the decisions that were made based on the testing results.

Back to Language Testing Home

______________________________________________________________________________________

ALTA Language Services, a national leader in language testing for global businesses, government agencies, and health care organizations, is the official independent language testing provider for several U.S. cities, including Los Angeles, California, where thousands of Spanish tests are administered each year.

The Angoff Method is a widely used standard-setting approach in test development.

In plain English, it is a kind of study that test developers use to determine the passing percentage (cutscore) for a test. The passing grade of a test can’t be decided arbitrarily; it must be justified with empirical data. The Angoff method relies on subject-matter experts (SMEs) who examine the content of each test question (item) and then predict how many minimally-qualified candidates would answer the item correctly. The average of the judges’ predictions for a test question becomes its predicted difficulty. The sum of the predicted difficulty values for each item averaged across the judges and items on a test is the recommended Angoff cut score. Here is a real world example that illustrates the process:

Read the rest of this entry »

We’ve all had the experience of taking a multiple choice test — one is given a question, and then has to choose the correct answer from a group of choices, usually “a,” “b,” “c,” and “d.”

From a test-taker’s perspective, these can sometimes be very intimidating. Does the following line of reasoning sound familiar?

Okay, the correct answer is there somewhere, and “a” looks pretty good — but wait, so does “c.” But “c” was the answer to the last three questions. Would they have made “c” the correct answer to four questions in a row? I doubt it. So maybe it’s “a.” But it could be “c” also. I just don’t know…It’s definitely not “d”, but “b” is also looking like a possibility now. Yeah, “a” could be right…

While test-takers have the difficult job of actually taking the test, the test-developers have a difficult task as well. Certainly, they are concerned with how the test questions are written, but another chief concern is how the answer choices perform. This includes all of them — the right ones and the wrong ones.

Read the rest of this entry »