From a young age, our lives are filled with assessments: standardized tests, driving exams, placement tests, and, if a business or government agency tests the English or foreign language skills of potential hires and current staff, there is a good chance that they use ALTA language testing. Creating a valid assessment of an individual’s language skill is an important and difficult task: one that requires us to take many steps to guarantee that every test we administer meets the Standards for Educational and Psychological Testing.

So, what does it mean for a test to be valid?

In psychometric terms, a test’s validity is the degree to which the theory behind the test and the interpretation of the test’s score accurately measure the test’s intended purpose. In other words, a valid language test works to assess language ability, and the scores can be defended.

ALTA spends a lot of time and resources to ensure that our language tests are valid. While the process is complex, here is a basic synopsis of 9 important steps we take, whereby each step contributes to the overall validity of a language assessment. Figure 1 illustrates this validation cycle, and each step is described below:

Figure 1: Validation Cycle

1. Job Analysis/KSAs:

The first step in test development is to identify the knowledge, skills, and abilities (KSAs) that the test will be designed to measure. For tests that are designed to qualify an individual to perform a specific job, these KSAs are identified through the performance of a job study in which individuals knowledgeable of what the job entails – or, subject-matter experts (SMEs) – are interviewed to collect this information. The identification of KSAs is a crucial step in providing focus to the development efforts that follow.

2. Create Test Blueprint:

The test blueprint is created based on the KSAs identified, and their relative importance to the job. The blueprint specifies to the test developer the content that will be included in the test, the amount of content in each skill area, and any other instructions needed to properly develop the content. Using the blueprint as a guide, test developers are engaged to create the actual test items.

3. Create Test Items:

Item development is carried out according to the specifications outlined in the test blueprint. More than the ample amount of test items are created to allow for the possibility that some of the items will need to be eliminated based on pilot-testing and item analysis results.

4. Review Test Items:

All test items are submitted to a separate panel for review and comment. This panel reviews each test item and verifies that each aligns with the specifications as outlined in the test blueprint. Any need for modification is recorded, and comments are provided to the developers so that the appropriate changes can be made. This review process is repeated for any changes that are made until the pilot version of the test is complete.

5. Pilot Test Items:

Once the final draft version has been reviewed and approved by test developers and the review panel, the items are pilot-tested to gather data around item performance. Pilot testing is done using a sample of candidates representative of the target population. Following the pilot-testing, psychometric analysis is performed on the results to determine the test’s performance.

6. Create Final Test Form:

Results from the statistical analysis yield the items that will constitute the final test form, and these items assembled into the operational version of the test.

7. Angoff Method:

Using the final test versions, an Angoff panel is assembled to determine the cut-score of the test, or the percentage of correctly-answered items that the candidate needs to successfully pass the test. Although various standard-setting methods exist, ALTA typically uses the Angoff method, which relies on the judgments of the panel as to the percentage of minimally-qualified candidates who would perform successfully on each item.

8. Administer Test:

Upon determining the cut-scores for the final test versions, the tests are available for operational use and are administered according to the operational policies set up by the test administrator using a prescribed scoring rubric.

9. Quality Assurance:

Quality assurance is performed continuously to ensure that the items are performing properly over time. Quality assurance also provides a method for monitoring overexposure and identifying items which may have been compromised.

It is important to note that validation is a cycle, and testing organization should continue reviewing the test and collecting evidence of the test’s validity. At various points in the lifetime of a test, each step may be revisited for review and/or revision.

____________________________________________________________________________________________

ALTA is a leader in language testing and large-scale language solutions for government agencies and corporations nationwide. In addition to being the official language testing provider for the cities of Los Angeles and New York, ALTA works with many of the country’s largest corporate organizations, from DELTA Airlines to Wells Fargo. Learn more about us at altalang.com

ALTA has completed a comprehensive study of court interpreter exams for the Judicial Council of California. The published report, California’s Assessment of the Consortium for Language Access in the Courts’ Exams, appears on the California Court’s website.

In May of 2009, the Judicial Council of California contracted ALTA to assess the Consortium for Language Access to the Courts’ (CLAC or Consortium) test development process. The aim of the study was to perform a comparative analysis between the examinations developed by CLAC to certify its member state court interpreters and the examinations developed by California to certify California state court interpreters.

The study includes literature and document reviews and interviews with members of the Consortium’s staff both past and present, including language consultants and exam writers. Additionally, ALTA and a team of psychometricians performed a psychometric audit of the Consortium’s test development process to identify its key strengths and any areas that were in need of improvement to meet test industry standards according to the Standards for Educational and Psychological Testing (AERA, APA, & NCME, 1999.)

Once the psychometric audit was complete, a group of subject-matter experts examined both the written and oral Consortium exams to identify the knowledge, skills, and abilities required for successful court interpreting , as determined in the Study of California’s Court Interpreter Certification and Registration Testing. Results from the comparative analysis showed a high degree of overlap between the California and CLAC written and oral exam KSAs, as well as a strong degree of comparability between the structure, content, and level of difficulty found in the oral exams for each program.

The final step of the study involved performing a standard-setting session applying the Angoff Method to establish the level of functional equivalency between the passing requirements of each program’s oral examinations.
____________________________________________________________________________________________

ALTA is a leader in language testing and large-scale language solutions for government agencies and corporations nationwide. In addition to being the official language testing provider for the cities of Los Angeles and New York, ALTA has worked with the Judicial Council of California on language testing studies for several years. Learn more about us at altalang.com

A new online resource for legal interpreters and translators has recently become available thanks to Vancouver Community College. A team of language professionals collaborated to compile an online legal dictionary consisting of 5000 Canadian legal and court-related terms in English, as well as six other languages: Chinese, Farsi, Punjabi, Russian, Spanish, and Vietnamese. This multilingual glossary of legal terms can be found here: Multilingual Legal Glossary

The resource is the result of the collaborative efforts of terminologists, bilingual lawyers, legal translators, and linguists, as well as other qualified and experienced members of the legal and multicultural communities of both Canada and the country of origin of each language included.

A quick search of one commonly used legal term, “defendant,” yielded excellent results in Spanish, providing a thorough definition for the term in both English and Spanish. This is an excellent resource for anyone working within the legal arena, especially legal translators in Canada. While the dictionary aims to “explain Canadian legal terminology and to provide foreign language equivalents for concepts related to Canadian law,” many of the included concepts and terms are universal across the English language.

Add this site to your working list of references and support Vancouver Community College’s efforts! To help spread the word, ALTA is going to add a link on the Free Resources for Translators page. I hope to see more online reference resources of this nature in the coming years.

Hurray! Yipee! Wowee! Let’s hear it for my favorite part of speech: the interjection! Interjections help to express emotion on the part of the speaker in written language. They are usually separated off from the rest of a sentence by commas or exclamations points. In any language, interjections make the written word more expressive.

Here is a list of my personal favorites in a few different languages:

Read the rest of this entry »

I recently read a book titled The Origins of Simultaneous Interpretation: The Nuremburg Trial that I thought I would recommend given that the pool of information regarding the history of interpreting can be somewhat limited, or difficult to find.

Written by Fracesca Gaiba, Origins describes how the Nuremburg Trial, conducted from November 1945 to August 1946, was the first international gathering in which simultaneous interpretation was used. Gaiba recreates the events that brought simultaneous interpreting to the court system through a combination of interviews with Nuremburg interpreters and staff, as well as extensive literature and audio review of resources pertaining to the trial. Four languages, English, French, Russian, and German were used throughout the trial. Every listener in the courtroom was equipped with a headset and a selector switch at his or her seat. The listener could then choose one of five “channels” he or she wanted to listen to during the trial: the verbatim speech, English, French, Russian, or German.

In Gaiba’s words:

Read the rest of this entry »

California has set a precedent yet again in the world of interpreting by becoming the first state in the U.S. to pass a law requiring that health insurance organizations provide interpreting and translating services to patients with limited English proficiency (LEP).

The law “requires health, dental and specialty insurers to provide subscribers with translators (sic), at least by telephone, while visiting their doctor, pharmacist, ophthalmologist or dentist.”

Senate Bill 853 was officially signed in 2003, but insurance companies had until January 1, 2009 to comply with the bill.

Kudos to California for being at the forefront of providing language services to LEP residents. Let’s hope that other states follow suit.

La Malinche, Doña Marina, La Chingada. No matter what name you use, there is no doubt that she is one of the most influential interpreters in history.

Doña Marina, who came to be known in Mexico as La Malinche, was born the daughter of a cacique during the rule of the Aztecs in the early 1500s. As the daughter of a cacique, she was considered part of the noble class and allowed the opportunity to attend school. Not much else is known about her early years except that her father died and she was sold into slavery (it is not known whether she was sold by her family or kidnapped and then sold), eventually ending up in the hands of Hernan Cortes and the Spaniards during their conquest of the Aztecs.

It is also not known how Cortes became aware of Malinche’s linguistic skills, but within a few weeks of being sold to him she had begun interpreting. In one of the first documented incidences of relay interpreting, La Malinche and a Spanish priest named Gerónimo Aguilar worked as a team to interpret for Cortes by transferring Nahuatl (the Aztec language) to the Chontol Mayan language to Spanish.

They continued this practice until La Malinche was able to learn Spanish and interpret directly from Nahuatl to Spanish for Cortes. Bernal Diaz del Castillo, an eyewitness to Malinche’s skill as an interpreter, stated that “Without the help of Doña Marina, we would not have understood the language of New Spain and Mexico.”

Read the rest of this entry »

The Angoff Method is a widely used standard-setting approach in test development.

In plain English, it is a kind of study that test developers use to determine the passing percentage (cutscore) for a test. The passing grade of a test can’t be decided arbitrarily; it must be justified with empirical data. The Angoff method relies on subject-matter experts (SMEs) who examine the content of each test question (item) and then predict how many minimally-qualified candidates would answer the item correctly. The average of the judges’ predictions for a test question becomes its predicted difficulty. The sum of the predicted difficulty values for each item averaged across the judges and items on a test is the recommended Angoff cut score. Here is a real world example that illustrates the process:

Read the rest of this entry »

Court interpreters have a difficult and complex job.

They are not only expected to know two languages completely, they must also know and use various forms of those languages. Court interpreters regularly work with a range of people, including judges, attorneys, witnesses, litigants, and experts in specific subject fields, such as medicine or forensics. To interpret for such a broad scope of courtroom participants, an interpreter must have a considerably large vocabulary that includes legal language, subject-specific language, formal standard English, and colloquial expressions in both the source and target languages. To top it off, there are actually three different kinds, or ‘modes’ of interpreting that each require slightly different skill sets. In the following paragraphs, I will describe each of the three modes.

Read the rest of this entry »