So, what does it mean for a test to be valid?

In psychometric terms, a test’s validity is the degree to which the theory behind the test and the interpretation of the test’s score accurately measure the test’s intended purpose. In other words, a valid language test works to assess language ability, and the scores can be defended.

ALTA spends a lot of time and resources to ensure that our language tests are valid. While the process is complex, here is a basic synopsis of 9 important steps we take, whereby each step contributes to the overall validity of a language assessment. Figure 1 illustrates this validation cycle, and each step is described below:

Figure 1: Validation Cycle

1. Job Analysis/KSAs:

The first step in test development is to identify the knowledge, skills, and abilities (KSAs) that the test will be designed to measure. For tests that are designed to qualify an individual to perform a specific job, these KSAs are identified through the performance of a job study in which individuals knowledgeable of what the job entails – or, subject-matter experts (SMEs) – are interviewed to collect this information. The identification of KSAs is a crucial step in providing focus to the development efforts that follow.

2. Create Test Blueprint:

The test blueprint is created based on the KSAs identified, and their relative importance to the job. The blueprint specifies to the test developer the content that will be included in the test, the amount of content in each skill area, and any other instructions needed to properly develop the content. Using the blueprint as a guide, test developers are engaged to create the actual test items.

3. Create Test Items:

Item development is carried out according to the specifications outlined in the test blueprint. More than the ample amount of test items are created to allow for the possibility that some of the items will need to be eliminated based on pilot-testing and item analysis results.

4. Review Test Items:

All test items are submitted to a separate panel for review and comment. This panel reviews each test item and verifies that each aligns with the specifications as outlined in the test blueprint. Any need for modification is recorded, and comments are provided to the developers so that the appropriate changes can be made. This review process is repeated for any changes that are made until the pilot version of the test is complete.

5. Pilot Test Items:

Once the final draft version has been reviewed and approved by test developers and the review panel, the items are pilot-tested to gather data around item performance. Pilot testing is done using a sample of candidates representative of the target population. Following the pilot-testing, psychometric analysis is performed on the results to determine the test’s performance.

6. Create Final Test Form:

Results from the statistical analysis yield the items that will constitute the final test form, and these items assembled into the operational version of the test.

7. Angoff Method:

Using the final test versions, an Angoff panel is assembled to determine the cut-score of the test, or the percentage of correctly-answered items that the candidate needs to successfully pass the test. Although various standard-setting methods exist, ALTA typically uses the Angoff method, which relies on the judgments of the panel as to the percentage of minimally-qualified candidates who would perform successfully on each item.

8. Administer Test:

Upon determining the cut-scores for the final test versions, the tests are available for operational use and are administered according to the operational policies set up by the test administrator using a prescribed scoring rubric.

9. Quality Assurance:

Quality assurance is performed continuously to ensure that the items are performing properly over time. Quality assurance also provides a method for monitoring overexposure and identifying items which may have been compromised.

It is important to note that validation is a cycle, and testing organization should continue reviewing the test and collecting evidence of the test’s validity. At various points in the lifetime of a test, each step may be revisited for review and/or revision.

____________________________________________________________________________________________

ALTA is a leader in language testing and large-scale language solutions for government agencies and corporations nationwide. In addition to being the official language testing provider for the cities of Los Angeles and New York, ALTA works with many of the country’s largest corporate organizations, from DELTA Airlines to Wells Fargo. Learn more about us at altalang.com

                          

In May of 2009, the Judicial Council of California contracted ALTA to assess the Consortium for Language Access to the Courts’ (CLAC or Consortium) test development process. The aim of the study was to perform a comparative analysis between the examinations developed by CLAC to certify its member state court interpreters and the examinations developed by California to certify California state court interpreters.

The study includes literature and document reviews and interviews with members of the Consortium’s staff both past and present, including language consultants and exam writers. Additionally, ALTA and a team of psychometricians performed a psychometric audit of the Consortium’s test development process to identify its key strengths and any areas that were in need of improvement to meet test industry standards according to the Standards for Educational and Psychological Testing (AERA, APA, & NCME, 1999.)

Once the psychometric audit was complete, a group of subject-matter experts examined both the written and oral Consortium exams to identify the knowledge, skills, and abilities required for successful court interpreting , as determined in the Study of California’s Court Interpreter Certification and Registration Testing. Results from the comparative analysis showed a high degree of overlap between the California and CLAC written and oral exam KSAs, as well as a strong degree of comparability between the structure, content, and level of difficulty found in the oral exams for each program.

The final step of the study involved performing a standard-setting session applying the Angoff Method to establish the level of functional equivalency between the passing requirements of each program’s oral examinations.
____________________________________________________________________________________________

ALTA is a leader in language testing and large-scale language solutions for government agencies and corporations nationwide. In addition to being the official language testing provider for the cities of Los Angeles and New York, ALTA has worked with the Judicial Council of California on language testing studies for several years. Learn more about us at altalang.com

                          

The resource is the result of the collaborative efforts of terminologists, bilingual lawyers, legal translators, and linguists, as well as other qualified and experienced members of the legal and multicultural communities of both Canada and the country of origin of each language included.

A quick search of one commonly used legal term, “defendant,” yielded excellent results in Spanish, providing a thorough definition for the term in both English and Spanish. This is an excellent resource for anyone working within the legal arena, especially legal translators in Canada. While the dictionary aims to “explain Canadian legal terminology and to provide foreign language equivalents for concepts related to Canadian law,” many of the included concepts and terms are universal across the English language.

Add this site to your working list of references and support Vancouver Community College’s efforts! To help spread the word, ALTA is going to add a link on the Free Resources for Translators page. I hope to see more online reference resources of this nature in the coming years.

                          

Hurray! Yipee! Wowee! Let’s hear it for my favorite part of speech: the interjection! Interjections help to express emotion on the part of the speaker in written language. They are usually separated off from the rest of a sentence by commas or exclamations points. In any language, interjections make the written word more expressive.

Here is a list of my personal favorites in a few different languages:

English
Wowzers!
Arrgh!
Eeek!
A’ight

Spanish
¡Ojo! (Watch out!)
Ja ja ja (Ha-ha-ha)
¡Ay! (Oh!)
¡Epa! (Hey!)

French
Zut! (Damn!)
Ouf! (Phew!)
Voilà! (Ta-da!)

German
Ja (yeah)
Igitt! (yuck!)
Uf! (whew)

                          

I recently read a book titled The Origins of Simultaneous Interpretation: The Nuremburg Trial that I thought I would recommend given that the pool of information regarding the history of interpreting can be somewhat limited, or difficult to find.

Written by Fracesca Gaiba, Origins describes how the Nuremburg Trial, conducted from November 1945 to August 1946, was the first international gathering in which simultaneous interpretation was used. Gaiba recreates the events that brought simultaneous interpreting to the court system through a combination of interviews with Nuremburg interpreters and staff, as well as extensive literature and audio review of resources pertaining to the trial. Four languages, English, French, Russian, and German were used throughout the trial. Every listener in the courtroom was equipped with a headset and a selector switch at his or her seat. The listener could then choose one of five “channels” he or she wanted to listen to during the trial: the verbatim speech, English, French, Russian, or German.

In Gaiba’s words:

When a French prosecutor was examining a German witness, for example, the prosecutor asked a question in French. The witness heard the German translation through the earphones and answered in German into the microphone. The prosecutor heard the answer translated into French and replied in French, and so on. At the same time an English-speaking listener heard the English version of both questions and answers (p. 61).

Aside from the technological equipment created to facilitate the multilingual simultaneous interpretations, one of the most interesting aspects of the book is Gaiba’s description of the selection process used to hire simultaneous interpreters for the trial:

Finally, the division discovered that many linguists with excellent academic backgrounds could not do the job because, even though they could translate Schopenhauer, they could not cope with subjects like toilet arrangements in concentration camps.



Basically, the division discovered that few, very few, people could do simultaneous interpreting. Alfred Steer who in a year tested more than 400 people, calculated that only five percent of the people tested, including experienced consecutive interpreters, could do simultaneous interpreting. More than 200 people were tested before the trial to obtain the first 36 interpreters. In all, more than 500 sat for the interpreting test during the trial. Soon after the testing started it was realized what a difficult task was required of a simultaneous interpreter (p. 48).

And you thought it was hard to pass the Federal Court Interpreter’s Certification Examination!

All kidding aside, this book is a great read that demonstrates the level of technique and skill needed to accurately perform simultaneous interpreting. Origins provides us with a perspective on the Nuremburg Trial which will be fascinating to any interpreter or language professional.

                          

The law “requires health, dental and specialty insurers to provide subscribers with translators (sic), at least by telephone, while visiting their doctor, pharmacist, ophthalmologist or dentist.”

Senate Bill 853 was officially signed in 2003, but insurance companies had until January 1, 2009 to comply with the bill.

Kudos to California for being at the forefront of providing language services to LEP residents. Let’s hope that other states follow suit.

                          

La Malinche, Doña Marina, La Chingada. No matter what name you use, there is no doubt that she is one of the most influential interpreters in history.

Doña Marina, who came to be known in Mexico as La Malinche, was born the daughter of a cacique during the rule of the Aztecs in the early 1500s. As the daughter of a cacique, she was considered part of the noble class and allowed the opportunity to attend school. Not much else is known about her early years except that her father died and she was sold into slavery (it is not known whether she was sold by her family or kidnapped and then sold), eventually ending up in the hands of Hernan Cortes and the Spaniards during their conquest of the Aztecs.

It is also not known how Cortes became aware of Malinche’s linguistic skills, but within a few weeks of being sold to him she had begun interpreting. In one of the first documented incidences of relay interpreting, La Malinche and a Spanish priest named Gerónimo Aguilar worked as a team to interpret for Cortes by transferring Nahuatl (the Aztec language) to the Chontol Mayan language to Spanish.

They continued this practice until La Malinche was able to learn Spanish and interpret directly from Nahuatl to Spanish for Cortes. Bernal Diaz del Castillo, an eyewitness to Malinche’s skill as an interpreter, stated that “Without the help of Doña Marina, we would not have understood the language of New Spain and Mexico.”

One could argue that without Doña Marina serving as his interpreter and enabling him to communicate with the Indians, Cortes may not have been able to defeat the Aztecs, or at the very least, not as readily.

La Malinche was the mother of Cortes’ first son, who is considered to be one of the first Mestizos (a person of mixed European and American indigenous ancestry). Several accounts indicate that La Malinche was also responsible for foiling more than one Aztec plan to attack Cortes and the Spanish army. Her various roles as interpreter, Cortes’ mistress, and informant, led to the less desirable labeling of her as “La Chingada” by modern Mexicans.

La Chingada can be translated to “the fucked one.” Mexicans sometimes refer to themeselves as “Hijos de La Chingada,” or quite literally, as the bastard children of the sexual union between Cortes and La Malinche. One can see the complicated aspects of La Malinche’s contribution to Mexican identity, which was most famously explored by Nobel laureate, Octavio Paz.

Understandably, many Mexicans regard La Malinche as a traitor. Her role as an interpreter has often been sullied by this perception. She has become the embodiment of the famous saying “Traduttore, traidore.”

Whether you see her as a victim of circumstance, a brilliant survivor, or as a traitor, there is no disputing that her language interpreting skills helped to seal her role in history forever, as well as the fate of the Aztec Empire. Her story, in part, serves to illustrate the importance that interpreters can play in shaping history, as well as the price that they sometimes pay when they do so.
_______________________________

Antonio Ruíz’s painting El sueño de la Malinche [“The Dream of Malinche”]: Oil on canvas, 11 7/8 x 15 3/4”. Galería de Arte Mexicano, Mexico City, 1939 courtesy of George Mason University’s Women in World History.

                          

The Angoff Method is a widely used standard-setting approach in test development.

In plain English, it is a kind of study that test developers use to determine the passing percentage (cutscore) for a test. The passing grade of a test can’t be decided arbitrarily; it must be justified with empirical data. The Angoff method relies on subject-matter experts (SMEs) who examine the content of each test question (item) and then predict how many minimally-qualified candidates would answer the item correctly. The average of the judges’ predictions for a test question becomes its predicted difficulty. The sum of the predicted difficulty values for each item averaged across the judges and items on a test is the recommended Angoff cut score. Here is a real world example that illustrates the process:

Let’s say a test developer needed to determine the passing grade for a language exam that tested a person’s ability to read Arabic. Using the Angoff Method, the developer would employ a number of SMEs (in this case, Arabic-language experts) and ensure that they were properly trained on how to use the Angoff Method, as well as informed on the test’s purpose.

The Arabic-Angoff Panel would then rate each test item based on whether or not a minimally-qualified candidate would answer the item correctly or incorrectly. Once the first round of ratings had been conducted, everyone on the panel would be given access to the ratings of the other SMEs so that they could compare what they determined about a particular item. Then, the SMEs would be asked to rate the items again for a second round. The second round of rating would give the SMEs the opportunity to review their initial rating of an item and decide whether or not they might like to change their decision based on the expert judgments of the other panelists. This second round of ratings would be averaged across the SMEs to determine the final cutscore for the test.

In the end, using the Angoff Method ensures that the passing grade of a test is determined empirically, which is necessary for a test to be legally defensible and meet the Standards for Educational and Psychological Testing. Other standard-setting methods that testing professionals sometimes employ include the Ebel, Nedelsky, and Bookmark methods.

___________________________________________________________________________________________
ALTA is a leader in Language Testing for government agencies and corporate clients. If you’d like more information about language testing for your organization, please call 404.920.3817.

                          

Court interpreters have a difficult and complex job.

They are not only expected to know two languages completely, they must also know and use various forms of those languages. Court interpreters regularly work with a range of people, including judges, attorneys, witnesses, litigants, and experts in specific subject fields, such as medicine or forensics. To interpret for such a broad scope of courtroom participants, an interpreter must have a considerably large vocabulary that includes legal language, subject-specific language, formal standard English, and colloquial expressions in both the source and target languages. To top it off, there are actually three different kinds, or ‘modes’ of interpreting that each require slightly different skill sets. In the following paragraphs, I will describe each of the three modes.

The three modes of interpretation are: simultaneous interpretation, consecutive interpretation, and sight translation. Although the frequency of use of these modes may vary slightly among courts, counties, and specific languages, court interpreters are widely expected to be able to successfully perform interpretations using all three modes.

Simultaneous Interpretation:

Simultaneous interpretation is the mode in which the interpreter lags slightly behind the source language speaker, interpreting the message into the target language at almost the same time as the original message is being said.

In judicial settings, simultaneous interpretation is most frequently practiced from English into other languages. It is used for most stages in proceedings that heavily involve speaking by or between attorneys and judges, such as arraignments, motions, and jury instructions. Because the majority of proceedings take place primarily in English with attorneys and judges speaking, many interpreters practice simultaneous interpretation more often than any other mode.

Consecutive Interpretation:

In consecutive interpretation the interpreter listens to a unit of speech in the source language and then conveys that message into the target language. Although the length of the passage rendered can vary, most literature points to a standard norm of approximately 40-60 words at a time. In the courts, consecutive interpretation is commonly used for interpreting messages from a litigant with limited English proficiency, which most often occurs in client/attorney interviews, or on the witness stand.

Many interpreters think that consecutive interpretation is the most challenging mode because of the necessary memory retention and note taking skills. In performing this mode, the interpreter must store heard information, recall the information, and then restructure the information in the appropriate linguistic form. The speed of the delivery of information, as well as the complexity of the subject matter, or length of the speech will likely alter depending on the speaker, requiring the interpreter not only to listen attentively to the delivered orations, but also to listen and process different levels and types of discourse. The possible range of content and potentially unpredictable nature of the discourse necessitates that an interpreter have an expansive vocabulary and complete competence in both languages.

Sight Translation:

Sight translation is the oral interpretation of a written document. In courtroom settings sight translation is often used to provide oral interpretations of standardized, routine written documents in English, such as waiver of rights forms, DUI forms, and police reports. In many routine procedures the sight translation of such documents is performed so that a defendant with limited English proficiency is able to understand. Occasionally, informal or formal documents in the foreign language are presented and the interpreter must interpret these into the English language for the court’s benefit.

Although interpreters may be given the opportunity to review a document prior to performing a sight translation, they usually must read quickly and anticipate the content of the written text without fully reading the entire text. Comparable to the skills needed for simultaneous interpretation, sight translation requires that the interpreter use predictive skills to help them process information and create context for an incomplete message. Interpreters are often reading ahead and processing upcoming information even as they are conveying a unit of the text.

As facilitators of communication, it is clear that the interpreter’s function in the courtroom is essential, and that the role of the interpreter carries great responsibility. Acting as the communication bridge between the court and a non-English speaking litigant, interpreters must convey not only the words of a message, but also the essence of that message, becoming the voice of all courtroom participants.

I look forward to my next post on the Ethics of Court Interpreting.

                          

Presently there are over 6,000 known living languages in the world.

Over the years, I have compiled a list of several great websites that I use as starting points for language research. Recently, as part of a language testing project, I had to locate people capable of administering Speaking and Listening tests in Marshallese.

What is Marshallese and who speaks it?

After an initial search on one of my favorite sites: Ethnologue, I learned that Marshallese is spoken by the 43,900 inhabitants of the Marshall Islands. I also discovered that Marshallese has different alphabetic systems depending on the religious affiliation of a speaker. This kind of knowledge becomes integral if a project involves Marshallese Writing assessments in addition to Speaking and Listening tests. For example, the type of writing assessment would depend on the religious affiliation of the target candidates for testing. If the candidates were all going to be from the same religious affiliation, I would have the writing assessment developed to target that particular religious affiliation’s alphabetic system. However, if the candidates were going to be from several religious affiliations, I would direct the developers to create a universal writing assessment that would take into account all of the nuances of the language, if possible. Finally, a more complicated and costly option would involve developing a different writing assessment for each religious affiliation that needed to be tested. As a project manager, it is my job to choose the option that will best meet my client’s expectations and ensure their satisfaction with the final product.

Another language I have worked with extensively is Tagalog. At first, I thought it was pronounced like my favorite Girl Scout cookie, but in fact, it is pronounced Tah-ga-log.

Tagalog is a language that originated in the Philippines and is spoken by a population of 15,900,098 people worldwide. It is also among the
top ten most commonly spoken languages in the United States. Many different areas of the language services industry require Tagalog speakers including: interpretation, translation, and proficiency testing. Check out I Love Languages for a great resource containing several Tagalog links, as well as information on hundreds of other languages.

I look forward to continue learning about the variety of languages across the world, and keeping you informed along the way with regular posts. Check out my next post for a detailed description of the different modes of interpreting.