Arabic has proven to be a difficult language to test.
In Western countries, the intricacies of the Arabic language are not common knowledge. The idea of a diglossia, in particular, has little context in English and other Germanic languages and Romance languages. What is meant by diglossia is the co-existence of two separate versions of a language, often representing a hierarchical delineation.
Typical daily life in Arabic-speaking countries is conducted in one of many colloquial Arabic dialects. Modern Standard Arabic (MSA) is the language of written discourse, university courses, and academic and scientific texts and conferences. Interestingly enough, it is also the language in which a majority of TV and radio news is delivered. It follows that a citizen of an Arabic-speaking country who does not have a formal education would not be able to understand the TV news or read a newspaper. Some villages may only have a handful of people with enough knowledge of MSA to impart the news of the world to their neighbors.
Ideally, students of Arabic would begin learning both Modern Standard Arabic and one of the most commonly spoken dialects, such as Egyptian Arabic, at the same time. Some university programs have caught on to this, but others lag behind, teaching only formal, written Arabic and ignoring the colloquial forms until the student has completed several semesters of an Arabic program.
The Interagency Language Roundtable (ILR) language-level scale, when applied to Arabic as a second language for native English speakers, presents more difficulties. Because of the comparatively small number of cognates between English and Arabic, and because the character system and units of pronunciation are so different from English, the jump from ILR Level 0 (no comprehension of the language) to ILR Level 0+ (comprehension of the alphabet, number system, and units of spoken language), and from ILR Level 0+ to ILR Level 1 (comprehension of elementary grammar and syntax), are much more involved than for, say, a native English speaker learning French.
The U.S. Armed Forces learned this the hard way. In the fall of 2007, the military had to pull the Modern Standard Arabic version of the DLPT 5 (Defense Language Proficiency Test) after an astoundingly low number of Arabic learners from their own language institute passed the test at the level that corresponded to their classroom studies. The Defense Language Institute had planned to introduce a revamped version of the test in January of this year; currently, I am finding no evidence on the DLI website that this did not happen as scheduled.
One could make a case, however, for the creation of a separate ILR scale for languages involving non-Roman characters, perhaps similar to the ACTFL proficiency guidelines for the less commonly taught languages.
Here at ALTA, we have been working with numerous clients who require Arabic language proficiency testing. The key to effectively working with the client, and with the language, is to make sure the client knows what they are up against in the Arabic language. Bringing the client to an awareness of the diglossia, the various dialects, and the learning curve for native English speakers is important, it ensures an understanding that being a beginner in Arabic is not the same thing as being a beginner in French, Spanish, or German.