The Effect of Menus on EFL Learners ' Look-up Processes

This paper emphasizes the importance of empirical research on dictionary users and, in particular, investigates the effect of the so-called "menu" (a list of definitions at the beginning of a polysemous article) on the EFL (English as a Foreign Language) learners' dictionary look-up processes. The menu has been increasingly popular in English learners' dictionaries in Japan, but no empirical evidence has ever shown that it is really effective for reference acts. Two groups of subjects with different levels of reference skills were observed looking up given information in two different types of mini-dictionaries, one with and the other without the menu. The results showed that the menu was not so effective for skilled users, but that it helped the less skilled users find the appropriate information.


INTRODUCTION
The development of second-language lexicography during the last two decades has been remarkable not only in Japan but world-wide.English pedagogical dictionaries in Japan used to be compiled on the basis of British and American lexicographical traditions.However, since we realized the importance of infonnation unique to our language-learning environments, our goal has shifted from mere translation of European dictionaries to more culture-specific and leamer-centred dictionary-making.
The publication of new dictionaries has encouraged more innovative ideas in tenns of infonnation categories and dictionary layout.Today, without something new, the product will not command attention.Dictionaries have become increasingly sophisticated tools, designed for skilled users, while_ no remedial work has been done for users with poor skills.Some lexicographers have therefore focussed our attention on the growing gap between the sophistication of dictionaries and the users' rudimentary reference skills.See, for example, Cowie (1983: 136).
While there are a great many EFL dictionaries in Japan, almost no research has been done in connection with dictionary users and their skills.The same can be said of lexicography in other countries.Until a decade ago, lexicographers compiled dictionaries according to their expectations about what users want to find in a dictionary and not according to what they really need.
Dictionaries in many countries still cling to the authoritarian tradition and their value and usefulness have hardly been questioned.The dictionary user has often been ignored and very little effort has been made to find out what the dictionary user does and why he behaves the way he does.It should be kept in mind that the purpose of lexicography is not only to describe words systematically, but also to produce a lexicographical output which should have practical utility.Therefore, we should seriously ask ourselves the following questions: Does the dictionary really meet users' needs and do the users actually utilise the information which the lexicographers thought was necessary?Empirical data on users' reference needs and skills is definitely necessary if we want to develop truly user-friendly dictionaries.At the same time, what the lexicographer believes to be innovative should be empirically tested.As in other areas of science, basic and applied research should go hand in hand.Such fundamental research and its application will contribute to a better lexicographical output and overall theory-construction.
In this paper, one of the recently highlighted dictionary-layout devices called a "menu" will be discussed as an example of dictionary~user research and its effect will be empirically tested.In Section 2 previous research will briefly be reviewed; in Section 3 the design of the present research will be specified; and in Section 4 the research findings will be discussed.

2.1
Research methods in dictionary-user study Fundamental questions in dictionary-user study are: How do dictionary users conduct their look-up processes and why do they behave in a specific way?
The answer to the first question implies a descriptive statement and the answer to the second implies a causal statement.The researcher, for instance, wants to ascribe dictionary users' poor reference skills to a lack of look-up training, the poor quality of dictionaries, etc.
In order to investigate these various aspects concerning dictionary users, we need to know the relevant research tools and their application.Unfortunately, not many lexicographers are versed in this kind of research methodology.This is quite natural, for it is not lexicographers themselves but psychologically-oriented metalexicographers who can best deal with this aspect of dictionary study.
In scientific study, a phenomenon is described, explained, and predicted; and each of these actions involves the following terms: observation, correlation, and experimentation.If we review the dictionary-user research done so far according to these three methods, it will show how much we are in need of experiments.

2.2
Descriptive studies

Historical research
The history of lexicography has been a major area of interest for decades, but it was not until recently that the importance of the user perspective was realised with regard to historical research.In 1987, Hartmann (l987a: 122) stressed that "a history of dictionary use is ... urgently needed".He suggested that the history of lexicography should incorporate more knowledge about the benefits dictionaries brought to their users.

Surveys
Probably the most widely used research method with regard to the dictionary user is the survey.Generally, surveys deal with the incidence, distribution and relationships of educational, psychological, and sociological variables (Wiersma 1991: 16).All variables are studied ex post facto, that is, as they exist in the situation.No experimental variables are manipulated.Furthermore, most of the surveys on dictionary-users' study are limited to describing the status quo.Reproduced by Sabinet Gateway under licence granted by the Publisher (dated 2011.)http://lexikos.journals.ac.zaBarnhart (1962) probably made the first attempt at quantifying our knowledge about the purposes and roles of the dictionary.108 questionnaires were sent out to teachers of English composition dasses, asking them to rate six types of information commonly offered in American college dictionaries according to the importance attached to them by freshmen students.The findings showed that, in order of priority, they regarded meaning, followed by spelling, pronunciation, synonyms, usage and etymology as the most important information types.Barnhart's paper was of considerable interest to lexicographers having to decide what kind of material should be included in a dictionary.
In spite of many interesting and useful comments, Barnhart's paper was not scientifically well motivated.Furthermore, his survey could not be replicated because he did not reproduce his questionnaire and numerical results.This made a follow-up study basically impossible.Quirk (1973) attempted to assess the attitudes, expectations, and prejudices of dictionary users.'The focus was no longer on the producer of the dictionary and its potential appeal to an anonymous market, " Hartmann (1987a: 127) commented, "but on the opinions of real users."Quirk did specify his survey method and numerical data, which rendered greater validity to his research.See also Crystal (1986).Tomaszczyk (1979) investigated the dictionary requirements of the foreign-language learner and translator.449 people completed his questionnaire.He found that the amount of dictionary use depends on the nature and extent of the activity.Again, however, the questionnaire was not reproduced and the problem of replicability arises again.
Bejoint (1981) also used the questionnaire to examine the language needs and reference skills of 112 French students of English at the University of Lyon.He prepared 21 questions for his questionnaire, but did not specify the purpose of each question, so that the results did not dearly show how the reference skills had been investigated.Hartmann (1983a) did research on learners of German in South West England.He found that bilingual dictionaries were so commonly used within and outside formal language classes and that monolingual dictionaries are so under-developed in languages other than English, that it would be very hard to 'wean away' the learner from the translation dictionary.He also found that grammar and meaning were the most important requirements for translation.This partly confirms the findings of Tomaszczyk (1979) andBejoint (1981).
Admitting the value of these surveys, we have to guard against the pitfalls in this type of research.Since they all comprise indirect surveying, the evidence may not reflect the subjects' real behaviour.It often happens that participants, instead of responding naturally, respond in ways they think the research demands, which leads to inaccurate interpretation of the data."More and more the suspicion is gaining ground," Hartmann (l987b: 15) mentions, "that indirect surveying of population samples needs to be supplemented or replaced by more carefully controlled direct observation." The most comprehensive and systematic survey to date is the research project on the use of leamer's dictionaries conducted by Beryl T. Atkins et al.The first preliminary report was given in Cowie (1987).It was followed by a detailed study in which the use of dictionaries by students of English as a foreign language, namely native speakers of French, Gennan, Italian and Spanish, was investigated.Over 1100 responses from seven countries were received.The survey consisted of the Dictionary-user Profile Fonn, the Placement Test, and the Dictionary Research Test.The results are now available in the EURALEX database.It is an excellent piece of research which provides us with many insights and interesting research questions.More of this type of descriptive study is definitely needed to improve the quality of correlational and experimental approaches.

Direct observation
In order to capture authentic data the questionnaire should be supplemented with exercises, interviews and tests.For example, Ard (1982) combined filmed protocols with oral interviews to observe the effect of bilingual dictionaries on ESL writing tasks.The design itself was interesting though the sample (just two) was too small to make any generalizations.Experiments with small examples need to have either many different levels of independent variables or the related variables must be very strictly manipulated and controlled (Elmes, et a1. 1981: 125).Krings (1986) used so-called thinking-aloud protocols to record the psycholinguistic complexities of the translation process.This approach is very widely used in psycholOgical research.Tono (1991) also observed the look-up behaviour of the good dictionary user.Recordings were made of the time taken for specific reference acts.The data on the degree of difference between good and poor dictionary users in a particular task was described in detail.This kind of approach, that is, a large number of observations made upon a small number of subjects, will also be promising in investigating the nature of the user's reference skills.

Correlational studies
Very few correlational studies have so far been conducted on the dictionary user.Quirk (1973) and Hartmann (1983a) attempted statistical correlations of their data, but their research interests and designs do not fit this category.In Israel, Marsha Bensoussan et a1. (1984) investigated the relationship between reading comprehension and the use of dictionaries.The results of several empirical tests with some 700 Israeli EFL university students led to the conclusion that the use of the dictionary has no significant effect on reading comprehension test scores.Furthermore, it makes no difference whether a monolingual or bilingual dictionary is used.Nor does the use of a dictionary affect the time needed to complete the test.Although the sample was qUite large, there are doubts as to whether all the confounding variables were properly controlled.For example, the level of the task and its mode of assessment may have been too artificial or inappropriate when we consider the proficiency level of the subjects -advanced learners in this case.
While Bensoussan et a!. (1984) investigated the static relation between dictionary use and reading comprehension, Tono (1988) was interested in the dynamic, developmental nature of dictionary skills and its relation to reading ability.Although the results were still inconclusive, he discovered that a significant difference in performance exists betWeen reading comprehension with and without dictionaries, and that continuous dictionary use has a positive effect on the development of reading ability.

2.4
Experimental studies An experiment is conducted when the environment is systematically manipulated in order to observe the effect of this manipulation on some kind of behaviour.Experiments are very cogent because, while non-experimental research techniques are limited to statements about correlation, experiments permit statements about causation -that is, independent variable A causes dependent variable B to change.
In dictionary-user study, it is essential not only to describe the behaviour of dictionary users, but also to explain the cause of their behaviour.In order to accomplish this, experiments are necessary.Hartmann (1987aHartmann ( , 1987bHartmann ( , 1988Hartmann ( , 1989) ) has been putting special emphasis on the need for experimental research, but still very few attempts have been made.Tono (1984) may be among the first who conducted controlled experiments on dictionary users.In order to investigate the parameters of dictionary look-up, 402 randomly selected students were tested on a translation task.The English pasSages contained a number of artificial words which were illustrated and explained in mini-dictionaries, carefully controlled for definition styles and the ordering of examples.Since the present study depends on these findings, this research will be discussed in more detail.
Suppose the subject comes across sentence (1) which contains an artificial word beduck and looks up the word in the mini dictionary which provides the information in ( 2): (1) I beducked his family of his death.
(2) beduck(vt.) The subject will have difficulty in deciding which meaning is appropriate for its translation equivalent.In Japanese, both meanings in (2) make perfect sense in this sentence.Therefore, the subject may choose either of them unless some other clues are available.Compare it with the following: In this case, the •collocational information «of ... », if the subject really uses it, will help him pick out the first rather than the second definition.
Tono invented two kinds of mini-dictionaries, one with information as in (3) and the other without such information.These different types of dictionaries were given to the subjects in their translation task and the subjects' choices of the definitions were recorded so as to see how the subjects used the information provided in the mini-dictionaries.The following seven types of information were selected to see if the subjects used them properly: grammatical information verb pattern countable vs. uncountable gloss collocation idiom run-on Besides the information categories in (4), Tono investigated the effect of illustrative examples and the definition order.
Statistical analysis of the data confirmed the hypothesis that users (at least of bilingual English-Japanese dictionaries) tend to choose the translation equivalent that appears as the first sub-entry and ignore definitions and examples in subsequent sub-entries unless there is an obvious negative clue which makes the initial sense unlikely.Figure 1 illustrates this process.
The results show that the skilled users employ semantic and grammatical information equally well, but that the poor users rely heavily on semantic information.If the first definition fits in the context, it will be chosen.Other information such as collocation, countable/uncountable, verb pattern, etc. is hardly used in a positive way to select appropriate definitions.Tono (1984) also found that if the entry had a long list of examples after the first definiti~n, the subjects were discouraged to go over to the second definition.This is why Tono suggested that the menu system, a list of senses without examples and detailed information, should be put at the beginning of each multiple entry.As a matter of fact, after this research had been conducted, many English-Japanese dictionaries adopted this menu system as an innovative device.
Figure 2 shows samples of the menu systems in some popular English-Japanese dictionaries.Nowadays, more than one third of all the English-Japanese pedagogical dictionaries contain the menu.In spite of its popularity, very little empirical investigation has been done on menu effect.

Summary
Thus far an overview has been given of the development of research methods in the field of user study and the need for more empirical data has been emphasized, especially that based on experimental designs.Tono (1984) shed light on the possibility of empirical research supporting more realistic and data-based, user-friendly dictionary making.Let us now consider scientifically exploring the effect of dictionary layout, particularly the menu, on the user's reference act. 3.

Hypothesis
In the last section, we observed the limited reference skiUs of dictionary users and the potential of the menu for helping them.Many recent English-Japanese leamer's dictionaries have adopted the menu for basic words, but its effect has never been tested.We would like to see if the menu really works, in other words, helps the users find the appropriate definition in spite of their limited experience of dictionary use.
For the sake of the experiment, the following null hypotheses were tested: (1) (a) The menu group and the non-menu group do not differ with regard to the time and effort spent on obtaining the appropriate information (in this case, for recognition purposes).(b) The menu effect and the level of reference skills are independent.
The corresponding alternative hypotheses are: (2) (a) The menu group is able to find the appropriate information more easily than the non-menu group.(b) The menu is effective only for those who have limited reference skills.
In other words, the independent variables for this study were the presence or absence of the menu, and the user's levels of reference skills.The dependent variables were the ease with which the users found the appropriate information in the dictionary.The details of the experiment are described in the following sections.

Subjects
57 first-year students, majoring in law at Keio University, and 182 third-year students from Setagaya Junior High School attached to Tokyo Gakugei University participated in this study.These two groups were choseri because they represented different levels of dictionary skills.The Keio University students had had a great deal of experience in dictionary use, for they had to prepare for the college entrance examination.On the other hand, the Setagaya Junior High School students are taught according to the communicative approach in which dictionaries are not extensively used.The difference in the levels of English proficiency, however, was not so serious a problem, because the English sentences to be translated into Japanese in this.study were not too difficult for junior high school students.

Instruments
A special test was developed to facilitate this study.The test consisted of nine English sentences, each of which contained one artificial word.The subjects were asked to translate the sentences into Japanese, using the attached minidictionaries.Two types of dictionaries were used: one with and the other without the menu.Artificial words were inserted into each sentence to ensure that all the subjects look up the same words and to control the information regarding the artificial words.In the case of (3) the artificial word stup was inserted: (3) If you say something like that, I'm sure he will be stup about it.
Suppose we invented two different definitions for the word stup such as the following: (4) (a) (b) sorry angry We cannot tell which definition is correct in (3) unless some other contextual clues are provided.
The word stup, however, would mean "angry" if the following collocational information were given in the dictionaries: ( Our goal is to determine whether this reference process is facilitated more effectively when the menu is provided at the beginning of each entry.If the menu does facilitate the look-up process, the subjects who use the menu-containing dictionaries should be able to choose the appropriate definitions with greater ease than those who use the dictionaries without the menu.As mentioned in Section 2, Tono (1984) found that many dictionary users had difficulty in proceeding from the first definition (polysemic meaning) to the second.Tono (1984) predicted that a menu system would assist the users, and some other scholars pointed out that this is an interesting possibility (Hartmann 1988(Hartmann , 1989;;Ripfel 1988).Present research aims at verifying this.
For further details of the test and the mini-<iictionaries, see Appendices A andB.

Design
Ideally, the selection of the experimental group and the control group should be based on the results of a pretest on dictionary-using skills.However, as it is very time-consuming to form a reliable and valid dictionary-skills battery, we chose economy rather than control.Moreover, by selecting homogeneous groups (i.e.college students and junior high school students), we believe that the two different levels of the independent variable (i.e. the dictionary-using skills) were fixed.As will be seen later, about ten percent of the junior high school students did not succeed in completing the tests in time, which shows that there was a slight proficiency effect on the test, although the overall picture of the results was not influenced by this variable.Therefore, while we are confident that our results are quite reliable and valid, the reader is advised to regard this study as quasi-experimental.A more controlled experiment might yield slightly different results.

Procedure
The test was conducted at the beginning of the class as part of the regular class activities.The teacher provided one half of the class with the dictionaries containing the menu and the other half with the dictionaries without the menu.The teacher did not mention the fact that there were artificial words in the test and that two different dictionaries were distributed.Instead, the teacher asked the class to translate the sentences into Japanese and told them that there were some difficult words and that the meanings were given in the mini-dictionaries.The time allowed for the test was fifteen minutes.

Data analysis
Each test was checked to see which definitions were chosen.The accuracy of the translation was ignored, and only the choices of the definitions for the artificial words were considered.The results, i.e. the definitions chosen, were recorded for both the menu group and the non-menu group.Since this variable was a nominal scale, a chi-square test was used to show the differences in the choice of the definitions.

RESUL TS AND DISCUSSION
The test results are shown in Tables 2 and 3.In the case of the junior high school students (see Table 2), there was a significant difference between the menu group and the non-menu group in the choice of the appropriate definitions.In comparison with the non-menu group, the menu group had chosen the meaning correctly in seven out of nine sentences.This means that, with the help of a menu in their dictionaries, the junior high school students found the necessary information more effectively.On the other hand, there was no significant difference between the two groups of college students (see Table 3).This indicates that for the college students the menu was not so useful in helping them find the appropriate information.
Let us take a closer look at each item.In the translation of the sentences containing the artificial words dandle and calluge there was no significant difference between the junior high school students and the college students provided with the menu and those without the menu.In the case of dandle and beduck most of the college students and fifty percent of the junior high school students chose the appropriate definitions.This indicates that junior high school subjects had greater difficulty in understanding the difference between transitive and intransitive verbs than the college subjects.
In spite of the grammatical information given in the dictionary, the subjects depended heavily on meaning.For example, in the case of foltage, most subjects chose "good result" instead of "influence".In the case of plamter and term us, many subjects chose wrong definitions because they were misguided by the putative "naturalness" of the translation.Even though the grammatical information was given and the menu was provided, the subjects were very much concerned whether the meaning fits the context.There was a strong tendency among the subjects to choose definitions according to semantic information only, which again confirms the results of Tono (1984).

Y. Tono
The difference in the look-up processes of the junior high school students and the college students was evident.Especially in the case of stup and atteasing, both of which contained collocational information in the dictionaries, college students did far better in choosing the appropriate definitions when the menu was not provided.When assisted by the menu, however, the junior high school students picked up the correct definitions equally well.It indicates that college students are more skilled in scanning the entry and finding the appropriate information.However, it also shows that the menu compensates for junior high school students' lack of reference skills by prOviding them with the necessary information at the beginning of the entry.
Consequently, both null hypotheses were rejected and the following research hypotheses were supported: (1) users who have the menu in their dictionaries are able to find the given information more easily than those who have no menu.( 2) (1) is true only for the users with limited reference skills.Skilled users utilize the menu less.'

CONCLUSIONS
The menu proves to be an effective aid for poor dictionary users.Many English-Japanese dictionaries published recently in Japan, contain menus for basic words.Their primary target is junior and senior high school students.
Since the menu is found to be especially useful for less skilled users, it is deemed appropriate to incorporate menus, particularly in beginners' dictionaries.
According to Hartmann (1988: 232) the menu system should be explored with regard to the computer-based dictionary.It is widely believed that the computer will bring about drastic changes in dictionary layout and therefore it would be promising to work on the possibility of designing an entirely new dictionary layout on computer.Meanwhile, however, the menu system should not remain a dream (Harbnann 1988(Harbnann : 2270.It should be implemented as an empirically tested and effective layout technique.
It goes without saying that this study has only covered the tip of the iceberg.Hopefully, though, this paper has made it clear that more scientifically justified analysiS and inquiry into dictionary-making and dictionary use will contribute greatly to a better lexicographical output.

Table 1 .
Table1summarizes the major survey results: Selected dictionary-user surveys