Dictionaries Integrated into English Learning Apps: Critical Comments and Suggestions for Improvement

: Digital applications to assist language learning are becoming increasingly popular. They typically incorporate one or two dictionaries to improve the service so that users avoid leaving the app to consult external resources. This paper deals with the two dictionaries used in a learning app for Chinese learners of English. Initially, it describes the functioning of the app as well as the two dictionaries that have different roles in the app. It then focuses on the one that is integrated into the course texts and can be activated by clicking on a word or a multiword unit. A number of deficiencies are discussed such as inconsistent treatment of words and senses, data overload, difficult access, and inconvenient location of the pop-up window that displays the lexi-cographical items. These deficiencies may impact negatively on the learners' motivation and the learning process in general. The paper traces the problems to the database that sustains the dictionary as well as the design of the user interfaces that filter the data offered to the users. Finally, and inspired by the classical Chinese Xun Gu tradition, it suggests an alternative, context-adapted approach that breaks with traditional features of the dictionary article and reduces the content of the pop-up window to an absolute minimum. The idea is to avoid a consultation process that inter-rupts the learners' reading flow and focus on learning.


1.
Introduction Lew (2007: 212) observed that lexicography "grows, branches out, and specializes".This holds true not only for the highly varied topics treated by the millennial discipline but also for the many new ways of presenting the final product to its end users.If it was the case in 2007, it is even more so today with the digital revolution developing almost exponentially.There are still countries, where some dictionaries are printed in relatively big editions.But the general tendency, both here and elsewhere, is the increasing publication of lexicographical products on a big variety of digital platforms.In China, the current transition to these platforms is referred to as the media convergence age, a concept that also includes videos, illustrations, interaction, and other ways of meeting user needs; see Zhang (2019a, b), Li and Wang (2020), and Kang (2020).
According to Fuertes-Olivera et al. (2018: 156-57), the overall transition can be further specified as a triple transformation from "the traditional stand-alone dictionary, either printed or online, to the integrated dictionary"; from "the standardized dictionary (…) to a more personalized dictionary"; and from "the dictionary as such to lexicographical data for different uses".The first of these three tendencies is obvious.The past few years have seen the integration of dictionaries into digital products like e-readers, writing assistants, translation tools, and learning apps; see Bothma and Prinsloo (2013), Tarp et al. (2017), and Ma (2019), among others.However, as we will show in this contribution, it seems to be much more difficult to design a lexicographical product that provides personalized assistance to its users.This has probably to do with deep-rooted traditions combined with a certain degree of resistance to sacrificing traditional dictionary features to the need for a more intuitive, digitalera design that focuses on the specific set of lexicographical data required by the users in each situation; that is, the third transformation referred to by Fuertes-Olivera et al. (2018).
In China, there is a millennial tradition called Xun Gu that may inspire us.When the old scribes copied manuscript works from earlier periods they frequently inserted glosses into the text to explain difficult or obsolete words and interpret the content; see Yong and Peng (2008: 21-25).The tradition continues even today when classical Chinese works are re-published.A similar tradition can be found in other parts of the world.McArthur (1986), Stathi (2006), and Hanks (2013) report that it was practiced in Europe in the Classical Greek Period, where the glosses subsequently were compiled into glossaries.This gave birth to the traditional dictionary form as we have known it ever since; that is, the dictionary as a structured collection of articles containing glosses (lemmata) and additional, explanatory items of various types.
However, the glossaries did not mark the beginning of lexicography as such, as it was already practiced embryonically by the scribes when they inserted glosses into manuscript copies.Tarp and Gouws (2019) characterize this practice as "lexicographical contextualization and personalization".In the classical Greek and Chinese traditions, the response to user needs is provided directly in the context where these needs occur, the consultation is intuitive, and the search time close to zero.In addition, the response consists exclusively of the lexicographical data required to meet the user's needs in a concrete situation, thus avoiding the adverse phenomenon of "information anxiety" (Wurman 1990).By contrast, data overload in lexicographical products may lead to a time-consuming consultation with the risk of retrieving the wrong information or no information at all, as discussed by Gouws and Tarp (2017).Hence, the Xun Gu and similar experiences seem to be a good starting point for a more personalized -and contextualized -lexicographical service in integrated digital products like the ones mentioned above.
Learning apps have already been discussed from different angles in the academic literature; see Zhang and Jin (2017), Rosell-Aguilar (2018), and Loewen et al. (2019), among others.In this contribution, we will focus on dictionaries integrated into second-language learning apps and approach them from the perspective outlined above.To our knowledge, this topic has not yet been treated comprehensively in the lexicographical literature.As a case study, we have chosen the Kaiyan OpenLanguage app that is used by Chinese learners of English as a second language.In the next section, we will argue why we have chosen this app among other possible candidates and explain how it works.In Section 3, we will describe the two dictionaries used in the Kaiyan OpenLanguage app.In the subsequent section, we will analyze how they perform and to what degree the design features satisfy the needs of Chinese learn-ers.Section 5 will summarize the effect on users and consultation processes, whereas Section 6 will discuss and suggest various ways of improving the app from a lexicographical perspective.Finally, we will conclude that most of these suggestions can be generalized and applied in other learning apps.

Description of English learning apps
With an overwhelming majority of Chinese people owning a smartphone or tablet, L2 learning apps, especially English ones, are becoming more accessible and popular in China.Among the learning apps can be mentioned Shanbei (扇贝), Baicizhan (百词斩), Yingyu Qupeiyin (英语趣配音), Yingyu Tingli (英语听力), and Kaiyan Openlanguage (开言英语).Some of these apps, like Shanbei and Baicizhan, allow most of their functionalities to be used freely.Others, like Kaiyan OpenLanguage, only offer a few free trials, after which their users have to buy a premium version to get access to most of the content and functionalities.Compared to traditional classroom teaching with time and location constraints, learning apps allow learners with smartphones or tablets, as well as Internet access, to attend online English classes at any time, at almost any location.They can stop when they are busy and continue learning when they are free.They can make plans for themselves and progress at their own pace.Learning apps can, therefore, be designed to cater for the users' individual needs.When learners use L2 learning apps, the integrated dictionaries are indispensable.Consultation of the meaning or grammar of unknown words helps them better understand the context and the whole content of the course they are following.In fact, these learning apps always incorporate one or two dictionaries.When studying the course material, learners frequently encounter difficulties related to words or word forms, which they either don't know, are uncertain about, or just want to confirm.In such cases, they can immediately resort to the integrated dictionary by simply touching or clicking on the word they want to consult.A box with a dictionary article (or an extract of such an article) will then appear directly above or below the activated word and allow the learners to retrieve the required information in terms of meaning or grammar to better understand the text.
Integrated dictionaries make up a new type of digital dictionaries.They are handier and more practical than the common dictionary apps because their users do not need to close the running learning app and start a separate dictionary app to look up words for more details.In this way, learners can easily and timely get access to the relevant lexicographical data.By reducing the time spent on consultation, they can focus more on the workflow and learning process without being interrupted by the constant switch between different apps.Thus, the whole learning process can be more efficient with the help of integrated dictionaries.
Kaiyan OpenLanguage is a very popular English learning app in China.It incorporates a podcast named Panji and Jenny tell you (潘吉Jenny告诉你), which is also commercialized separately and ranks among the top popular English learning podcasts for Chinese learners in the App Store.Kaiyan OpenLanguage offers audio courses, vocabulary courses, pronunciation courses, and reading courses, among others.It is under continuous development with several newly added interactive video courses.
The main Kaiyan OpenLanguage course consists of a large number of virtual classes divided into different levels from A1 to C2.Each class has four parts: vocabulary, lesson (texts in dialogue or video clips), exercise (on vocabulary, translation, reading, and listening), and oral practice (imitation).Vocabulary is a list of new words that are explained in the lesson.The lesson is composed of a text mainly in the form of dialogues.Two teachers (one native English and one Chinese English teacher for beginners and two natives for intermediate levels) will teach the lesson together by explaining new words, grammar, content, and cultural issues occurring in the text.Their way of teaching is very relaxing and interactive, like two friends chatting with each other.Some newly uploaded lessons are in the form of video clips.The exercise consists of multiple choices, match pairs, and fill in blanks to test if users have mastered the lesson, to practice their vocabulary, translation, reading, and listening.Oral practice is mostly given in form of a dialogue between two or more people from the lesson, where the learners have to play one of the roles.After the oral practice, learners will receive a score for their performance that indicates whether their pronunciation is good or should be improved.
The class does not last long.Each lesson is about 10 to 30 minutes, with exercises and practices taking another 20 minutes.It also has three complementary parts: grammar point, practical sentences, and culture, to enrich and deepen the understanding and learning of the lessons.It is a very comprehensive and practical English learning course for Chinese learners, as it is explained on its website: Our courses involve real-world scenarios such as traveling, workplace, food culture, including grammar, dialogue, vocabulary, and other modules to learn, making it easy for you to use the fragmented time to learn spoken English that you can use right away. 1   Unlike the Kaiyan OpenLanguage app, other learning apps such as Baicizhan, Yingyu Tingli, and Shanbei only focus on one aspect of the English learning process.For instance, Baicizhan (百词斩) and Shanbei Danci (扇贝单词) focus on vocabulary, whereas Baicizhan AiYuedu (百词斩爱阅读) and Shanbei Yuedu (扇贝阅读) emphasize reading, Yingyu Tingli listening, and Yingyu Qupeiyin oral practice.Although some companies offer various learning services, they commercialize these services in separate apps.Baicizhan company, for instance, markets vocabulary and reading in two separate learning apps: Baicizhan and Baicizhan Yuedu.This is also the main reason why we have chosen the Kaiyan OpenLanguage to conduct a case study on the design features and performance of integrated dictionaries.

3.
Presentation of the integrated dictionaries Two dictionaries are used in the Kaiyan OpenLanguage app: Concise Dictionary and Oxford English-Chinese Dictionary.No further information is provided about publishing houses, specific editions, or updates.The publisher of the second dictionary is undoubtedly Oxford University Press (maybe in association with a Chinese partner), and we suppose that the Concise dictionary is published by the same company as the learning app, although it is not confirmed anywhere.The two dictionaries have different status and can be accessed in different ways.During all learning activities, except for those related to the exercise part, learners can touch or click on a word to get lexicographical assistance.A black box will immediately pop up (see Figure 1).This pop-up window may contain different types of lexicographical data such as pronunciation, part of speech, variants, senses, definitions, equivalents, and even cultural data.Users can also look up words that do not appear in the canonical form; for instance, verbs in present participle or past tense (implementing, thought), and nouns in plural (bosses, trips).They will then be connected directly to the canonical word form with a short explanation of the relation between the canonical form and the one occurring in the text.When the users, for example, click on the word thought in the text, the lemma thought will appear in the pop-up window together with various senses structured according to part of speech.In the second line, thought is classified as a verb with a gloss indicating that it is the past tense of think.If learners are not satisfied with the default data presented in the pop-up window, they can click on the signifier 查看完整释义 at the bottom to access the whole dictionary article with more lexicographical items (see Figure 1).The default dictionary used in this type of consultation is the Concise Dictionary that contains both a bilingual English-Chinese part and a monolingual English one.However, if this dictionary does not contain the lemma in question, the user can have a second try in the Oxford English-Chinese Dictionary as will be explained below.In this respect, the latter seems to function as a safety net to guarantee a better user service.Apart from the described form of consultation, the Kaiyan OpenLanguage app also makes allowance for another way of looking up in the dictionaries.At the bottom of the front page, the app has a search box with the signifier 查词 (Consult a word), which affords direct access to the Concise Dictionary.A pop-up window will then be displayed with a traditional dictionary article that provides definitions in English and concise explanations in Chinese, as well as a list of lessons where the word appears.Users who prefer to consult the Oxford English-Chinese Dictionary can do so by clicking on the signifier 牛津 (Oxford) that provides access to this dictionary.If they are not satisfied, they can click on the signifier 简明 to return to the Concise dictionary.In this way, users can switch between the two dictionaries as illustrated in Figure 2. The difference between the two types of consultation described above calls for a terminological clarification.The dictionary accessed from the front page by entering a word in the search box functions fundamentally like any other digital dictionary placed in a separate dictionary app or online portal.The consultation of these dictionaries also requires that their users enter a word or word string in a search box.The only difference is that learners can access the dictionary directly from the learning app and get lexicographical assistance without leaving the app, thus skipping a time-consuming external consultation process every time they experience a lexicographically relevant information need.Dictionaries with these characteristics can be defined as embedded dictionaries.By contrast, dictionaries that are activated by clicking on a word in the course text can be defined as integrated dictionaries.They differ from the former in that only the words occurring in the text can be consulted.Although the two types of dictionaries used in the Kaiyan OpenLanguage app upload their data from the same lexicographical database, they are nonetheless different, not only in the way they work, but also in the perspectives of achieving a more personalized user service.Embedded dictionaries cannot "know" what information a user is looking for in a concrete consultation, whereas integrated dictionaries, if well designed, will be context-aware and, thus, "know" the concrete sense of a word relevant to the user.This context-awareness seems to be the http://lexikos.journals.ac.za; https://doi.org/10.5788/31-1-1626most urgent lexicographical challenge to the Kaiyan OpenLanguage and other similar learning apps.
As already discussed above, integrated dictionaries have some obvious advantages like personalization, efficiency, and time-saving.But the advantages largely depend on the quality of the lexicographical database and the design of the user interfaces; that is, the dictionary as presented to its target users.The two aspects, the database and the interfaces, are interwoven and must be of a high standard.If the quality of the database is low or the interfaces badly designed, users may find the consultation process frustrating, time-consuming, and even unsuccessful.In the following section, we will therefore consider both aspects when we analyze how the two dictionaries are integrated into the learning app.

Design features and performance
This section will focus on the dictionary integrated into the Kaiyan OpenLanguage app; that is, the dictionary users can consult directly from the course text whenever they have an information need in terms of understanding.It will look at the design features of the user interfaces and discuss some of the problems observed.These problems concern lemma selection (single words, multiword units, and compound words), treatment of polysemous words, grammatical categories, translation equivalents, data overload, and position of the pop-up window. 2

Inconsistent presentation
The treatment of the words occurring in the learning courses is inconsistent.This problem comprises their selection and treatment as lemmata, grammatical categorization, and presentation in the pop-up window.

Selection and treatment of single-word lemmata
The selection of lemmata and their treatment in the incorporated dictionaries seem, at least to some extent, to be arbitrary.Some words appearing in the lessons are not presented as lemmata in the pop-up window but only in the embedded dictionary on the front page.Other words do not perform as lemmata in any of the two options, although they occur in the course texts.Examples of this are Covid and buzzwords, which do not seem to be stored at all in the lexicographical database that feeds the learning app.It goes without saying that such omissions are not helpful to the users' understanding and learning of new words.An example of a word that can be consulted on the front page but not directly from the course text is geeks.It appears in the learning course in a con-http://lexikos.journals.ac.za; https://doi.org/10.5788/31-1-1626text where one of the teachers explains the word esoteric as "not very accessible because it is mostly made by geeks for geeks".However, when users click on geeks, they will get a message informing them that there is "no explanation for the moment" ( 暂无解释).
If they instead try to consult geeks by entering this word in the search box on the front page, a window will be displayed with an extract of an article with the lemma Geeks with capital letters (see Figure 3).Geeks and geeks are two different words.They have different spelling, grammar, and meaning.Geeks is "a brain game on the internet; freak, a boring person", whereas geeks is the plural of geek, which the dictionary defines as "someone who does a clumsy funny show, someone abnormal; savage; someone smart, excellent at studying, but poor in dealing with social networking" (see Figure 4).Consequently, the two words are selected as two different lemmata and presented in separate articles.
The problem is clearly not related to the lexicographical database, but to the underlying programming that does not allow the relevant items contained in the database to be displayed in the user interface.The design of the latter is at fault.An appropriate design would refer the users directly to the article geek when they click on geeks in the text or enter the word in the search box on the front page.And it would also provide a small explanatory note informing them that geeks is plural of geek.

Selection and treatment of multiword lexical units
In his classical Manual of Lexicography, Zgusta (1971: 144) wrote the following about multiword units: Multiword lexical units are very frequent.They carry lexical meanings as wholes (units); in this respect, they function in the sentence in the same way as those lexical units which consists of one word only.
The dictionaries used in the Kaiyan OpenLanguage app do not treat these frequent lexical units in a consistent and user-friendly way.Some of the units appear as lemmata in the pop-up window with equivalents and other lexicographical items when users click on them in the course text.Others cannot be looked up as a whole.For instance, when users click on sick leave in the text, they will not get any definition or other relevant data concerning this multiword unit as a whole but only the words sick and leave separately.However, if the users instead enter the whole unit in the search box on the front page, they will immediately get access to an article that treats sick leave as a normal lemma.Hence, the lexicographical database that feeds the learning app does include sick leave as a multiword lemma.The problem is not the database but, as above, the underlying programming that does not allow the users to get immediate lexicographical assistance in the user interface by clicking on multiword units appearing in the text.
A similar example is dry wit.Learners who want to know the meaning of dry wit can only activate dry or wit but not dry wit as a lexical unit.However, the combined meanings of dry and wit do not explain the meaning of dry wit.The users may therefore find the consultation useless and time-wasting.Other multiword units like get the game on lock, pan seared, go ahead, and right now have the same problems.
Some compound words, like hit-and-miss and start-up, cannot be consulted as a whole either.Only articles treating one of the single words will be displayed when users click on them in the text.A click on the compound start-up will, for instance, activate an article with the lemma start.However, other compounds like self-deprecating and cord-cutting can be activated and displayed as a whole (see Figure 5).This shows an inconsistent treatment of compound words in the Kaiyan OpenLanguage app.

Confusion of grammatical categories
As mentioned, some multiword units do appear as lemmata in the Kaiyan learning app now and then.In such cases, the pop-up window may contain relevant lexicographical items like pronunciation, part of speech, senses, and equivalents.But sometimes part of speech is wrongly indicated, and the equivalents that help the Chinese learners to understand the meaning do not belong to the same grammatical category as the lemma.The expression put to good use, for instance, is presented as a noun, although it is a verbal phrase (see Figure 6).The confusion of grammatical categories may explain why the first of the two equivalents provided is the compound noun 有效使用 (efficient use), whereas the second is the much more suitable verbal phrase 用在该用的地方 (use where things should be used).This may hamper the learning process as well as the understanding of the lemma put to good use.In this case, the problem seems to be a low database quality originating from a poor compilation or storage of the lexicographical data that are presented in the pop-up window on demand.

Inconsistent presentation of senses
When a word occurs in one of the course texts, it always appears in a specific context with a specific meaning.Learners who do not understand the word in this context and activate the default pop-up window will need a definition of its specific meaning to satisfy their needs.However, the relevant senses of some words occurring in the lessons cannot be found in the Concise Dictionary, but only in the Oxford English-Chinese Dictionary.One such example is the senses addressed to the word chunk.When learners click on this word in the course text, an article with the senses "big block" and "stubby person or thing" will pop up (see Figure 7).If learners are aware of the problem and want a suitable definition of chunk as it appears in the above context, they can try to access the whole article by clicking on the signifier 查看完整释义.However, they will once again be disappointed.The article displayed is identical to the former one in terms of the senses explained in Chinese (see Figure 8).The only difference is that the whole article also offers two definitions of the noun chunk written in English ("a compact mass" and "a substantial amount") and one of the verb chunk ("put together indiscriminately"), none of them helpful in the concrete case.(Besides, definitions written in English may be difficult to understand for learners with a low English proficiency level).Users who do not give up can have a third try and consult the Oxford English-Chinese Dictionary by clicking on 牛津 (Oxford).However, as this dictionary lists chunk as both a noun and a verb, they are first directed to an in-between page where they have to click on one of the two word classes to proceed.A new page will finally open with an article that contains various senses of the noun chunk.One of these is the right one in the specific context.It defines the linguistic term chunk as it appears in the lesson as "a phrase or group of words which can be learnt as a unit by sb who is learning a language" and provides two examples to support this definition (see Figure 9).Only users who persist and consult the complete article in either the Concise dictionary (only with a definition in English) or the Oxford dictionary (with a Chinese equivalent) will get the required response to their information need.
The detected problems show that it makes sense to use the Oxford dictionary as a safety net.The problems originate in the Concise database that does not contain the pertinent sense of chunk.Four clicks to get a proper answer may be too much for many learners who just want a quick explanation that allows them to continue reading without interruptions.In this respect, the interface of the default pop-up window is also to blame.It should never invite users to consult the whole article when there is nothing relevant to add.Instead, they should be referred directly to the Oxford dictionary, thus skipping a useless step that may stress them and even result in an abortive consultation.

Between data overload and data underload
Fuertes-Olivera and Tarp (2014: 64) assert that the ideal online dictionary should contain "as much data as possible in terms of all possible consultations", whereas its individual articles "should include as little data as possible" to avoid information overload.From this perspective, a personalized service requires that the dictionary only presents the exact "types and amount of data", which the individual user needs in each consultation, "neither more nor less".These reflections are especially relevant for dictionaries integrated into learning apps.This type of dictionaries should be designed to offer assistance each time the learner asks for it.But at the same time, they should avoid disturbing the learning process with too much data.It is undoubtedly a delicate balance.
We have previously seen various examples of data underload.The default pop-up window that is supposed to provide first aid to users with comprehension problems does not address some lexical units occurring in the course texts.However, there are also problems with some polysemous words.They pose a special challenge because one or several of their relevant senses are sometimes missing.The word buds, for instance, appears in the lesson The explosion of podcasts in the following context: Yes, and of course, podcasts aren't the only form of media that gives you that feeling.I mean a book could also give you that feeling.But there is something about a podcast, you know, you have the buds in your ears, and it's going right into your head.
When users click on buds in this context, an article with three senses will visualize in the pop-up window (see Figure 10).None of them reflect the specific meaning in the context (earbuds or ear headphones).Hence, they do not help learners who do not understand buds in the concrete context.The relevant sense is missing.It is a case of data underload.

Figure 10: Pop-up window with buds in Kaiyan app
Polysemous words do not only consist of two or more senses (sometimes more than ten).They frequently also belong to different parts of speech.Traditionally, there has been a vivid discussion among lexicographers on how to structure senses in dictionary articles.Several ordering principles -like chronology, markedness, frequency, and logic -have been suggested; see Lew (2013: 291-294).The problem in print dictionaries is that the structure is there for good.Digital dictionaries, at least in theory, are more flexible.However, it transpires that the dictionary integrated into the learning app has not found a technological solution to this challenge.The order of both word classes and senses seems to be unrelated to their concrete meaning in the course text.Consequently, the relevant sense and part of speech are not necessarily listed first when users click on a concrete word.The pop-up window with deal is a good example (see Figure 11).It has been activated by clicking on the noun deal in the sentence: "Can you give me a better deal?"The window contains 20 senses (or equivalents) distributed over one verb and two nouns (one of them a proper noun).This structure, together with a certain degree of data overload, will probably put users in a difficult situation when the required meaning is not the first one listed.They will have to struggle to find their way among the many possible solutions.

Figure 11: Pop-up window with deal in Kaiyan app
There are currently excellent tagging programs that can detect the parts of speech of words in a text.But they are far from perfect.Maybe five or ten percent of all words will be assigned to the wrong part of speech.It therefore makes perfect sense that the Kaiyan OpenLanguage app presents different parts of speech together in the pop-up window.However, this does not exclude the use of existing technology to give priority to the senses with the most likely part of speech and place them first in each case.This would, to some extent, alleviate the burden on the users, although it would not remove the general problem of data overload.Many common words belong to more parts of speech and have more senses and equivalents than deal.One of these is set.Figure 12 shows the default popup window with this word.The white frame marks the location where the user can find its specific meaning in the given context: (物品的) 一套, 一组, which means a "suite or series, group (of things)".This meaning item is not easy to find at first glance.It will probably take several seconds to detect, evaluate, and choose the right one.The large amount of irrelevant data obstructs the information search process.So far so bad.The learners' odyssey is not over yet.Apart from being a noun and a verb, set can also be an adjective with several senses and equivalents.These items do not turn up in the default pop-up window shown in Figure 12.Hence, if set appears in the course text as an adjective, learners will have to waste even more time to find and assimilate its concrete meaning.The only way forward is to click on the signifier 查看完整释义 to access the whole article where the adjective is explained (see Figure 13).But before reaching this goal, there are plenty of possibilities that some learners will leave the consultation without any solution to their problem.They will waste their time and sacrifice their energy on the altar of data overload.A different design of the user interface is required.

Miscellaneous
Besides the challenges discussed in the previous sections, the default Concise Dictionary also presents some inadequate definitions and old-fashioned features.Most of these problems can be solved with good lexicographical craftsmanship.In this respect, the Concise Dictionary still has a long way to go to catch up with its Oxford twin in terms of quality.Some words are explained in a way that may confuse even native Chinese speakers.For instance, one of the equivalents used to explain topping is 浇头, a dialect word that may be unknown to many Chinese learners.When they see this equivalent, they will have to consult a Chinese monolingual dictionary to confirm its meaning.Another example is lanyard.One of the equivalents provided is 系索, which is also unfamiliar to many native Chinese speakers.Even monolingual dictionaries do not include 系索 as a lemma.Learners, who look for it in either the dictionaries used in the learning app or an external monolingual dictionary, will find nothing useful.
Moreover, some translational equivalents are inaccurate and even wrong.Issued, for example, is labeled as a verb, but the equivalent is a noun ("someone who issues, publishes things").The equivalent of gumbo is 秋葵浓汤, which means a thick soup with okra.However, gumbo consists of more ingredients than just okra.The first definition of this word in the online Longman Dictionary of Contemporary English is "a thick soup made with meat, fish, and okra". 4 The Chinese definition of some words has a positive connotation, although they appear in the context with a negative connotation.For instance, the first equivalent of perpetuate is 使不朽, which means "keep as eternal glory".This Chinese word has a positive connotation and does not seem to be an appropriate equivalent of perpetuate in the concrete context: "Yeah, as long as it's the truth.What I don't do is perpetuate a culturally-endorsed lie."Perpetuate is here used with an unfavorable meaning.Other words like goblin and silverware have similar problems that may confuse the learners.
Another problem observed in both the Concise and the Oxford dictionary is the use of abbreviations like n, v, vi, vt, adj, etc.This practice had its justification in the space constraints of paper dictionaries.But it is no longer a must in the media convergence age where these constraints have disappeared.In fact, users may sometimes find it difficult to assimilate the meaning of such abbreviations.They are an unnecessary burden on the shoulder of learners who have to confront more relevant challenges when studying a second language.
A final problem is the phonetic transcription of pronunciation, which is a leftover from the print dictionary.Traditionally, many dictionary users have struggled to convert this transcription into a more or less correct pronunciation.It is completely superfluous as a default data in a multimedia learning app like Kaiyan OpenLanguage where users can listen to the correct pronunciation by clicking on the speaker icon.

Position and size of the default pop-up window
The default pop-up window is situated immediately above or below the word that users have clicked on.This position is sometimes problematic as it may hide the relevant context.The users' eyes often move between the window and the context while trying to pick up the right meaning during the consultation.Figure 14 illustrates the problem.The screenshot is from a smartphone and shows how the pop-up window covers the context in which perpetuate occurs.This position makes it difficult for the users to grasp the negative connotation discussed above.The only option for the users is to close the window to see the context and then reactivate the window to know more about the word.This switching between the pop-up window and the course text may be necessary several times until they find the right answer.It will prolong the consultation process and probably stress the learners because it takes focus away from the lesson.Many of them may be tempted to stop the consultation right away.An alternative solution is therefore required.From a technological point of view, there seem to be two options that go hand in hand.The first is to place the pop-up window elsewhere on the screen or, at the least, make it movable so the users can place it where it is most convenient.The second is to reduce its size and make it less dominant.The latter is the most complicated.It requires a combined technological and lexicographical effort that aims to reduce the number of items in the window to an absolute minimum, as recommended by Fuertes-Olivera and Tarp (2014).We will discuss this challenge in Section 6.

Negative effects on consultation and learning
In her study of the information search process, Kuhlthau (1991: 367) observed a whole spectrum of feelings in different phases of the process: Affective symptoms of uncertainty, confusion, frustration, and anxiety prevail when users encounter challenges and difficulties.A sense of personal inadequacy may even prompt some of them "to abandon the search altogether".These symptoms are replaced by feelings of certainty, relief, and confidence when users make progress.The study focused on information search in libraries.But similar feelings may also exist among users of learning apps with integrated dictionaries.Kuhlthau's findings somehow converge with Krashen's Affective Filter hypothesis.According to the author of this hypothesis, various affective variables "relate to success in second language acquisition" (Krashen 1982: 31).These variables include motivation, self-confidence, and anxiety.Learners generally do better in second language acquisition when they are highly motivated, self-confident, and not anxious.By contrast, if the teaching material is too complex or too difficult for them to assimilate, they may feel anxious and lose their self-confidence.This state of mind would certainly have negative effects on the learning process.
In the previous section, we detected several problems related to the dictionaries used in the Kaiyan OpenLanguage app, especially the Concise Dictionary that functions as the default dictionary in the pop-up window.We have seen lexical units that are not treated in the pop-up window, and sometimes not even in the dictionaries when consulted from the front page.We have seen polysemous words where some senses are missing in the default dictionary or only available after accessing the whole article.We have seen words with inaccurate and even wrong definitions.We have seen examples of data overload with senses and equivalents that are irrelevant in the concrete context.We have seen users who have to click three or four times to get an answer or no answer at all.We have seen how the position of the pop-up window that is supposed to help users sometimes has the opposite effect and make it more difficult to pick up the meaning of a word. 5 All these problems show that the dictionaries not only assist the learners with lexicographical data.They also create all sorts of obstacles.As a result, learners will have to invest more time and energy in searching for the required information.Even so, their efforts will often be in vain.The time wasted in searching will interrupt the reading of the course text.The many fruitless lookups will impede their understanding of the context and the learning of new words.The unpleasant user experience may cool the learners' interest in dictionaries and make them stop using them.They will become anxious and less motivated.Their English learning process will not go as smoothly as they and the app designers expected.They may even lose confidence in themselves and their ability to learn English.At worst, they may eventually stop using the learning app and give up learning English altogether.
Nobody wants this to happen, neither the designers nor the learners.Something has to be done.Dictionary consultation should be a support to language learning, not an obstacle as described above.The next section will come up with some suggestions to improve the app from a lexicographical perspective.

Suggestions
The central function of the integrated dictionary is to help users understand the course texts.In this respect, learners who have problems need a quick lexicographical response to prevent them from losing focus and reading flow.The design of the pop-up window must therefore be as user-friendly and functional as possible.There is, however, also another relevant function that has to do with the complex relationship between incidental and intentional learning.Incidental learning prevails when learners gradually pick up new words, senses, and grammatical structures during reading.The pop-up window sup-http://lexikos.journals.ac.za; https://doi.org/10.5788/31-1-1626ports this part of the learning process.But learners are learners with a curious and exploring mind.When they have digested the lexicographical data displayed in the window, many of them will, from time to time, and for one reason or another, tend to go into detail about words, meaning, grammar, and cultural connotations.They will then pass from incidental to intentional learning.It thus makes sense that the learning app allows them to access the whole article either from the default pop-up window or the front page.It is a matter of course that this part of the lexicographical support should also be high quality.It represents the second layer of assistance to the users, without which the following suggestions for improvement of the pop-up window would be less attractive.

The ideal pop-up window
In the Introduction, we referred to the Chinese Xun Gu tradition and a similar one in Europe.This millennial experience has inspired us to design what we consider the ideal pop-up window.Figure 15 depicts how we imagine this window.It represents the lexicographical response to users who click on set in the context "based upon their specific skill set", which we also used in Figure 12.The main idea is that the pop-up window should only include items that can be justified by the immediate user needs.Thus, it merely consists of a speaker icon, a meaning discriminator followed by two equivalents, and a signifier (>).The central item is the definition (or equivalents) that directly assists understanding of the course text.Other senses of the noun set are considered to be irrelevant in the concrete context and therefore excluded.The window also includes a speaker icon to service learners who, as recommended by language didactics, read aloud and may need to listen to some of the words to pronounce them right.Finally, it provides a widely used signifier that affords access to the whole article and is well-known to most netizens.In addition to these three compulsory items, we also envisage a note that relates a grammatical word to its canonical form (for instance, thought and think).In some cases, this note may be sufficient for the users to understand a word appearing in the text.
In the proposed pop-up window, the lemma has disappeared.The inspiration from the Xun Gu tradition is crucial here.The lemma is simply superfluous.The old scribes did not need it, and neither do the digital-era scribes.When discussing a similar pop-up window in a writing assistant, Tarp (2019: 240) observes that the lemma "seems to be completely redundant as the user perfectly well knows from which word the article has been accessed."This way of excluding irrelevant items prevents data overload with negative consequences as anxiety, frustration, and abortive consultation.The design and layout shown in Figure 15 are provisional.The final design should be human-centered and optimized by test-driven development (TDD), as recommended by Norman (2013) and adapted to lexicography by Tarp and Gouws (2020).This type of optimization concerns both the presentation of the respective items in the pop-up window and the appearance, color, shape, and default position of the window itself.

How to achieve the ideal pop-up window
The method to be applied is data filtering, a technique developed within information science; see Bothma (2011).It implies that the user interfaces upload the lexicographical data from the database and offer them to users in carefully metered doses to meet their specific needs in different situations.With this technique, a single database can feed various types of dictionaries and digital products.If well-designed, it can provide relevant data to both the integrated and the embedded dictionaries in the learning app.However, even cuttingedge filtering techniques assisted by artificial intelligence cannot by themselves guarantee the reduction of relevant senses to the minimum shown in Figure 15.
The minimization is also attributable to a unique characteristic of the course texts, which differ in one important aspect from those that can be found on the Internet or in e-readers like Kindle (see Bothma and Prinsloo 2013).This aspect is the limited and controlled number of words occurring in the course texts.It allows a different lexicographical approach to text reception.Artificial intelligence contributes more and more to both language learning and lexicography.Among other things, it can detect text structures and patterns that escape the human eye, but it is nonetheless stupid as it does not understand either single words or texts as a whole.It is therefore necessary to incorporate human-assisted intelligence so that the dictionary will "know" the specific meaning of lexical units in the concrete context where they occur.How can that be done?
It requires a combination of programming and manual work.A language expert goes through the existing course texts (and the new ones that will be added in the future).This expert should be a native speaker or another person with a high proficiency level in English, preferably an experienced second-lan-http://lexikos.journals.ac.za; https://doi.org/10.5788/31-1-1626guage teacher who knows where the shoe pinches.He or she clicks on relevant lexical units, including the extended units of meaning that require special attention (see Sinclair 2010 andRundell 2018).After the default dictionary is displayed in the pop-up window, the expert marks the right meaning items.This information is automatically recorded by a program specially developed for this purpose.When learners subsequently click on the corresponding lexical units, the pop-up window will only present the lexicographical data marked by the expert; that is, data that help them understand lexical units in their concrete context.In this respect, the tool has become context-aware.If the users want additional information, they can click on the arrow (>) to access the complete article with its conventional features.
The suggested method is highly productive.A well-trained expert will probably be able to mark two-three words or units per minute.The solution is also economical as the described program is relatively simple and easy to write. 6Over the long term it could be supplemented by various types of user feedback.All this will greatly improve the quality of the learning app.

Interdisciplinary collaboration
The lexicographical database used to sustain the Kaiyan OpenLanguage app was probably compiled before the app; that is, without considering the specific requirements of this tool.It would explain many of the deficiencies observed.Fuertes-Olivera and Tarp (2020: 268) discuss a similar problem related to a writing assistant.They express serious doubts about the capacity of existing databases to feed innovative digital tools like writing assistants and conclude that "it seems necessary to restart almost from scratch".We can recognize many of their observations and reservations.We nevertheless believe that it is possible to improve the database that feeds the Kaiyan OpenLanguage app as long as there is a close collaboration between the different stakeholders.The analysis of the integrated dictionary in Section 4 revealed several cases where single words, multiword units, and specific senses were missing in the database, although they occurred in the course texts.When the language expert who scrutinizes the texts discovers these omissions, they should be marked and sent to the compilers of the dictionary for immediate response.Depending on the work burden, the new items could be ready and activated within 24 hours.This requires close interdisciplinary collaboration between the app designers and the lexicographers responsible for the database.
The described collaboration and interaction may not be possible in terms of the Oxford dictionary.But it is probably much easier with the Concise dictionary as we consider the same company to be behind both this dictionary and the learning app.If it is not the case, agreements on close interdisciplinary collaboration should be made between companies when licensing a dictionary to be used in digital tools like the learning app.The database must necessarily be dynamic and under continuous development, improvement, and updating, among other things because new texts are regularly added to the course.If not, the company will never get high-quality lexicographical support for its learning app.

Conclusions
Digital applications to assist second-language learning are becoming increasingly popular in China and elsewhere.These apps typically incorporate one or two dictionaries to improve the service and avoid that users leave the app to look up in external dictionaries.In this contribution, we have analyzed the dictionary integrated into the Kaiyan OpenLanguage app, which is one of the most comprehensive app used by Chinese learners of English.The study has detected a number of inconsistencies and deficiencies such as words, multiword units, and senses that occur in the course texts but are not treated in the pop-up window or are difficult to find because of data overload.These problems may impact negatively on learners' motivation and the learning process as such.Thus, we have suggested an alternative, context-adapted approach inspired by the work of classical Chinese and Greek scribes.The content of the proposed pop-up window has been reduced to an absolute minimum.The idea is to avoid a consultation process that interrupts the learners' focus and reading flow.
The study focuses on a particular Chinese learning app.Yet, we believe that its main conclusions and suggestions can be generalized, first and foremost to be adapted to other English learning apps for Chinese users.But it is probably also interesting for designers of apps that assist learners of English with other mother tongues as well as the learning of other languages.

2.
Our observations are based on numerous consultations between October 20, 2020, and January 14, 2021.Some of the discussed examples may have changed after being consulted.

3.
The pop-up window is redesigned to facilitate reading.In this and other figures, an English translation has been added for the benefit of non-Chinese speaking readers.4.

5.
One of the authors has used the app more than three months and recognizes some of the adverse feelings.The idea to the article was triggered by this experience.6.
This was confirmed by Kasper Fisker, a computer scientist at the Center of Excellence in Language Technology at Ordbogen A/S, Denmark.

Figure 1 :
Figure 1: Lexicographical pop-up window activated from the text

Figure 2 :
Figure 2: Extract of article activated from the front page

Figure 3 :
Figure 3: The article Geeks accessed from front page in Kaiyan app 3

Figure 4 :
Figure 4: The article geek accessed from front page in Kaiyan app

Figure 5 :
Figure 5: Pop-up window with cord-cutting in Kaiyan app

Figure 6 :
Figure 6: Pop-up window with put to good use in the Kaiyan app

Figure 7 :
Figure 7: Pop-up window with chunk in Kaiyan app However, the senses explained in Figure 7 are not relevant in the following context quoted from the course: Yeah.So this is a chunk.It's a collocation.They always go together ...

Figure 8 :
Figure 8: Extract of article chunk accessed by clicking on SEE WHOLE ARTICLE

Figure 9 :
Figure 9: The three senses of the noun chunk in Oxford A similar example is the verb furnish.It appears in the lesson with the meaning "to put furniture in a house, room, etc."But the only definition in the default pop-up window is "to offer; to supply; to equip with".Hence, the definition does not inform the learner with what a house or room is equipped (furniture).Only users who persist and consult the complete article in either the Concise dictionary (only with a definition in English) or the Oxford dictionary (with a Chinese equivalent) will get the required response to their information need.The detected problems show that it makes sense to use the Oxford dictionary as a safety net.The problems originate in the Concise database that does not contain the pertinent sense of chunk.Four clicks to get a proper answer may be too much for many learners who just want a quick explanation that allows them to continue reading without interruptions.In this respect, the interface of the default pop-up window is also to blame.It should never invite users to consult the whole article when there is nothing relevant to add.Instead, they should be referred directly to the Oxford dictionary, thus skipping a useless step that may stress them and even result in an abortive consultation.

Figure 12 :
Figure 12: Pop-up window with set in Kaiyan app

Figure 13 :
Figure 13: The adjective set displayed on a new page with the whole article

Figure 14 :
Figure 14: Pop-up window with perpetuate in Kaiyan app

Figure 15 :
Figure 15: Content of ideal pop-up window with the noun set The difference between the proposed window and the one currently used in the Kaiyan OpenLanguage app becomes crystal-clear if we compare Figures 15 and 12.The proposal not only contains an absolute minimum of items to meet the users' needs in the concrete context.It also breaks with well-established features of the traditional dictionary article.The main idea is that the pop-up window should only include items that can be justified by the immediate user needs.Thus, it merely consists of a speaker icon, a meaning discriminator followed by two equivalents, and a signifier (>).The central item is the definition (or equivalents) that directly assists understanding of the course text.Other senses of the noun set are considered to be irrelevant in the concrete context and therefore excluded.The window also includes a speaker icon to service learners who, as recommended by language didactics, read aloud and may need to listen to some of the words to pronounce them right.Finally, it provides a widely used signifier that affords access to the whole article and is well-known to most netizens.In addition to these three compulsory items, we also envisage a note that relates a grammatical word to