Web based Dialogue and Translation Games for Spoken

Web Based Dialogue And Translation Games For Spoken-PDF Download

  • Date:13 Jan 2020
  • Views:65
  • Downloads:0
  • Pages:8
  • Size:575.01 KB

Share Pdf : Web Based Dialogue And Translation Games For Spoken

Download and Preview : Web Based Dialogue And Translation Games For Spoken

Report CopyRight/DMCA Form For : Web Based Dialogue And Translation Games For Spoken


Figure 1 Screen shot of Web based drill exercise in which the student must solve a weather scenario The student is provided explicit. feedback on any tone errors correcting jia1 qi1 and hui4 in the example. 2 Web based Exercises and Games phrases and sentences in Chinese The system exploits a spo. ken language translation system both to provide translation as. All aspects of learning a new language are difficult to master sistance and to assess the student s performance The initial. and require persistent exposure and concentration Comput prototypes of this system were designed for telephone based. ers can potentially assist the student in many aspects such as interaction The system allows the student to speak in either. vocabulary and pronunciation acquisition as well as reading English or Chinese and it replies first with a paraphrase in the. skills Computerized flash cards can aid in vocabulary building language spoken followed by a translation into the other lan. and pronunciation assessment technology can provide feedback guage Thus the student can say a sentence in English and. on the quality of the student s oral productions 5 20 A com then attempt to repeat the spoken translation provided by the. puter can help a student to read by following along as they read system Versions of this prototype were built in three domains. out loud a presented passage 4 weather 28 flights 30 and a general travel phrasebook do. Our focus however is on oral communication skills Our main. intent is to design a variety of different interactive activities. that are based on spoken communication Each activity is as This system design was problematic in that it put the bur. sociated with a particular set of vocabulary items and linguistic den on the student to think of things to say within the domain. constructs It is important for the game to be goal directed and and did not set any explicit goals In order to convert it into a. it is crucial to provide adequate scaffolding such that the stu more task oriented approach we needed to design a Web based. dent can make progress without getting unduly frustrated We version such that the assigned goal s could be presented on the. are also coming up with ways to reward success by graduating screen Our first attempt along these lines made use of scenar. to higher difficulty levels By contextualizing their speech we ios in the weather domain randomly generated by the computer. can greatly improve the recognition accuracy since the design and presented to the student as simple English keywords rain. can support a narrowly defined linguistic space at any time Dallas Friday as illustrated in Figure 1 The intial inter. face only supported typed input The student was tasked with. To help the student prepare for a given interactive activity typing in a sentence using pinyin format to query the computer. we are developing a set of translation exercises which each fo for the constraints specified by English slot fills A nice feature. cus on precisely the subset of the language needed to master the is that the system can automatically correct any errors in tone. corresponding interactive game The student would logically and highlights the corrected tones Since English speakers have. first practice the focused translation exercises where the goal is tremendous difficulty learning tone this is an effective way to. to learn how to formulate and speak sentences in the domain help them acquire tonal knowledge. They would then be able to eavesdrop on computer computer. simulations of dialogues in the domain Finally after such ex Once we solved the problem of speech capture at a Web. tensive preparation they would be ready to play the interactive page it was relatively straightforward to design computer. dialogue game guided translation exercises on the Web that allowed the stu. dent to practice speaking in Chinese An example interface for a. 2 1 Translation Exercises flight domain translation game is shown in Figure 2 The system. randomly generates a small set of English utterances from tem. We have developed a set of translation exercises in a number plates and displays them on the screen The student is prompted. of different domains allowing a student to practice speaking for each utterance in turn and they then attempt to speak an ut. Figure 2 Web interface for a translation game in the flight domain The system poses English phrases for which the student is tasked. with speaking or typing an equivalent sentence in Mandarin The dialogue unfolds from bottom to top in the upper window The student. can at any time type a word or phrase in English in the type in window to obtain a spoken translation. terance of equivalent meaning in Chinese The system responds the weather domain 31 the flight domain 30 and the hobbies. with a paraphrase in Chinese followed by a translation into En and schedules domain 3 Translation reversibly between En. glish and then congratulates the student if it matches the prompt glish and Chinese utilizes an interlingual method We have con. at the meaning level and moves on to the next utterance If the figured a slightly different variant for the hobbies and schedules. student is stuck they can ask the system to provide a transla domain which gives the student a little more control Instead. tion which they can then attempt to repeat A natural score of the system prompting for each uttterance in turn it simply. emerges from this process based on the total number of turns presents the entire set for the session on the page and invites the. taken on average for each utterance student to translate any of the utterances The system performs. This system has been configured with ten graded difficulty NM N. an x match between the best list from the recognizer. levels ranging from isolated words at the lowest level to com M. and the candidates on the screen, pound and complex sentences at the highest level The student. is automatically graduated to higher levels based on good 2 2 Interactive Dialogue Games. performance and can slide to lower levels if they do poorly. Sentences are generated randomly from thousands of templates Telephone based Systems. generated from real user data such that the game does not ap When we first began to think of using spoken dialogue in. pear monotonous or repetitive even if they stay at the same teraction for language learning it was logical to simply adapt. level The student s translation is evaluated at an attribute our existing multilingual dialogue systems for language learn. value level rather than as a string match such that multiple ing purposes Thus we developed phone based systems config. variants of the Chinese translation are accepted The translation ured to support speech understanding in both English and Chi. and student evaluation procedures are described more fully in nese Dialogue interaction involved asking questions about the. Section 3 Please see 30 for further details weather 34 27 or planning an air travel itinerary 25 30 The. We have piloted this flight domain translation game in student was required to communicate with the system in Chi. a user study which yielded encouraging results in an exit nese but could speak an English sentence at any time in which. poll 33 All of the twelve subjects thought the system was case the system would speak a sentence of equivalent meaning. helpful at improving their Chinese and most would play it again in Chinese The student could also ask for a translation of the. and recommend it to their Chinese learning friends system s response at any time. The exercise demands extremely high performance in terms While these systems were challenging to build and fun to. of translation accuracy and reasonably high speech recogni play with the lack of a structured game made the student less. tion accuracy in the face of the heavily accented and disfluent motivated to persist in conversing with the system on random. speech of a language learner Speech recognition accuracy can topics What was needed was a set of explicit goals along with. be significantly improved by exploiting context the utterance a schema for advancement to higher difficulty levels Also. prompt to influence best selection as described in 32 the topic of conversation was not the most appropriate for a. Furthermore the student can exploit the help mechanism to beginner student. hear a translation spoken by a speech synthesizer They then. only need to produce an accurate imitation of what they heard The Novice Student. We now have systems capable of high quality translation in We have recently come up with a design for an interactive. game based on family relationships which assumes no prior Student What are you doing tomorrow afternoon. knowledge of Chinese This game is designed to permit the System I m playing basketball Would you like to. student to deduce the Chinese names for relationships by trial join me. and error It is configured with multiple difficulty levels to Student No I don t like to play basketball. provide the student with a concrete sense of progress A novely System What do you like to do. of the interface is that pictures of family members also operate Student I like playing video games and watching movies. as record buttons such that the student can simultaneously System Would you like to get together to watch a. select an image for context and initiate recording I will not say movie this evening. more about this game here as it is fully described in 18 Student No I m going to study this evening. System Are you free tomorrow evening, A Video Game Design Student Yes Let s watch a movie tomorrow evening. We are exploring a number of different options for config. uring games that will achieve the look and feel of a video game Figure 4 Example English dialogue in the Hobbies and Sched. while still focusing on teaching a foreign language Ideally the ules domain. game would include some element of competition either with. the clock or with an opponent in order to increase its intrinsic. entertainment value At the same time it would be convenient rections midstream It will also alert them to ambiguities due to. to allow a developer to easily swap in new vocabulary items inadequate specification by highlighting multiple items. to help with vocabulary building We are currently in a brain We have an opportunity with this game design to explore. storming phase considering options for a card game simi two party human human computer games where two players. lar to Go Fish or Gin Rummy where the cards would be compete at separate computers on filling the slots in the second. manipulated through spoken interaction in Chinese and partic row in a race condition We have built a prototype of this design. ipants could be either virtual or human making use of a collab and members of our laboratory have played the two party game. orative game interface on the Web in Chinese We are encouraged by the potential of this game. to achieve an element of fun and excitement rarely found in. language drill exercises, Furthermore this game represents a prototype for a number. of variants that will allow us to substitute other objects in place. of the basic shapes which will then make it more generally. useful for vocabulary building,Spoken Dialogue Interaction Game. Our most sophisticated dialogue interaction game was de. veloped with the goal of supporting the curriculum of a typi. cal first year student We have provided a significant amount. of scaffolding to assist the student with the task However. the scaffolding is incrementally peeled away as the student ad. vances to higher difficulty levels We consulted a high school. text book 29 for guidance and settled on a Hobbies and. Schedules game that would exercise vocabulary and grammar. constructs introduced in the first year textbook The game is. based on controlled scenarios where the student is assigned a. specified persona and a future schedule and is tasked with. Figure 3 Screen shot of the Shapegame Web interface finding a time in the future to jointly participate with the simu. lated dialogue partner in an activity that both parties like. A game that meets many of these design goals was re. Access to the dialogue game is obtained by simply enter. cently developed by a student in our group Alex Gruenstein 7. ing a Web page following a log in step The student s assigned. The screen is populated with displayed objects that represent. preferences as well as a schedule of events in the next few days. large small colored shapes triangles rectangles squares cir. are displayed on the screen The student engages in interactive. cles as illustrated in Figure 3 The student is tasked with first. spoken dialogue with the computer in Chinese to solve the sce. selecting verbally one of the shapes in the last two rows and. nario A typical dialogue interaction in English is shown in. then moving the selected object to one of the empty slots in row. 2 They are then required to alter the shape verbally to make. it look identical to the one displayed just above it A screen shot of the dialogue game is shown in Figure 5. This game now exists in both English and Chinese Typi Here the student has just completed a successful dialogue with. cal commands in the game are Pick up the large red triangle the system to arrange to meet to watch television together to. and place it in the fifth slot Make it small Interestingly the morrow evening The system misunderstood the user when. Web based Dialogue and Translation Games for Spoken Language Learning Stephanie Seneff MIT Computer Science and Arti cial Intelligence Laborator y 32 Vassar Street Cambridge MA 02139 seneff csail mit edu Abstract It is widely recognized that one of the best ways to learn a for eign language is through spoken dialogue with a native speaker

Related Books