f hh diva portal org

F Hh Diva Portal Org-PDF Download

  • Date:24 Nov 2019
  • Views:111
  • Downloads:0
  • Pages:45
  • Size:1.13 MB

Share Pdf : F Hh Diva Portal Org

Download and Preview : F Hh Diva Portal Org


Report CopyRight/DMCA Form For : F Hh Diva Portal Org


Transcription:

Chatbot for Information,Retrieval from Unstructured. Natural Language Documents, Bachelor Thesis for the Computer Science and. Engineering Programme, Falk Ho ppner Joakim Fredriksson. Halmstad University, Supervisor Eric Ja rpe, Examiner Struan Gray. June 7 2019, Abstract, This thesis brings forward the development of a chatbot which retrieves infor .
mation from a data source consisting of unstructured natural language text . This project was made in collaboration with the company Jayway in Halm . stad Elasticsearch was used to create the search function and the service. Dialogflow was used to process the natural language input from the user A. Python script was created to retrieve the information from the data source . and a request handler was written which connected the tools together to create. a working chatbot The chatbot correctly answers questions with a accuracy. of 72 according to testing with a sample of n 25 The testing consisted of. asking the chatbot questions and determining if the answer is correct Possible. further research could be done to explore how chatbots might help the elderly. or people with disabilities use the web with a natural dialogue instead of a. traditional user interface , Sammanfattning, Denna rapport presenterar utveckling av en chatbot som ha mtar informa . tion fra n en dataka lla som besta r av ostrukturerad naturlig text Detta pro . jekt utfo rdes i sammarbete med fo retaget Jayway i Halmstad Elasticsearch. anva ndes fo r att skapa so kfunktionen och tja nsten Dialogflow anva ndes fo r. att behandla det naturliga spra ket fra n anva ndaren Ett programm skrevs. i Python som ha mtar information fra n dataka llan och en request handler. skrevs som sammankopplar de olika delarna fo r att skapa en fungerande chat . bot Chatboten svarar korrekt med en nogrannhet pa 72 enligt testning med. n 25 Testningen bestod av att sta lla fra gor till chatboten och avgo ra om sva . ret a r korrekt Mo jlig vidare forskning skulle kunna bero ra hur chatbotar kan. hja lpa a ldre och personer med funktionsnedsa ttningar anva nda webben genom. att anva nda naturlig dialog ista llet fo r ett traditionellt anva ndargra nssnitt . Contents,1 Introduction 1, 1 1 Aim 1, 1 2 Problem definition 2. 2 Background 3, 2 1 Natural language conversation frameworks 3. 2 2 Chatbot advantages 3, 2 3 Full Text Search Engine 4. 2 4 Related work 5, 2 5 Prerequisites 6,3 Theory 9.
3 1 Natural language processing 9, 3 2 N Grams 9, 3 3 Term Frequency Inverse Document Frequency 10. 4 Method 13, 4 1 Specification 14, 4 2 Analysis 14. 4 3 Scraper 14, 4 4 Elasticsearch 15, 4 5 Dialogflow 16. 4 6 Request Handler 17, 4 7 Security 17, 4 8 Programming languages 18. 4 9 Platform 18,5 Results 21, 5 1 The chatbot 21, 5 1 1 Version A 21.
5 1 2 Version B 22, 5 1 3 Version C 24, 5 2 Analysis 24. 5 2 1 Jayway Feedback 24, 5 2 2 Testing 25, 6 Discussion 27. 6 1 Milestones 27, 6 1 1 Scraper 27, 6 1 2 Search engine 27. 6 1 3 Natural language processing 28, 6 1 4 Chatbot 28. 6 2 Societal demands on technical product development 29. 7 Conclusion 31,Bibliography 33,A Questions 35, 1 Introduction.
Traditional visual websites and user interfaces may soon be on their way out . Chatbots look to be a prime candidate for replacing or enhancing the way a. user interacts with their devices , More and more data and information is available on the world wide web . However a lot of this information is in the form of free form text which. can make it difficult to extract the desired information Currently search. engines are commonly used to find the information users are interested in One. disadvantage of this method is that the user has to be very specific with their. search query Machine learning makes it possible to design a more intuitive. method for searching and to present the results in a personalised way . This method of searching uses a dialogue instead of a traditional search query . The input from the user does not need to consist of a couple of keywords . but may be comprised of complete sentences The input is then interpreted. by a chatbot in order to search for the desired information Nowadays the. majority of searches are conducted via text A chatbot that holds a natural. conversation is also more suitable for using a voice based interface as the bot s. responses more naturally fit in a conversation than results from a traditional. search engine , A study about why people would like to use chatbots was performed in 2017. by Petter Bae Brandtzaeg and Asbj rn F lstad In this study 146 participants. were asked why they would want to use a chatbot 100 of the 146 participants. believed it would help them with their productivity The most reported reason. was ease of use in information and assistance retrieval 1 . This kind of search feature has already started being used by a number of. companies One of them is an insurance group from Switzerland who use. a chatbot which makes it possible to report a bike theft using a messaging. application 2 , The purpose of the project is to develop a chatbot for the company Jayway . which can be used to answer questions from employees or guests The bot is. intended to be an alternative to the search engine available on Jayway s internal. wiki The users questions can be anything from what the password to the Wi . Fi network is or how to go about completing certain work tasks Jayway s. internal wiki will be used as the source of information for the chatbot As the. wiki is only available in English the chatbot will only be able to communicate. using English , One possible extension of the project is to make it possible for the bot to re . trieve answers from other sources such as Wikipedia 1177 se or the Halmstad. Municipality website for example This may take considerable effort and will. only be attempted if there is time for it , Google Home integration is a further goal As the bot will already work with.
free form natural text as a user interface it ought to be relatively simple to. extend it to also use speech as a source of input and to integrate it with Google. Home As with the previous extension this will only be done if there is time. left over ,1 2 Problem definition, When creating systems with some sort of intelligence it is quite difficult to. determine the quality of the system In the case of the development of a. chatbot that will answer questions the result will have to meet some criteria . Is the response made in such a way that it feels human Do the results have. too much information or too little These are some problems that have to be. taken into consideration when developing the system . To complete the project the following milestones have been determined . Scraper Can a program be built to retrieve the data from the internal. wiki and build a database , Search engine Is it possible to build a functional traditional search. engine that works with the database , Natural language processing Can natural language processing be used. as a user interface to this search engine , Chatbot Is it possible to connect the milestones listed above to make. a functional chatbot , 2 Background, A natural dialogue has become very desired in today s development in inter .
action with different platforms Much research has been done on how people. converse and how human language can be used as a form of input 3 Nowa . days it is not unusual that companies develop chatbots for different purposes . For example there are Google Assistant Siri and Amazon Alexa which can. all answer general questions and hold a conversation with the user They can. also complete simple tasks for example reminding the user about something . or adding an appointment to the user s calendar Other companies use chat . bots to solve more specific problems for instance the front end of customer. support or as a method for their customers to place orders . 2 1 Natural language conversation, frameworks, A number of different cloud based tools and frameworks for developing chat . bots have also entered the market 4 These services are made for the devel . opment of systems that use natural language as a way to communicate with. the user To simplify development the services come with a wide range of tools. that facilitate the analysis of natural language built in Some of the services. have the possibility to integrate the system with a smart speaker that will be. available to get input by voice communication Each service has libraries for. different programming languages however a lot of the systems have languages. such as Java Python and Node js in common ,2 2 Chatbot advantages. Because of an increase in the amount of people using smart phones and a. decrease in desktop computers or laptops as seen in figure 2 1 companies have. tried to make user interfaces as easy and intuitive as possible As more modern. product development processes like DPD Dynamic product development 5 . put a larger focus on the user than the product companies are trying different. ways to communicate with the user to build a more personal relationship . Chatbots are very suitable for this kind of solution as the communication is. Figure 2 1 Graph of Mobile and Desktop usage retrieved from http gs . statcounter com, very natural and can be presented in a messaging platform which the majority. of people who have a smartphone are already familiar with This leads to the. user being more involved in the communication and leads to a more positive. experience 1 , When searching for information a traditional search engine will be more ef . ficient as it only requires the entry of keywords while a conversation with a. chatbot may contain extraneous information such as greetings and filler words. that are used to build sentences Nevertheless a traditional search engine may. not be the most preferred way of receiving information as it may be perceived. as dull and impersonal However for some users this is still the preferred. way of searching for information but for companies that focus on building a. positive relationship with the users a chatbot is the preferred solution . 2 3 Full Text Search Engine, To be able to search and retrieve snippets from the wiki some sort of search.
engine has to be implemented Today there are many different full text search. engine that can be adapted for this application instead of having to build. a custom solution These search engines are made for retrieving information. from unstructured natural language databases Information retrieval from such. text databases is quite difficult as unstructured data is hard to index It is. also difficult to determine which words should be considered more important in. the search In contrast in a structured database every value has some sort of. unique identifier which makes information retrieval significantly more straight . forward To solve these problems these libraries make use of methods and. algorithms to give each word some kind of value to make information retrieval. possible When it comes to full text search most of the libraries are built. around term frequency inverse document frequency or TF IDF as described. in section 3 3 which searches using a query and tries to find documents which. match with every term in the query ,2 4 Related work. One of the very first chatbots that was created was called ELIZA 6 and was. completed in 1966 The program was developed at the Massachusets Insti . tute of Technology in their Project on Mathematics and Computation MAC . could hold simple conversations with users and answer with natural human. language ELIZA was mainly based on the identification of keywords in the. user input and predefined transformations applied to the input The anal . yser iterated over the text and retrieved a keyword The keyword had a form. of rank which was used for selecting the keyword that is most likely correct . Therefore if a keyword with a higher rank was found the previous keyword. would be discarded This keyword iteration continued until punctuation was. found If a keyword had been located before the punctuation the remaining. text would be discarded from the input and would therefore not be analysed . If a keyword was not located the same procedure would begin after the punc . tuation ELIZA was very advanced for the time however ELIZA was quite. limited from today s point of view , In 2006 IBM developed their chatbot Watson 7 Watson is significantly. more advanced than ELIZA and can analyse sentences with deeper and more. effective methods It uses deep natural language processing which is a method. that uses machine learning to analyse sentences Watson uses context and a. large database of knowledge to be able to answer with a very high degree of. accuracy Watson answers by first dividing the question into smaller parts . and then retrieving similar questions from its knowledge database After this . hundreds of different possible answers are retrieved and compared to similar. questions With this information a value is calculated which indicates the. probability that an answer is correct and the answer with the highest score is. presented , Compared to the time when ELIZA was created the average person now has. access to a cellphone which is able to communicate. were asked why they would want to use a chatbot 100 of the 146 participants believed it would help them with their productivity The most reported reason was ease of use in information and assistance retrieval 1 This kind of search feature has already started being used by a number of companies One of them is an insurance group from

Related Books