|
1 | | -# NODE-NLP TS |
| 1 | + |
| 2 | + |
| 3 | +# NLP.ts |
| 4 | +[](https://github.com/Leoglme/node-nlp-typescript/actions/workflows/main.yml) |
| 5 | +[](https://www.npmjs.com/package/node-nlp-typescript) |
| 6 | +[](https://www.npmjs.com/package/node-nlp-typescript) |
| 7 | +[](https://www.npmjs.com/package/node-nlp-typescript) |
| 8 | + |
| 9 | + |
| 10 | +node-nlp-typescript is a general natural language utility for nodejs. Currently supporting: |
| 11 | + |
| 12 | +- Guess the language of a phrase |
| 13 | +- Fast _Levenshtein_ distance of two strings |
| 14 | +- Search the best substring of a string with less _Levenshtein_ distance to a given pattern. |
| 15 | +- Get stemmers and tokenizers for several languages. |
| 16 | +- Sentiment Analysis for phrases (with negation support). |
| 17 | +- Named Entity Recognition and management, multi-language support, and acceptance of similar strings, so the introduced text does not need to be exact. |
| 18 | +- Natural Language Processing Classifier, to classify an utterance into intents. |
| 19 | +- NLP Manager: a tool able to manage several languages, the Named Entities for each language, the utterances, and intents for the training of the classifier, and for a given utterance return the entity extraction, the intent classification and the sentiment analysis. Also, it is able to maintain a Natural Language Generation Manager for the answers. |
| 20 | +- 40 languages natively supported, 104 languages supported with BERT integration |
| 21 | +- Any other language is supported through tokenization, even fantasy languages |
| 22 | + |
| 23 | + |
| 24 | + |
| 25 | +### TABLE OF CONTENTS |
| 26 | + |
| 27 | +<!--ts--> |
| 28 | + |
| 29 | +- [Installation](#installation) |
| 30 | +- [Example of use](#example-of-use) |
| 31 | +- [False Positives](#false-positives) |
| 32 | +- [Log Training Progress](#log-training-progress) |
| 33 | +- [Contributors](#contributors) |
| 34 | +- [Who is behind it](#who-is-behind-it-) |
| 35 | +- [License](#license) |
| 36 | + <!--te--> |
| 37 | + |
| 38 | +## Installation |
| 39 | + |
| 40 | +If you're looking to use NLP.js in your Node application, you can install via NPM like so: |
| 41 | + |
| 42 | +```bash |
| 43 | + npm install node-nlp-typescript |
| 44 | +``` |
| 45 | + |
| 46 | +## Example of use |
| 47 | + |
| 48 | +You can see a great example of use in the folder [`/examples/02-qna-classic`](https://github.com/axa-group/nlp.js/tree/master/examples/02-qna-classic). This example is able to train the bot and save the model to a file, so when the bot is started again, the model is loaded instead of being trained again. |
| 49 | + |
| 50 | +You can start to build your NLP from scratch with a few lines: |
| 51 | + |
| 52 | +```javascript |
| 53 | +import { NlpManager } from 'node-nlp-typescript'; |
| 54 | + |
| 55 | +const manager = new NlpManager({ languages: ['en'], forceNER: true }); |
| 56 | +// Adds the utterances and intents for the NLP |
| 57 | +manager.addDocument('en', 'goodbye for now', 'greetings.bye'); |
| 58 | +manager.addDocument('en', 'bye bye take care', 'greetings.bye'); |
| 59 | +manager.addDocument('en', 'okay see you later', 'greetings.bye'); |
| 60 | +manager.addDocument('en', 'bye for now', 'greetings.bye'); |
| 61 | +manager.addDocument('en', 'i must go', 'greetings.bye'); |
| 62 | +manager.addDocument('en', 'hello', 'greetings.hello'); |
| 63 | +manager.addDocument('en', 'hi', 'greetings.hello'); |
| 64 | +manager.addDocument('en', 'howdy', 'greetings.hello'); |
| 65 | + |
| 66 | +// Train also the NLG |
| 67 | +manager.addAnswer('en', 'greetings.bye', 'Till next time'); |
| 68 | +manager.addAnswer('en', 'greetings.bye', 'see you soon!'); |
| 69 | +manager.addAnswer('en', 'greetings.hello', 'Hey there!'); |
| 70 | +manager.addAnswer('en', 'greetings.hello', 'Greetings!'); |
| 71 | + |
| 72 | +// Train and save the model. |
| 73 | +(async() => { |
| 74 | + await manager.train(); |
| 75 | + manager.save(); |
| 76 | + const response = await manager.process('en', 'I should go now'); |
| 77 | + console.log(response); |
| 78 | +})(); |
| 79 | +``` |
| 80 | + |
| 81 | +This produces the following result in a console: |
| 82 | + |
| 83 | +```bash |
| 84 | +{ utterance: 'I should go now', |
| 85 | + locale: 'en', |
| 86 | + languageGuessed: false, |
| 87 | + localeIso2: 'en', |
| 88 | + language: 'English', |
| 89 | + domain: 'default', |
| 90 | + classifications: |
| 91 | + [ { label: 'greetings.bye', value: 0.698219120207268 }, |
| 92 | + { label: 'None', value: 0.30178087979273216 }, |
| 93 | + { label: 'greetings.hello', value: 0 } ], |
| 94 | + intent: 'greetings.bye', |
| 95 | + score: 0.698219120207268, |
| 96 | + entities: |
| 97 | + [ { start: 12, |
| 98 | + end: 14, |
| 99 | + len: 3, |
| 100 | + accuracy: 0.95, |
| 101 | + sourceText: 'now', |
| 102 | + utteranceText: 'now', |
| 103 | + entity: 'datetime', |
| 104 | + resolution: [Object] } ], |
| 105 | + sentiment: |
| 106 | + { score: 1, |
| 107 | + comparative: 0.25, |
| 108 | + vote: 'positive', |
| 109 | + numWords: 4, |
| 110 | + numHits: 2, |
| 111 | + type: 'senticon', |
| 112 | + language: 'en' }, |
| 113 | + actions: [], |
| 114 | + srcAnswer: 'Till next time', |
| 115 | + answer: 'Till next time' } |
| 116 | +``` |
| 117 | +## False Positives |
| 118 | +
|
| 119 | +By default, the neural network tries to avoid false positives. To achieve that, one of the internal processes is that words never seen by the network are represented as a feature that gives some weight to the `None` intent. So, if you try the previous example with "_I have to go_" it will return the `None` intent because 2 of the 4 words have never been seen while training. |
| 120 | +If you don't want to avoid those false positives, and you feel more comfortable with classifications into the intents that you declare, then you can disable this behavior by setting the `useNoneFeature` to false: |
| 121 | +
|
| 122 | +```javascript |
| 123 | +const manager = new NlpManager({ languages: ['en'], nlu: { useNoneFeature: false } }); |
| 124 | +``` |
| 125 | +
|
| 126 | +## Log Training Progress |
| 127 | +
|
| 128 | +You can also add a log progress, so you can trace what is happening during the training. |
| 129 | +You can log the progress to the console: |
| 130 | +
|
| 131 | +```javascript |
| 132 | +const nlpManager = new NlpManager({ languages: ['en'], nlu: { log: true } }); |
| 133 | +``` |
| 134 | +
|
| 135 | +Or you can provide your own log function: |
| 136 | +
|
| 137 | +```javascript |
| 138 | +const logfn = (status, time) => console.log(status, time); |
| 139 | +const nlpManager = new NlpManager({ languages: ['en'], nlu: { log: logfn } }); |
| 140 | +``` |
| 141 | +
|
| 142 | +### Pour plus d'informations, vous pouvez consulter la documentation de [nlp.js](https://github.com/axa-group/nlp.js). |
| 143 | +
|
| 144 | +## Contributors |
| 145 | +
|
| 146 | +[](https://github.com/axa-group/nlp.js/graphs/contributors) |
| 147 | +
|
| 148 | +## Who is behind it`?` |
| 149 | +
|
| 150 | +This project is developed by AXA Group Operations Spain S.A.and leoglme. |
| 151 | +
|
| 152 | +If you need to contact us, you can do it at the email <a href="mailto:opensource@axa.com">opensource@axa.com</a> |
| 153 | +
|
| 154 | +## License |
| 155 | +
|
| 156 | +Copyright (c) AXA Group Operations Spain S.A and leoglme. |
| 157 | +
|
| 158 | +Permission is hereby granted, free of charge, to any person obtaining |
| 159 | +a copy of this software and associated documentation files (the |
| 160 | +"Software"), to deal in the Software without restriction, including |
| 161 | +without limitation the rights to use, copy, modify, merge, publish, |
| 162 | +distribute, sublicense, and/or sell copies of the Software, and to |
| 163 | +permit persons to whom the Software is furnished to do so, subject to |
| 164 | +the following conditions: |
| 165 | +
|
| 166 | +The above copyright notice and this permission notice shall be |
| 167 | +included in all copies or substantial portions of the Software. |
| 168 | +
|
| 169 | +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, |
| 170 | +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF |
| 171 | +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND |
| 172 | +NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE |
| 173 | +LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION |
| 174 | +OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION |
| 175 | +WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
0 commit comments