Installation and usage

Installation and Usage

Installation

To install via NPM:

npm i --save finnlp

Usage

Example:

// import
import * as Fin from "finnlp";

// instantiate new input
let processed = new Fin.Run("This is a some text");

Processing

If you console.log the processed instance variable, you'll get something like this:

{
    raw:"this is some text",
    intercepted:"this is some text",
    sentences:[{
        sentence: "this is some text",
        tokens: ["this","is","some","text"],
        tags: ["DT","VBZ","DT","NN"],
        deps: [
            {
                "label": "NSUBJ",
                "type": "NP",
                "parent": 1
            },
            {
                "label": "ROOT",
                "type": "VP",
                "parent": -1
            },
            {
                "label": "MWE",
                "type": "NP",
                "parent": 3
            },
            {
                "label": "ATTR",
                "type": "NP",
                "parent": 1
            }
        ]
    }]
}

Detection

As we've mentioned earlier, the above processing result might not be very useful in a real world use case.

However, you should consider it as the starting point on which you build your detectors to get useful information out of the passed text.

What you would want to build is what's called detectors those are just functions that should be added to the Run function prototype and utilize the above processing result to get more meaningful results.

For example, a detector may utilize the above object and since the sentence in question have an infinitive verb as it's ROOT, then sentence must be in present tense.

How to write detectors

WARNING The following example illustrate how to write detectors in TypeScript. The same way is applicable in JavaScript but without the ambient declaration (declare module) and without the type declarations.

The following example will detect what's being an object to negative sentiment.

// we should have a dictionary of verbs that
// imply negative sentiment to it's object

const negativeVerbs = [
    "hate",
    "detest",
    "despise",
    "abhor",
    "loathe"
];

import * as Fin from "finnlp";

// first we need to tell typescript that we're adding the function
declare module "finnlp" {
    export interface Run {
        negative: () => Array<Array<string>>;
    }
}

// finally, let's add the function to the prototype
Fin.Run.prototype.negative = function(this:Fin.Run){
    const objects = [];
    this.sentences.forEach(function(sentence, sentenceIndex){
        // find negative verbs index
        const negativeVerbIndex = sentence.tokens.findIndex(function(token){
            return negativeVerbs.indexOf(token)>-1);
        }
        // if we didn't find one, let's return..
        if(negativeVerbIndex === -1) return;
        // otherwise 
        const objectIndex = sentence.deps.findIndex(function(dep){
            return dep.parent === negativeVerbIndex && dep.label === "DOBJ";
        });
        if(objectIndex === -1) return;
        objects.push(sentence.tokens[objectIndex]);
    });
    return objects;
};

Using the above detector:

import * as Fin from "finnlp";
// just by importing the detector file
// we'll be able to use it.
import "./detector";
const instance = new Fin.Run("I hate Jimmy");
console.log(instance.negative());

The above example should give:

["Jimmy"]

Please note that the above example is just a demonstration of how you can write detectors. If you need a negative sentiment detector, there are many things you should consider. In fact, I maintain a sentiment detector extension that you should check out for a more sophisticated example.

Pre & Post Processors

Preprocessors and postprocessors are function that can be hooked in the the main Run function either to intercept the input or modify the result.

The following example will look for "alex" and change it into "Alex".

import * as Fin from "finnlp"

Fin.preProcessors.push(input => input.replace(/\balex\b/g,"Alex"));

An example of postprocessors is a function that converts all the verb POS tag annotations into "VERB".

import * as Fin from "finnlp"
Fin.postProcessors.push(resultObj => {
    result.sentences = result.sentences.map((sentence)=>{
        sentence.tags = sentence.tags.map((tag)=>{
            if(tag.startsWith("V")) return "VERB";
            else return tag;
        });
        return sentence;
    });
    return result;
});

JavaScript & TypeScript

This library is written in TypeScript and compiled to JavaScript, so you can import the library in both and have type safety and auto completion in TypeScript or just use it as any regular node module if you want to.