com.articulate.sigma.nlp

Class TFIDF

java.lang.Object

com.articulate.sigma.nlp.TFIDF

public class TFIDF
extends java.lang.Object

Field Summary

Fields

Modifier and Type	Field and Description
java.util.ArrayList<java.lang.String>	lines

Constructor Summary

Constructors

Constructor and Description
TFIDF(java.util.List<java.lang.String> documents, java.lang.String stopwordsFilename)
TFIDF(java.lang.String stopwordsFilename)
TFIDF(java.lang.String stopwordsFilename, boolean resource)
TFIDF(java.lang.String filename, java.lang.String stopwordsFilename)

Method Summary

Modifier and Type	Method and Description
void	addInput(java.lang.String input) add a new document to the set
protected void	calcDFs()
static void	main(java.lang.String[] args) Run a series of tests containing a filename, a query and an expected answer.
java.util.ArrayList<java.lang.String>	matchBestInput(java.lang.String input)
java.lang.String	matchInput(java.lang.String input)
java.util.List<java.lang.String>	matchInput(java.lang.String input, int n)
protected java.util.TreeMap<java.lang.Float,java.util.ArrayList<java.lang.Integer>>	matchInputFull(java.lang.String input)
void	newLine(java.lang.String line)
void	prepare(java.util.List<java.lang.String> documents, java.lang.String stopwordsFilename)
protected void	prepareLine(java.lang.String line)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

lines

public java.util.ArrayList<java.lang.String> lines

Constructor Detail

TFIDF

public TFIDF(java.lang.String stopwordsFilename)

TFIDF

public TFIDF(java.lang.String stopwordsFilename,
             boolean resource)

TFIDF

public TFIDF(java.util.List<java.lang.String> documents,
             java.lang.String stopwordsFilename)

TFIDF

public TFIDF(java.lang.String filename,
             java.lang.String stopwordsFilename)

Method Detail

prepare

public void prepare(java.util.List<java.lang.String> documents,
                    java.lang.String stopwordsFilename)

newLine

public void newLine(java.lang.String line)

prepareLine

protected void prepareLine(java.lang.String line)

calcDFs

protected void calcDFs()

addInput

public void addInput(java.lang.String input)

add a new document to the set

matchBestInput

public java.util.ArrayList<java.lang.String> matchBestInput(java.lang.String input)

Returns:

a list of matches ranked by relevance to the input. If there is a cluster of top matches, return all elements of the cluster. If no answer has a good match, return "I don't know". Iterate the number of clusters until the top cluster is no more than 3.

matchInput

public java.lang.String matchInput(java.lang.String input)

matchInputFull

protected java.util.TreeMap<java.lang.Float,java.util.ArrayList<java.lang.Integer>> matchInputFull(java.lang.String input)

Returns:

a list of matches ranked by relevance to the input.

matchInput

public java.util.List<java.lang.String> matchInput(java.lang.String input,
                                                   int n)

Returns:

a list of strings which are the top n best guess matches to the input. If global variable alternating is set to true, the return the next line in the input file, which is therefore treated like a dialog in which the best response to a given input is the line after the line in the dialog that matches. If there's more than one reasonable response, pick a random one.

main

public static void main(java.lang.String[] args)

Run a series of tests containing a filename, a query and an expected answer.