nlp
Natural-language analysis pack: statistical corpus analysis over text with zero ML dependencies. Lexicon-based sentiment (AFINN-style), RAKE/TF keyword extraction, extractive summarization, n-gram...
nlp — natural-language analysis pack
Statistical / lexicon-based corpus analysis over text. Pure JS, zero npm deps, no ML models — everything is deterministic (frequency counts, an AFINN-style lexicon, stopword profiles, Flesch formulas).
Tools
| Tool | What it does | ||
|---|---|---|---|
sentiment({ text }) | Lexicon-based (AFINN-style, data/afinn.json) with negation flipping → `{ score, comparative, label:'positive'\ | 'negative'\ | 'neutral', positive:[], negative:[] }`. |
keywords({ text, top=10 }) | RAKE-style keyword/keyphrase extraction (candidate phrases scored by word degree/frequency, stopwords excluded) → ranked terms with scores. | ||
summarize({ text, sentences=3 }) | Extractive summary: sentences scored by content-word frequency, top-N returned in original order. | ||
ngrams({ text, n=2, top? }) | n-gram frequency table. | ||
languageDetect({ text }) | Stopword-profile heuristic across en/es/fr/de/it/pt/nl (+ unknown) → { language, confidence, scores }. | ||
readability({ text }) | Full Flesch Reading Ease + Flesch-Kincaid grade, syllable estimate, avg sentence length. | ||
| `tokenize({ text, mode='word'\ | 'sentence' })` | Word or sentence tokenization. | |
wordFrequency({ text, stopwords=true, top=20 }) | Word-frequency table (optionally excluding stopwords). | ||
entities({ text }) | Heuristic entities: capitalized sequences → proper nouns, plus regex for emails / urls / dates / money / numbers, grouped. | ||
stats({ text }) | chars / words / sentences / paragraphs / avgWordLength / uniqueRatio. |
Usage
import nlp from './index.js';
nlp.adapters.sentiment({ text: 'I love this, it is great and wonderful' });
// → { score: 10, comparative: 1.25, label: 'positive', positive: ['love','great','wonderful'], negative: [], tokens: 8 }
nlp.adapters.languageDetect({ text: 'el gato está en la casa y la comida es muy buena' });
// → { language: 'es', confidence: ..., scores: { en, es, fr, de, it, pt, nl } }
nlp.adapters.readability({ text: 'The cat sat on the mat. It was a sunny day.' });
// → { fleschReadingEase: <number>, fleschKincaidGrade: <number>, ... }Every tool takes one args object (text required) and returns a plain JSON-serializable object.
DRY boundary vs domain/a-text
a-text= per-string transforms: case conversion, base64, ciphers, digests, extraction,
a basic readabilityScore, and stemming on a single word/string.
nlp(this pack) = statistical corpus analysis: sentiment, keyword/summary extraction,
language detection, the full readability metrics (Flesch Reading Ease + Flesch-Kincaid grade, where a-text only offers a basic readability score), n-grams, entity/stat roll-ups over a whole document.
Rule of thumb: need a number about the text → nlp; need to reshape a string → a-text.
Data
data/afinn.json— ~230-word AFINN-style sentiment lexicon (scores -5..+5).data/stopwords.json— English stopword list (shared by keywords / summarize / wordFrequency / entities).data/lang-profiles.json— top-function-word profiles for the 7 detected languages.