Statistical Language Model for Intelligent Text Search & Analysis
┌─────────────┐ ┌──────────────┐ ┌─────────────┐
│ Text │ --> │ Language │ --> │ Ranked │
│ Corpus │ │ Model │ │ Sentences │
└─────────────┘ └──────────────┘ └─────────────┘
Multi-metric similarity scoring combining:
- 🔤 TF-IDF - Term frequency analysis
- 📝 Character-level - LCS-based matching
- 🔗 Co-occurrence - Semantic relationships
# 1. Clone & Install
git clone https://github.com/ItzmeJan/NarrowMind-L2.git
cd NarrowMind-L2
# 2. Add your text to input.txt
echo "Your text corpus here..." > input.txt
# 3. Run
node index.js=> your query here
| Feature | Description |
|---|---|
| 🧠 Multi-Metric Scoring | TF-IDF + Character + Co-occurrence |
| 🔍 Semantic Analysis | Word co-occurrence & n-gram patterns |
| ✂️ Text Processing | Custom stemming & filler word filtering |
| 📈 Interactive Shell | Real-time query analysis |
node index.js======================================================================
NarrowMind S2 - Statistical Language Model
======================================================================
=> where did aria walk
📊 QUERY ANALYSIS
• walked → TF: 0.0180 | IDF: 3.0794
• aria → TF: 0.0270 | IDF: 2.7918
🔗 Common Co-occurrences
• aria - co-occurs with 2 query tokens
📈 RANKING RESULTS
1. [Score: 0.2639] ████████████
"Aria walked home late after her evening class"
import { NarrowMindModel } from './model.js';
const model = new NarrowMindModel(textData);
// Rank sentences
const results = model.rankSentences(
"your query",
10, // top N results
0.70, // TF-IDF weight
0.10, // Character weight
false, // filter fillers
0.20, // Co-occurrence weight
'jaccard' // method
);TF-IDF: ████████████████████ 70%
Character: ████ 10%
Co-occurrence: ██████ 20%
- Jaccard - Set-based similarity (intersection/union)
- PMI - Pointwise Mutual Information
NarrowMind-L2/
├── 🚀 index.js # Interactive shell
├── 🧠 model.js # Core language model
├── ✂️ stem.js # Stemming utility
├── 📋 fillers.json # Stop words
└── 📄 input.txt # Your corpus
// Main methods
model.rankSentences(query, topN, ...weights)
model.getTokenStats(token)
model.getTopCoOccurrences(word, topN)
model.calculateCoOccurrenceScore(word1, word2, method)1. Text Processing
└─> Tokenize → Stem → Filter
2. Feature Extraction
└─> TF-IDF vectors
└─> Co-occurrence matrix
└─> Character features
3. Similarity Calculation
└─> Multi-metric scoring
4. Ranking
└─> Weighted combination → Results
ISC
Built with ❤️ for intelligent text understanding