API Reference

Complete reference for all 75+ functions and pipeline utilities in the nlp_text_preprocessing library. Functions are logically organized into 6 core capability areas — each is fully null-safe and ready for production Pandas .apply() pipelines.

Null Safety Guarantee

All functions accept None or np.nan values gracefully, returning empty strings or zeros instead of throwing exceptions. This makes it completely safe to use .apply() on messy pandas datasets.

Interactive Code Examples

Click on any function name in the reference tables below to open an interactive code popup with a copyable Python usage example and live output demonstration.

1. General Feature Extraction & Statistics

Extract statistical metrics, word counts, sentence lengths, readability metrics, and text density features.

Function Parameters Return Type Description
x: str int Calculates total whitespace-separated word count in input text string.
x: str int Counts total characters in input text, excluding whitespace characters.
x: str float Computes average word length in characters. Returns 0.0 for empty text.
x: str int Counts total English stop words in input text (case-insensitive).
x: str int Returns count of social media hashtags (#tag) in text.
x: str int Returns count of social media user mentions (@user) in text.
x: str int Returns total count of standalone numeric tokens in text.
x: str int Returns count of words written entirely in uppercase.
x: str int Counts total sentences using sentence terminal punctuation (. ! ?).
x: str float Calculates average sentence length in words per sentence.
x: str float Returns proportion of punctuation characters relative to total text length.
x: str float Returns proportion of uppercase words relative to total word count.
x: str float Calculates vocabulary richness as the ratio of unique words to total words.
x: str float Calculates Flesch Reading Ease score to measure text readability.
x: str dict Returns a comprehensive dictionary containing all summary feature statistics.
df: DataFrame, column: str DataFrame Computes word_count, char_count, avg_word_len, sentence_count, and stop_words_count in a single pass over a DataFrame column.

2. Text Cleaning & Normalization

Sanitize noise, remove URLs/emails/HTML, normalize emojis, mask PII, and standardize punctuation.

Function Parameters Return Type Description
x: str str Converts input text to lowercase safely without crashing on null values.
x: str str Expands English contractions (e.g. "don't" -> "do not", "I'm" -> "I am").
x: str str Strips email addresses matching standard RFC 5322 patterns.
x: str int Returns total count of email addresses present in input text.
x: str int Returns total count of HTTP/HTTPS/FTP web URLs present in input text.
x: str str Removes HTTP, HTTPS, FTP links and www web domain URLs.
x: str int Returns count of retweet markers (RT @user) in social media text.
x: str str Strips retweet markers (RT @user) from social media text.
x: str str Strips HTML, XML tags, and inline markup using BeautifulSoup parser.
x: str str Normalizes unicode accented characters (NFKD decomposition -> ASCII).
x: str str Removes user mentions (@user) from social media text.
x: str str Removes special characters and punctuation symbols from input text.
x: str str Truncates character repetitions beyond 2 consecutive occurrences (e.g. "coooool" -> "cool").
x: str str Removes English stop words from input text (case-insensitive).
x: str str Strips all emoji symbols and unicode pictographs from input text.
x: str list Extracts a list of emoji characters present in input text.
x: str str Converts emojis in input text to descriptive word representations (e.g. "🔥" -> "fire").
x: str str Strips phone numbers in standard international or domestic formats.
x: str int Counts phone numbers present in input text.
x: str str Redacts emails, phone numbers, and card/ID numbers with [REDACTED].
x: str str Collapses consecutive spaces, tabs, and newlines into a single space.
x: str str Splits camelCase or PascalCase hashtags into separate words (#MachineLearning -> Machine Learning).
x: str str Replaces smart quotes (“ ” ‘ ’), em-dashes (—), and ellipsis (…) with ASCII equivalents.
x: str str Strips all numeric digits from input text.
x: str str Strips common currency symbols ($, €, ₹, £, ¥) from text.
x: str str Canonical combination of lowercasing and extra whitespace collapse.
text: str str Runs the default end-to-end cleaning pipeline: lowercase, expand contractions, remove URLs/emails/HTML, remove special characters & extra spaces.

3. Linguistic Processing, Semantics & Grammar

spaCy and NLTK/TextBlob powered lemmatization, NER, dependency parsing, POS tagging, and lexical lookup.

Function Parameters Return Type Description
x: str str Lemmatizes verbs and nouns into base forms using spaCy while preserving word order.
x: str str Lemmatizes all tokens in input text to dictionary lemmas using spaCy.
x: str list[tuple] Extracts Named Entities (PERSON, ORG, GPE, DATE, etc.) as (entity, label) tuples.
x: str list[tuple] Returns Part-Of-Speech (POS) tags for each token in the text.
x: str list[dict] Extracts grammatical dependency parse tree tokens, dependency relations, and head tokens.
x: str str Detects ISO language code (e.g. "en", "es", "fr") of text using TextBlob.
x: str, top_n: int list[tuple] Extracts top_n keyword tokens based on term frequency.
x1: str, x2: str float Calculates Jaccard token similarity index between two texts (0.0 to 1.0).
word: str list[str] Retrieves synonym list for target word via NLTK WordNet synsets.
word: str list[str] Retrieves antonym list for target word via NLTK WordNet synsets.
x: str list[dict] Extracts noun phrase chunks with start/end character offsets using spaCy.
x: str, save_path: str, show: bool WordCloud Generates, displays, or saves a WordCloud visualization object for input text.
x: str str Corrects spelling errors in text using TextBlob probabilistic spellchecker.
x: str list[str] Extracts noun phrase strings from text using TextBlob.
x: str, n: int list[tuple] Generates word N-Grams (bigrams, trigrams, etc.) from input text.
x: str str Converts plural nouns to singular form.
x: str str Converts singular nouns to plural form.

4. Safety, Moderation & Sentiment Analysis

Profanity detection, toxicity scoring, spam signal detection, sentiment classification, and emotion breakdown.

Function Parameters Return Type Description
x: str bool Returns True if input text contains profanity or explicit words.
x: str str Censors profanity words with asterisks (****).
x: str float Calculates baseline toxicity score (0.0 to 1.0) based on insult/hate keywords.
x: str dict Detects spam signals (excessive caps, punctuation, URL density).
x: str dict Performs sentiment classification using TextBlob NaiveBayesAnalyzer.
x: str dict Returns pattern-based sentiment polarity (-1.0 to 1.0) and subjectivity (0.0 to 1.0).
x: str dict Calculates lexicon-based emotion counts (joy, anger, sadness, fear).

5. Pipeline Engine & Custom Workflows

Chain transformations, build custom execution pipelines, process pandas DataFrames, and generate diff reports.

Function Parameters Return Type Description
steps: list Pipeline Pipeline engine class for chaining and executing sequential text cleaning steps.
step: callable | str Pipeline Appends a step function or registered step name to the pipeline instance.
text: str str Runs all registered pipeline steps sequentially on a single text string.
texts: list[str] list[str] Runs pipeline transformation across a list of text strings.
series: pandas.Series Series Applies pipeline transformation across a pandas Series.
None Pipeline Factory classmethod returning a pre-configured 7-step default cleaning pipeline.
name: str decorator Decorator to register a custom python function into the global pipeline registry.
before_list: list, after_list: list dict Generates summary diff report comparing text before and after pipeline execution.

6. Environment & Dependency Management

Manage spaCy language models and download required NLTK corpora automatically.

Function Parameters Return Type Description
model_name: str Language Lazy-loads and caches spaCy language model (downloads automatically if missing).
None None Downloads required NLTK corpus and model packages (punkt, stopwords, wordnet, etc.).