Remove duplicate lines.
Paste a list, hit Dedupe, copy or download. Runs in your browser. Nothing is uploaded.
Remove duplicate lines with browser-based speed and absolute privacy
This tool provides a local, client-side utility to clean messy data files and lists without transmitting any information over the network. By performing all computations directly in the browser, your data remains fully private on your device. No login or registration is required to use any of the features.
For lists up to 500,000 rows, heavy deduplication and sorting tasks are offloaded to background Web Worker threads. This prevents the browser window from freezing, maintaining an interactive and responsive user experience even when processing large spreadsheets or documents.
Supported formats
Plain text
Paste a list or upload raw text files. The tool compares entries line by line and removes exact matching duplicates.
CSV files
Upload comma-separated lists. Use the advanced selector to choose key columns and deduplicate rows based on specific identifiers.
Excel spreadsheets
Parse .xlsx and .xls workbooks. Key columns can be selected to collapse rows while preserving all other spreadsheet values.
How it works
Paste or drop file
Copy your data list to the text area, or drag and drop a .txt, .csv, or .xlsx file directly into the tool zone.
Select options
Choose simple or advanced mode, set matching criteria like case-insensitivity, trim whitespace, and sort preferences.
Clean and export
Execute the deduplication instantly. Copy the clean unique result, or export it to TXT, CSV, XLSX, MD, or PDF.
Common uses
Email lists
Clean list exports by collapsing duplicate contact addresses.
CRM exports
Remove duplicate customer records before importing spreadsheet datasets.
Keyword research
De-conflict search terms collections for search engine optimization campaigns.
Survey responses
Clean duplicate entries or identical form submissions from exports.
Product SKUs
Ensure inventory sheets contain unique product SKU identifiers.
Questions
Excel's built-in tool works fine for small spreadsheets but stalls on large files and gives you no control over matching logic. dedup.ing handles up to 500,000 rows in the browser, lets you choose case-insensitive or whitespace-tolerant matching, and works on any device without needing Excel installed.
By default, two entries are duplicates if they match character for character. Enable case-insensitive matching and "apple", "Apple", and "APPLE" collapse to one. Enable trim whitespace and "apple " matches "apple". You control the definition — the tool applies whatever combination you set.
No. The CSV parser follows the RFC 4180 standard, so values wrapped in quotes are treated as single fields even if they contain commas, line breaks, or other special characters. If your data is plain text with one item per line, use the plain text input and commas are treated as regular characters.
After deduplication, the result panel shows three numbers: total input rows, unique rows kept, and duplicates removed. If you had 1,000 rows and 40 were duplicates, you see 1,000 in, 960 out, 40 removed. This is useful for gut-checking the result before you copy or download it.
Yes. Enable trim whitespace and the tool strips leading and trailing spaces from every entry before comparing. Blank lines are treated as empty strings and deduplicated the same way — if you have multiple blank lines, they collapse to one. If you want to remove blank lines entirely, enable the remove blanks option before running.