Word List Duplicate Remover
Clean a one-item-per-line list while preserving the first occurrence.
Free, no sign-up, no ads, and fully browser-based
What this tool does
Paste vocabulary or terms, choose how case and sorting should work, and receive a clean list with removal counts. The cleaner trims leading and trailing whitespace, drops empty lines, and optionally converts a comparison key to lowercase. It keeps the spelling and position of the first occurrence while removing later matches.
How to use it
- 1
Paste one word or item per line.
- 2
Keep Ignore letter case checked or clear it, add Sort ascending or Add numbering if you want, then click Remove duplicates.
- 3
Review the removed count, then copy the clean list or download it as a TXT file.
Normalize each line before checking duplicates
Sorting happens only after deduplication, so the original order remains available. Inflected forms, punctuation, accents, and full-width characters remain distinct because combining them could erase genuinely different study items.
Before importing the result, check whether repeated words represent different meanings. Keep separate entries when the part of speech or definition changes, and label that distinction in the prompt. Cleaning spelling variants before card creation prevents duplicated reviews and makes the deck easier to search. A unique word list is only a source: it still needs an unambiguous question and answer to become effective retrieval practice.
Cleaning runs in this browser and needs no account. Once the list is tidy, add a clear answer to each item and practice retrieving them after a gap.
Method
- Last updated:
- Content reviewed by:
- Tachibana Koichi (Memly CMO and Content Lead)
Frequently asked questions
Can it treat Apple and apple as the same?
Yes. Keep Ignore letter case checked, which is the default. The capitalization from the first occurrence is kept.
Does it preserve the original order?
Yes, unless you enable sorting. Each first occurrence stays in its original relative position.
Does this tool upload my study data?
No. The calculation runs in your browser. Avoid entering private information, and copy, print, or write down any result that you need to keep.
Will it catch near-duplicates such as plurals or typos?
No. It only removes lines that match exactly after extra spaces are trimmed, and after letter case is ignored if you keep that option on. Plurals, other word forms, accents, and full-width characters count as different lines, so skim the cleaned list before using it.