Remove Duplicate Lines is a free tool that deletes repeated lines from a list and keeps the first copy of each, in the original order. It is useful for email lists, keywords, names, IDs and log files. Your list is processed in your browser and never uploaded.
How to remove duplicates
- Paste your list into the left box, one item per line, or open a
.txtor.csvfile. - The list without duplicates appears on the right, with the number of lines removed underneath.
- Adjust the options if needed, then copy or download the result.
Options explained
- Match upper/lower case exactly: off by default, so “[email protected]” and “[email protected]” count as the same. Turn it on when case matters, such as for passwords or codes.
- Ignore spaces at the ends: treats “apple” and “apple ” as duplicates. On by default, because stray spaces are the most common reason duplicates slip through.
- Keep blank lines: leaves empty lines where they are, to preserve the layout of text with paragraphs.
Common uses
- Cleaning a mailing list before importing it into an email service.
- Merging keyword lists from several sources.
- Finding the unique values in a spreadsheet column: copy the column, paste it here, and paste the result back.
To put the result in alphabetical order as well, use Sort Lines. If you see duplicates that weren’t removed, run the list through Remove Extra Spacesto clear invisible characters first.
Frequently asked questions
Which copy is kept?
The first time a line appears is kept and later copies are removed, so the list stays in its original order. To also sort it, use Sort Lines afterwards.
Are “Apple” and “apple” duplicates?
By default, yes: differences in upper and lower case are ignored. Tick “Match upper/lower case exactly” to treat them as different.
Why weren’t two identical-looking lines removed?
One of them may contain an invisible character such as a non-breaking space. Run the text through Remove Extra Spaces first, which fixes invisible characters.
Is there a size limit?
No fixed limit. Lists with hundreds of thousands of lines work on a typical computer.
Last updated