Re: How to remove duplicate words from text

Giganews Newsgroups
Subject: Re: How to remove duplicate words from text
Posted by:  Lasse Reichstein Nielsen (l…
Date: 22 Jul 2003

carlit… (Voetleuce en fênsievry) writes:

> I'm not a JavaScript author myself, but I'm looking for a method to
> remove duplicate words from a piece of text.  This text would
> presumably be pasted into a text box.
> I have, for example, a list of town names, but there are hundreds of
> duplicates, like:
> "Aberdeen Aberdeen Aberdeen Edinburg Edinburg Inverness etc."

So your list is a string, and not, e.g., an array.
We need to be able to delimiter the names, so for now I'll assume that
spaces delimiter town names.

> Can you direct me to a script (or write one, please) that will remove
> the excess names and leave just one instance of each name, like:
> "Aberdeen Edinburg Inverness etc."

function uniqueTowns(towns){
  var arrTowns = towns.split(" ");
  var arrNewTowns = [];
  var seenTowns = {};
  for(var i=0;i<arrTowns.length;i++) {
    if (!seenTowns[arrTowns[i]]) {
  return arrNewTowns.join(" ");
This function splits the argument string "towns" into an array
of strings, then runs through them and adds the string to a new
array the first time it is encountered. This new array is then
joined to a string again and returned.

> [E-mail answers appreciated, but I'll also check the newsgroup.]

Good choice :)

Lasse Reichstein Nielsen  -  l…
  'Faith without judgement merely degrades the spirit divine.'


In response to

How to remove duplicate words from text posted by carlit… (Voetleuce en fênsievry) on 22 Jul 2003