Linux uniq command summary with examples (4:07)
Videos can also be accessed from the YouTube Playlist.
Our thirty-second word, or command to memorize is
uniq from our category
uniq allows you to find repeating
Recall from the previous video, we created a sorted text file with repeating lines of text, well here we'll do something worthwhile with that file.
Before we start, it helps to think of commands as mini programs and
most follow this structure:
command -option(s) argument(s).
uniq command has
12 options, and arguments include the file you'd like
to process and optionally, where you'd like the output.
Like most commands, help is available with double-dash
-c option counts the number of matches
-s allows you to skip a number of
characters on the front end of lines.
So why is
uniq an important command?
Well, we're often faced with messy data sets and
uniq allows you to clean them up
before, say importing into a database or statistical analysis program.
And now you know how to do that.
Okay, the best way to embed this in your memory is by typing in your own terminal window.
Find this on your Mac using a program called Terminal. On Linux use Terminal or Konsole, and currently Microsoft is adding this functionality to Windows.
Here we go, the
cat command from
video 10 shows us our
See, we have repeats. Well, like this
uniq will show us each repeat.
Okay, next, let's count the repeats using
uniq -c video32.txt.
Okay good, two for each, makes sense. Oh and that misspelling (thrid) still bothers me. We'll get that.
And last, what if out front a field made the line unique, but you
wanted to ignore it, like a timestamp field for example. I'll spend
a couple seconds here editing a few things using the program
nano to make the
point, like adding a character that would make it unique. And then
(the file below is before edits)
(the file below is after edits)
Now when I use
uniq video32.txt and
you can see the difference (below).
Now, we could use the
uniq --skip-chars=1 option to skip
the first character.
Okay now you know how to use
And you know the syntax for commands, options and arguments.
One last tip about the
uniq helps us make large data sets
useful, right? So we'll stick with that theme for the next few
Okay, thanks for visiting today. I hope this was a helpful
introduction to the
For an overview of 100 videos, the 8 quizzes, a cheat sheet, the categories and a Q&A section, visit:
If you haven't paired text-based tutorials with our videos it may help you learn faster. Subscribe and check out the YouTube Channel.
sortcommand, hit Back.
cutcommand, click Next.