uniq : a cool nix filter
Atul.Singh 270002HUVT Visits (1814)
Yesterday, I was sitting on my desk and doing usual boaring task . Then I got a seq file containing more than 3 millions records and 5 column. What I have to do is fetch the duplicates records based on first column. It's quite easy in Nix environment with out taking help of any tool.
So i thought, I should share this to you also. Here It comes.....
Be warned that it is a bad idea to use uniq or any other tool to remove duplicate lines from files containing financial or other important data. In such cases, a duplicate line almost always means another transaction for the same amount, and removing it would cause a lot of trouble for the accounting department. Do not do it! Whatever you want to do, please keep a copy of original.
For more detail, please visit.........