This article is the second of a two-part series. Before reading this article, please take a look at Part 1, which will introduce you to the autotag.pl application and the rationale for the various modules used in it. We pick up directly where we left off last time.

Preparing for the main loop

The main loop of autotag.pl will identify and tag music. In order to do that, some preparations are in order. First, I create the FreeDB search object using the WebService::FreeDB module.

The search object inherits the DEBUG setting from the autotag.pl configuration, so that the user does not have to remember a FREEDB_DEBUG setting for autotag.pl. The host is provided by the autotag.pl configuration as well.

my $cddb = WebService::FreeDB->new(DEBUG => $config->DEBUG(),
                                   HOST => $config->FREEDB_HOST);

die "Could not initialize the FreeDB service"
 unless defined $cddb;

Next, I create some hashes: %discs, %olddiscinfo, and %disc_counts. Also, the @common list is created. All these variables will be useful in the course of the main loop. Note that every search result in FreeDB is identified by a unique ID, and that's all I'm storing until much later in the program.

I step through all the user-provided command-line searching switches, such as -artist and -album (using the keys of the %freedb_searches hash instead of listing the switches manually). The get() method of AppConfig can be used to get the value of an individual parameter; because the parameters of interest are always array references, I automatically de-reference them. If no searching switches are provided, I enter the interactive mode where the user can provide search criteria interactively.

my $search_count = 0;
foreach my $search (keys %freedb_searches)
{
 $search_count += scalar @{$config->get($search)};
}

print "Search count is $search_count\n"
 if $config->DEBUG();

It may seem like a little thing, but using the %freedb_searches hash to get the list of searching switches makes the code shorter and more maintainable. You should always look for such ways to eliminate repetition of constants and string literals in your programs.

Armed with my knowledge of the search count, I may need to enter the interactive query mode (the user is asked if he wants to do so, and if not, the program quits gracefully). In the interactive mode, the user starts with some primitive guesses of the artist and track name using the appropriately named guess_artist_and_track() function. These guesses are made across all the files given to autotag.pl, and guesses accumulate in the %guessed hash, using sub-hashes so that repeated finds of an artist's name in all the MP3 files will only generate one guess. The user is then asked whether those guesses are good for each search. For instance, when I ask for artist searches, the artist name guesses are offered to the user first.

Thus, in due order I come to the interactive query for searches. For each search in the %freedb_searches hash, the user adds more searches. If he just presses Enter, the read_line() function will return an empty string, and such input is taken as an indication that the user wants to go on.

while (my $data =
 read_line("Add a search by $search or ENTER to go on: ", ''))
{
 last unless defined $data && length $data;
 push @{$config->get($search)}, $data;
}

Again, I use the get() method of AppConfig to get an array reference to the configuration list and push the user-provided data into that array.

Back to top

Initial searches

The FreeDB searches are done with the search criteria given earlier, either interactively or through command-line switches. Again, I use the %freedb_searches hash to make a list of searches. For each search, I specify the search words given by the user and add the words given to the -all switch. Note that each "word" can contain spaces; the words are single search units as far as autotag.pl and FreeDB are concerned.

foreach my $search (keys %freedb_searches)
{
 # @keywords will contain all the keywords (e.g. -artist "Pink Floyd")
 my @keywords = @{$config->get($search)};
 # we join in the -all keywords for every search
 push @keywords, @{$config->get(SEARCH_ALL)};

 print "Asked for keywords @keywords, search $search\n"
  if $config->DEBUG();
 # remember the searches and keywords done
 push @{$freedb_searches{$search}->{keywords}}, @keywords;

 # do the search
 foreach my $keyword (@keywords)
 {
  print "Searching with keyword $keyword, search $search\n"
   if $config->DEBUG();
  my %found_discs = $cddb->getdiscs($keyword, [$search]);

  if ($config->OR())                    # any search with OR
  {
   push @common, keys %found_discs;
  }
  elsif (scalar @common)                # second or more search without OR
  {
   my @new = keys %found_discs;
   my $lc = List::Compare->new(\@common, \@new);
   @common = $lc->get_intersection();
  }
  else                                  # first search without OR
  {
   @common = keys %found_discs
  }

  foreach my $disc (keys %found_discs)
  {
   $discs{$disc} = $found_discs{$disc};
   $disc_counts{$disc}++;       # we'll use this to remove matches later
  }
 }                              # foreach @keywords
}                               # foreach keys %freedb_searches

Once results are returned from the FreeDB database, they are put in the %found_disks hash. This hash is created for every keyword, so old results don't show up. When the -or switch is specified by the user, I simply add the results to the other results. Otherwise, the @common array is used to see what results are in common with previous results (the AND mode is implicit, meaning that by default search results must match all keywords requested). The List::Compare module is used to make an intersection of two lists. Doing it manually is possible and not too hard, but why spend the time and effort when a tested and probably much faster implementation already exists?

Note again that all search results are just string IDs that correspond to an album in the FreeDB database. Thus, I can use them as hash keys, lists of strings to be searched, etc.

Finally, I add the filtered results to the %discs hash, and increment the %disc_counts entry for each found album. The disk counts will be used later. Incidentally, I use "disc" instead of "disk" in variable names because that's what FreeDB uses.

When the results are all in place, I delete the ones that are not in @common. Recall that with -or, the @common array contains all the results.

foreach my $disc (keys %discs)
{
 next if grep { $_ eq $disc} @common;
 print "Deleting search result $disc, it was not in all searches\n"
  if $config->DEBUG();
 delete $discs{$disc};
}

unless (scalar keys %discs)
{
 print "The search you requested returned no discs, sorry.  Exiting.\n";
 exit;
}

The loop through the keys of %discs is simple, using grep() to see if a disc is in @common. I could have used a hash to optimize this loop further, but frankly I don't think it would have made a difference at all unless the user was finding hundreds of thousands of albums.

If no albums were left in %discs, I'm done and make a graceful exit with a message.

Back to top

Select the disks of interest

I now have a list of albums that correspond to what the user thinks he wants. This is usually a list bigger than what the user really wants, so now I give the user a chance to select only the albums of interest. This is also a chance for the user to quit if the albums found were not those they wanted. The -accept_all switch to autotag.pl will bypass the selection menu and let the user operate on all the found albums. The final results, regardless of how I get there, will reside in the @selecteddiscs list.

my @selecteddiscs;
if ($config->ACCEPT_ALL())
{
 @selecteddiscs = keys %discs;
}
else
{
 print "Enter the albums of interest for files [@ARGV]\n";
 @selecteddiscs = $cddb->ask4discurls(\%discs);
}

unless (scalar @selecteddiscs)
{
 print "You selected no albums, exiting...\n";
 exit 0;
}

%olddiscinfo = %discs;                  # save the old data for ask2discurls
%discs = ();                            # clear the search results

# populate %discs with full search results
foreach my $disc (@selecteddiscs)
{
 my %discinfo = $cddb->getdiscinfo($disc);
 $discs{$disc} = \%discinfo;
}

if ($config->DUMP())
{
 print Dumper \%discs;
 exit 0;
}

The WebService::FreeDB module has an ask4discurls() function that was nice, because I would have had to write it otherwise. It prints a list of albums and lets the user pick the desired ones.

Now that the final list of albums is in @selecteddiscs, %discs gets only the albums whose ID is there. The getdiscinfo() function fills out the album entry with track and album information. It is a slow function, which is why I use it only now that a small number of albums are left.

Back to top

At last, the main loop

For each file given on the command line (in @ARGV), I get the ID3 tag and create it if necessary. For files that can't abide an ID3 tag for various reasons, I print an informative message and skip them. There is a difference here between files that are simply not in existence -- for instance, a directory name given as an MP3 file -- and files that are not accessible, such as a file with insufficient permissions.

foreach my $file (@ARGV)
{
 my $tag = get_tag($file, 1);

 unless (defined $tag)
 {
  if (-r $file && -f $file)
  {
   print "Could not get a tag from file $file, skipping";
  }
  else
  {
   print "Nonexistent file $file, skipping";
  }
  next;
 }
... the rest of this loop is explained later ...
}

The %discs_of_interest hash is a copy of %discs. I tried using modules for approximate (fuzzy) string matching to narrow the selection of disks that are interesting. For instance, I tried matching the album name fuzzily (with 50% to 90% precision), and no setting worked well. The problem is that some words like "love" are very common, while other words like "U2" are too short. There may be a good algorithm to narrow choices, and I've left the %discs_of_interest hash in place in case that algorithm comes about, but it seems from my personal experience that the best thing to do is let the human brain pick an option in 0.01 seconds. Sometimes, trying to solve a problem with a computer is simply not as efficient as a few million years of evolution.

Now comes the while(1) loop. This is an endless loop that reflects how the user often will cycle between choices until he's made only one. I could have written this loop with variable controls, but using next() and last() in an endless loop seemed more natural.

I get a single album with the following loop:

my @chosen = ();

# do the following unless only one album is selected
if (1 == scalar keys %discs_of_interest)
{
 @chosen = (keys %discs_of_interest)[0];
}
else
{
 # get the ask4discurls special format back from %olddiscinfo
 my %ask4discurls_special_hash;

 foreach (keys %discs)
 {
  $ask4discurls_special_hash{$_} = $olddiscinfo{$_};
 }

 do
 {
  print_tag_info($file, $tag);
  print "Choose a single album or none (to skip file) from the current list\n";
  @chosen = $cddb->ask4discurls(\%ask4discurls_special_hash);
 } while (scalar @chosen > 1);
};

last if scalar @chosen == 0;
next if scalar @chosen != 1;
my $disc = $discs{$chosen[0]};

If only one album exists anyhow, I just take it. Otherwise, I get the lists of disks of interest using the ask4discurls() function. Note that I print out the file tag info before that question with print_tag_info(), so the user is reminded of the file's information. Users are naturally forgetful, so every shortcut and reminder the programmer can offer them is appreciated. Users are also imperfect, so I don't assume that just because I told them to pick one album, that they did so. With a GUI, similar selection rules can be enforced on listboxes -- but in the text interface autotag.pl has, input validation has to be done this way. Actually, that's not entirely true: There are some CPAN modules that can help here, but the scale and scope of autotag.pl did not seem to merit a text-mode UI framework.

If no album was chosen, I skip to the next file.

Now, $disc contains the album that is specifically applicable to the current file the user is examining.

my $track_number_guess = guess_track_number($file, $tag);

my $tracks = $disc->{trackinfo};
my $track_number;
do
{
 # ask the user for the track number, while trying to be helpful
 print_tag_info($file, $tag, "Old tag");
 $cddb->outstd($disc);
 $track_number =
  read_line(
   sprintf(
    'Choose a track number 1 - %d, 0 to quit, -1 to select another album: ',
    scalar @$tracks),
   $track_number_guess);
} while (not defined $track_number ||
  	     $track_number < -1 ||
	     $track_number > scalar @$tracks);

# cycle to the album selection again if the user wants to select another album
next if $track_number == -1;

Another endless loop: Until the user has picked a suitable track number, I simply can't go on. The track number is like Moby Dick to autotag.pl's Ahab. It must be found, or else all this work is meaningless. I print out the tag info again to remind the user what it is we're talking about, then print out the disk information with the outstd() function from WebService::FreeDB.

The default track number the user can enter is guessed from the file name or from the previously existing track. This is always just a suggestion, but if the user wants to accept it he can just hit Enter. Now that's service.

When the track number is found, and it's good, the tagging of the MP3 file is done:

# if the user selected a track...
if ($track_number > 0)
{
 my $new_tag = make_tag_from_freedb($disc, $track_number);
 print_tag_info($file, $new_tag, "New tag info") if defined $new_tag;
 # do this if the new tag was created, DRYRUN was not specified, and the
 # user says YES
 if ($new_tag
     && !$config->DRYRUN()
     && read_yes_no(
"Apply new tag (you'll get a chance to modify it)?", 1))
 {
  my $modify_tags = read_yes_no("Modify tag elements?", 0);
  # copy each new element (but don't overwrite valid old ones)
  foreach my $element (keys %$new_tag)
  {
   my $old_tag_element = $tag->{$element} || '';

   if ($modify_tags)
   {
    # the user can press Up Arrow to get the old tag element
    $term->addhistory($old_tag_element);
    $new_tag->{$element} =
     read_line("New value of $element (was '$old_tag_element'): ",
	 $new_tag->{$element});

    # put the artist and album $new_tag changes back in $disc so the
    # next file can also use them

    if (exists $info2freedb{$element})
    {
     $disc->{$info2freedb{$element}} = $new_tag->{$element};
    }
   }

   $tag->{$element} = $new_tag->{$element};
  }
  set_tag ($file, $tag);
 }				# if apply_new_tag...
}				# if $track_number > 0
last;

First of all, recall I'm in an endless while(1) loop here. The last() at the end means that if I got this far, I should exit the loop.

I start with the make_tag_from_freedb() function to make an ID3 tag from the FreeDB tag. This is hidden into a function because it's not a straightforward mapping.

Given the new tag and a final "yes" from the user, I proceed to the tagging of the file. The user now has a chance to modify each individual tag element. Each choice there is stored in the input object history (read the Term::Readline documentation for details). That way, the user can press Up Arrow and grab the old inputs instead of having to retype them. Finally, and this really makes the user's life easier when tagging multiple files, I store the modified information that should persist back in the %$disc hash. Thus, if a user modifies the artist of an album, on the next file the modified name will now be the default name.

When all is said and done, the tag is set with set_tag().

Back to top

Usage

I have used autotag.pl for over a month to catalogue and retag my MP3 collection (it used to have only ID3 1.1 tags). I'd like to think that using autotag.pl is easy, but it may be that I've become so used to its inadequacies that I don't notice them anymore. Don't be afraid to suggest improvements to autotag.pl, especially to its command-line switches and text UI.

If you have some MP3s with "good" tags and just want to rename them to a common format, use the -ro option.

If you have some MP3s with incorrect track numbers, use the -g option to get the track number quickly. The track number is just a guess so you'll get a chance to confirm each guess.

If you have some MP3s with incorrect comments, use the -sc option to strip the comments. It's also possible with mass tagging by explicitly using "COMM=" but -sc is more convenient.

If you have some MP3s that all come from a single album and you know the tags you want to set, use the -m option and set the tag entries the way you want them. The -help option will print out the list of supported tag entries (frames in ID3).

Back to top

Conclusion

Writing autotag.pl was grueling but fun. I used fuzzy string matching, FreeDB searches, ID3 versions 1 and 2, and lots of text-mode user interactions. It all came together in an application that I tested thoroughly over the course of a month.

The most difficult part of writing autotag.pl was picking the right modules for the job. In the first article in this two-part series, I mentioned all the modules I rejected for autotag.pl. I did not reject them by simply reading the documentation; the documentation is not always right, and sometimes it is simply wishful thinking. The best test for each module was to write for it, and see how well it worked. Thus, every choice I made for autotag.pl is battle-tested, and I hope you will find those choices alone useful if you ever do ID3-related Perl work.

I plan to offer autotag.pl for download permanently and to keep improving it, so feel free to suggest features or improvements for it. Bugs spotted will be gleefully terminated, so do let me know if you notice any bugs.

Resources

• Get an introduction to autotag.pl and links to all of the Perl modules it uses in "Fun with MP3 and Perl, Part 1" (developerWorks, September 2003).
  
  
• Read all of Ted's Perl articles in the "Cultured Perl" series on developerWorks.
  
  
• Download the autotag application (rename the file to autotag.pl in order to run it).
  
  
• Download the plug-in modules used by autotag.pl from the CPAN module archive.
  
  
• You'll find resources on the many different audio formats including MIDI, MP3, and Ogg Vorbis at the Open Directory.
  
  
• In the article "ThinkingXML: Manage metadata with MusicBrainz" (developerWorks, December 2002), XML guru Uche Ogbuji discusses the use of the MusicBrainz service for managing MP3 music files.
  
  
• Playing around with audio? Learn more about voice-enabling your apps in the articles "Introducing XHTML + Voice -- IBM's proposal to the W3C on developing multimodal UIs" (developerWorks, August 2003) and "Multimodal applications" (developerWorks, July 2002).
  
  
• You'll also want to take a look at IBM alphaWorks' Voice Toolkit Preview.
  
  
• Or try Music Sketcher, a graphical composition tool from the multimedia gurus at IBM Research.
  
  
• Find more resources for Linux developers in the developerWorks Linux section.
  
  
• Browse for books on these and other technical topics.
  
  

About the author

Teodor Zlatanov graduated with an M.S. in computer engineering from Boston University in 1999. He has worked as a programmer since 1992, using Perl, Java, C, and C++. His interests are in open source work on text parsing, three-tier client-server database architectures, UNIX system administration, CORBA, and project management. Contact Teodor at tzz@bu.edu.

Rate this article

Comments

Back to top

Table of contents

• Preparing for the main loop
• Initial searches
• Select the disks of interest
• At last, the main loop
• Usage
• Conclusion
• Resources
• About the author
• Comments