MyLibrary Extract Processing

Step 1 : Authentication

(See Patron Authentication Database Processing)

Step 2 : Extract

The raw extract is created on libdb.syr.edu. The "Smylib" shell script was created by Augie Teska and resides in the "/m1/voyager/syrdb/local/" directory. This program generates from the catalog a subset of records based on the criteria of the records being included in the "elink_index" report and the records containing a ANY occurrance the word "MyLibrary". This subset of the catalog is saved in blob format in the file "mylib.out" in the "/data3/mylib/" directory.

• This file is FTP down to libportal.syr.edu and the results are processed. The program "processExtracts.pl" resides in "/export/home/mdesalvo/perl/mylibrary" directory and cron'd. The program processes the extract and checks for the standard 58 set of disciplines. If any of the data from the extract does not match those 58 disciplines, the processing is stopped. If all records contain a valid discipline then the Databases and Ejournals are processed from the extract. The program compares the existing MyLibrary Databases and Ejournals to the extract and the appropriate removals and additions occur. Note, deletions and additions of MyLibrary Databases and Ejournals are recorded in a single MyLibrary database for display on the "News" page of MyLibrary.

---

Smylib (excerpt)

#!/bin/csh
#  voyager extract for MyLibrary database
#   written:   3/2002
#   author:    A. L. Teska
#

# extract data from oracle, write to
sqlplus -S syrdb/Lsyrdb  @mylib -silent > mylib.extractlog

# run bibmerge perl pgm to merge blobs and filter mylib data
#   output file is: /data3/mylib/mylib.out
/m1/voyager/syrdb/local/bibmerge > mylib.bibmergelog

---

mylib.sql (excerpt)

remark  extract bib data for mylibrary project
remark  written 2/19/02 by A.L. Teska
spool /m1/voyager/syrdb/local/sqlplus/elinkbib.out
set colsep '%%'
select Bib_data.BIB_ID || '%%' || Bib_data.SEQNUM || '%%' || translate( Bib_data
.RECORD_SEGMENT, chr(10), ' ') || '$$'
 from SYRDB.Elink_index, SYRDB.Bib_data, SYRDB.Bib_master
 WHERE (Elink_Index.Record_Id=Bib_Data.Bib_Id)
  AND (Elink_Index.Record_Id=Bib_master.Bib_id)
  AND (Elink_Index.Record_Type='B' AND Elink_Index.Seqnum=1 AND Bib_Master.Suppress_In_Opac='N' )
 ORDER BY Bib_data.BIB_ID;
spool off
#   exit

---

bibmerge (excerpt)

#!/usr/local/bin/perl -w
# perl program to merge voyager blobs extracted for MyLibrary db
#  written 02/21/02 by A.L. Teska
#

# open input file (voyager extract)
#  first line begins with bib id, successive lines are strings of text
#  bib id may repeat again, followed by more text

open(BIB,"/data3/mylib/mylib.out") || die "cannot open output file - data for mylib database\n";

# process next record in BIB input file
#  if record contains %% delimiter, then it is a header line with bibid
#    otherwise it is text to be appended to previous bibstring

    ###############################################################
    #               main processing loop                          #
    ###############################################################

while (($line=)) {

### print "Input = $line\n";
   $recs_in++;
   chomp($line);

   # parse $line
   $line =~ m"(\d*)\%\%(\d*)\%\%(.*)\$\$";
   $curr_bib_id = $1;
   $curr_segnum = $2;
   $curr_bibstring = $3;

 ###  print $1,"//", $2,"//", $3, "\n";

      # if bib id has changed, then process previous record

      if ($curr_bib_id == $prev_bib_id)
         { # $line is continuation of prev bib record

  ### print "Bib id has not changed \n";

         # append to prev bibstring
         $prev_bibstring = $prev_bibstring . $curr_bibstring;

  ### print "Prev bibstring= \n", $prev_bibstring, "\n";

         }

      else { # $line is start of a new bib record

         # fully process previous bib record

  ### print "About to look for MyLib,  prev_bibstring = $prev_bibstring\n";

         # decide whether to output previous bib record
         if ($prev_bibstring =~ /MyLibrary/ )
              {
              $recs_out++;
              print MYLIB $prev_bib_id,"%%",$prev_bibstring;
 ###  print " MYLIB FOUND IN ",$prev_bib_id," ...";
              }
         else { # MyLib not found in prev bibstring
              # bypass prev bib rec - do not write to MYLIB file
  ###     print "...$prev_bib_id skipped...";
              }

         # initialize for next record after processing prev rec
         $prev_bib_id = $curr_bib_id;
         $prev_bibstring = $curr_bibstring;

         } # $line is start of new bib record

  } # end of main loop

---