Metacat

Harvester Class Descriptions               Revision 1.1         12/12/03

Class       Description

Harvester

            database, creating a HarvestSiteSchedule object for each entry in

            the HARVEST_SITE_SCHEDULE table. Manages a list of HarvestLog

            entries to keep a permanent record of harvest operations. Provides

            operations to read the harvest log from the database, insert new

            log entries, and write the harvest log to the database.

            Manages site scheduling and other data for a site.

            Corresponds to a single entry in the HARVEST_SITE_SCHEDULE

            table. For a given site, stores its contactEmail address, the

            date of its last harvest, the date of its next harvest (if stored

            explicitly), the document list URL, the LDAP DN, the harvest

            frequency, and the frequency unit (e.g. days, weeks, or months).

            If the date of the next harvest has not been stored explicitly,

            it is derived from the date of the last harvest and the

            harvest frequency. Provides operations to get and parse the site?s

            document list. Provides operations to interact with the

            HARVEST_SITE_SCHEDULE database table. Manages a list a

            HarvestDocument objects, one for each entry in the site's

            document list.

HarvestDocument

            the document (documentURL, documentType, identifier, revision,

            scope). Provides operations to get the document from the site

            and put (insert or update) the document to Metacat. Queries

            Metacat to determine whether Metacat already has the document

            and to determine the highest revision number of the document

            stored in Metacat.

HarvestLog

            Represents a single harvest log entry. For a given Harvest

            operation, records the date, type of operation, message string,

            that involved a harvest document; see HarvestDetailLog below).

            Interacts with the HARVEST_LOG database table, and retrieves

            information about operations from the HARVEST_OPERATION table.

            Stores detailed information about a harvest operation on a document

            when the operation results in an error (e.g. the document could not

            be retrieved from the site, or could not be inserted or updated into

            Metacat). Stores a unique identifier (detailLogID), the identifier

            of its associated HarvestLog object (harvestLogID), the error message,

            and a HarvestDocument object. Interacts with the HARVEST_DETAIL_LOG

            table.

1. Harvester starts up. (Add log entry to record startup operation.)

2. Harvester reads the Harvest registry from the database, creating a

   HARVEST_SITE_SCHEDULE table.

    4. If this site is due to be harvested today:

                    log an exception and continue.)

                12. Insert or update the document to Metacat.

                13. Log the result of inserting or updating the document to

                    Metacat (including any exceptions).

                highest revision of the document in Metacat and provide

                this information in the report to the site contact (15).

        15. Generate and send an email report to the site contact. The report

            scheduled date of the next harvest.

    could be written to a log file instead of emailed. An email message could

    simply contain the location of the log file and a brief summary.)