Harvest User's Manual

Darren R. Hardy, Michael F. Schwartz, Duane Wessels, Kang-Jin Lee

1. Introduction to Harvest

• 1.1 Copyright
• 1.2 Online Harvest Resources

2. Subsystem Overview

• 2.1 Distributing the Gathering and Brokering Processes

3. Installing the Harvest Software

• 3.1 Requirements for Harvest Servers
• 3.2 Requirements for Harvest Users
• 3.3 Retrieving and Installing the Harvest Software
• 3.4 Building the Source Distribution
• 3.5 Additional installation for the Harvest Broker
• 3.6 Upgrading versions of the Harvest software
• 3.7 Starting up the system: RunHarvest and related commands
• 3.8 Harvest team contact information

4. The Gatherer

• 4.1 Overview
• 4.2 Basic setup
• 4.3 RootNode specifications
• 4.4 Generating LeafNode/RootNode URLs from a program
• 4.5 Extracting data for indexing: The Essence summarizing subsystem
• 4.6 Post-Summarizing: Rule-based tuning of object summaries
• 4.7 Gatherer administration
• 4.8 Troubleshooting

5. The Broker

• 5.1 Overview
• 5.2 Basic setup
• 5.3 Querying a Broker
• 5.4 Customizing the Broker's Query Result Set
• 5.5 World Wide Web interface description
• 5.6 Administrating a Broker
• 5.7 Tuning Glimpse indexing in the Broker
• 5.8 Using different index/search engines with the Broker
• 5.9 Collector interface description: Collection.conf
• 5.10 Troubleshooting

6. Programs and layout of the installed Harvest software

• 6.1 $HARVEST_HOME
• 6.2 $HARVEST_HOME/bin
• 6.3 $HARVEST_HOME/brokers
• 6.4 $HARVEST_HOME/cgi-bin
• 6.5 $HARVEST_HOME/gatherers
• 6.6 $HARVEST_HOME/lib
• 6.7 $HARVEST_HOME/lib/broker
• 6.8 $HARVEST_HOME/lib/gatherer
• 6.9 $HARVEST_HOME/tmp

7. The Summary Object Interchange Format (SOIF)

• 7.1 Formal description of SOIF
• 7.2 List of common SOIF attribute names

8. Gatherer Examples

• 8.1 Example 1 - A simple Gatherer
• 8.2 Example 2 - Incorporating manually generated information
• 8.3 Example 3 - Customizing type recognition and candidate selection
• 8.4 Example 4 - Customizing type recognition and summarizing
• 8.5 Example 5 - Using RootNode filters

9. History of Harvest

• 9.1 History of Harvest
• 9.2 History of Harvest User's Manual