Open Source Projects Europe

                3.7. Sorting search results

                3.7. Sorting search results

                3.8. Search tips, shortcuts

                3.8. Search tips, shortcuts

   4. Installation

   4. Installation

                4.1. Installing a prebuilt copy

                4.1. Installing a prebuilt copy

   documents will appear first.

   documents will appear first.

   You do not need to remember in what file or email message you stored a

   You do not need to remember in what file or email message you stored a

   given piece of information. You just ask for related terms, and the tool

   given piece of information. You just ask for related terms, and the tool

   will return a list of documents where those terms are prominent, in a

   will return a list of documents where those terms are prominent, in a

   Recoll tries to determine which documents are most relevant to the search

   Recoll tries to determine which documents are most relevant to the search

   terms you provide. Computer algorithms for determining relevance can be

   terms you provide. Computer algorithms for determining relevance can be

   very complex, and in general are inferior to the power of the human mind

   very complex, and in general are inferior to the power of the human mind

   to rapidly determine relevance. The quality of relevance guessing by the

   to rapidly determine relevance. The quality of relevance guessing by the

   application.

   application.

   In many cases, you are looking for all the forms of a word, not for a

   In many cases, you are looking for all the forms of a word, not for a

   specific form or spelling. These different forms may include plurals,

   specific form or spelling. These different forms may include plurals,

   different tenses for a verb, or terms derived from the same root or stem

   different tenses for a verb, or terms derived from the same root or stem

   expand queries to all such related terms (words that reduce to the same

   expand queries to all such related terms (words that reduce to the same

   stem). This expansion can be disabled at search time.

   stem). This expansion can be disabled at search time.

   Stemming, by itself, does not accomodate for misspellings or phonetic

   searches. Recoll currently does not support these features.

   searches. Recoll currently does not support these features.

     ----------------------------------------------------------------------

     ----------------------------------------------------------------------

1.3. Recoll overview

1.3. Recoll overview

   The resulting index can be big (roughly the size of the original document

   The resulting index can be big (roughly the size of the original document

   set), but it is not a document archive. Recoll can only display documents

   set), but it is not a document archive. Recoll can only display documents

   that still exist at the place from which they were indexed. (Actually,

   that still exist at the place from which they were indexed. (Actually,

   there is a way to reconstruct a document from the information in the

   there is a way to reconstruct a document from the information in the

   index, but the result is not nice, as all formatting, punctuation and

   index, but the result is not nice, as all formatting, punctuation and

   Recoll stores all internal data in Unicode UTF-8 format, and it can index

   Recoll stores all internal data in Unicode UTF-8 format, and it can index

   files with different character sets, encodings, and languages into the

   files with different character sets, encodings, and languages into the

   same index. It has input filters for many document types.

   same index. It has input filters for many document types.

   Stemming depends on the document language. Recoll stores the unstemmed

   Stemming depends on the document language. Recoll stores the unstemmed

   versions of terms and uses auxiliary databases for term expansion. It can

   versions of terms and uses auxiliary databases for term expansion. It can

   switch stemming languages, or add a language, without reindexing. Storing

   documents in different languages in the same index is possible, and useful

   documents in different languages in the same index is possible, and useful

   in practice, but does introduce possibilities of confusion. Recoll

   in practice, but does introduce possibilities of confusion. Recoll

   currently makes no attempt at automatic language recognition.

   currently makes no attempt at automatic language recognition.

   Recoll has many parameters which define exactly what to index, and how to

   Recoll has many parameters which define exactly what to index, and how to

   classify and decode the source documents. These are kept in a

   classify and decode the source documents. These are kept in a

   configuration file. A default configuration is copied into a standard

   configuration file. A default configuration is copied into a standard

   location (usually something like /usr/[local/]share/recoll/examples)

   location (usually something like /usr/[local/]share/recoll/examples)

   during installation. The default parameters from this file may be

   during installation. The default parameters from this file may be

   Indexing is started automatically the first time you execute the recoll

   Indexing is started automatically the first time you execute the recoll

   search graphical user interface, or by executing the recollindex command.

   search graphical user interface, or by executing the recollindex command.

   Searches are performed inside the recoll program, which has many options

   Searches are performed inside the recoll program, which has many options

   confidential data is indexed, access to the database directory should be

   confidential data is indexed, access to the database directory should be

   restricted.

   restricted.

   As of version 1.4, Recoll will create the configuration directory with a

   As of version 1.4, Recoll will create the configuration directory with a

   mode of 0700 (access by owner only). As the index data directory is by

   mode of 0700 (access by owner only). As the index data directory is by

   default a subdirectory of the configuration directory, this should result

   in appropriate protection.

   in appropriate protection.

   If you use another setup, you should think of the kind of protection you

   If you use another setup, you should think of the kind of protection you

   need for your index, and set the directory and files access modes

   need for your index, and set the directory and files access modes

   appropriately.

   appropriately.

     ----------------------------------------------------------------------

     ----------------------------------------------------------------------

2.3. The indexing configuration

2.3. The indexing configuration

   Values set in the system-wide configuration file (named like

   Values set in the system-wide configuration file (named like

   /usr/[local/]share/recoll/examples/recoll.conf) can be overriden by those

   set in the personal one, named $HOME/.recoll/recoll.conf by default or

   set in the personal one, named $HOME/.recoll/recoll.conf by default or

   $RECOLL_CONFDIR/recoll.conf if RECOLL_CONFDIR is set.

   $RECOLL_CONFDIR/recoll.conf if RECOLL_CONFDIR is set.

   The most accurate documentation for editing the file is given by comments

   The most accurate documentation for editing the file is given by comments

   inside the central one. If you want to adjust the configuration before

   inside the central one. If you want to adjust the configuration before

   empty configuration files.

   empty configuration files.

   The configuration is also documented inside the installation chapter of

   The configuration is also documented inside the installation chapter of

   this document, or in the recoll.conf(5) man page.

   this document, or in the recoll.conf(5) man page.

   (ie: pdf, postscript, ms-word...) are described in the external packages

   (ie: pdf, postscript, ms-word...) are described in the external packages

   section

   section

     ----------------------------------------------------------------------

     ----------------------------------------------------------------------

   indexing thread inside the recoll program (use the File menu). Both

   indexing thread inside the recoll program (use the File menu). Both

   programs will use of the RECOLL_CONFDIR variable or accept a -c confdir

   programs will use of the RECOLL_CONFDIR variable or accept a -c confdir

   option to specify the configuration directory to be used.

   option to specify the configuration directory to be used.

   If the recoll program finds no index when it starts, it will automatically

   If the recoll program finds no index when it starts, it will automatically

   start indexing (except if canceled).

   It is best to avoid interrupting the indexing process, as this may

   It is best to avoid interrupting the indexing process, as this may

   sometimes leave the index in a bad state. This is not a serious problem,

   sometimes leave the index in a bad state. This is not a serious problem,

   as you then just need to clear everything and restart the indexing: the

   as you then just need to clear everything and restart the indexing: the

   index files are normally stored in the $HOME/.recoll/xapiandb directory,

   index files are normally stored in the $HOME/.recoll/xapiandb directory,

   be disabled globally in the preferences).

   be disabled globally in the preferences).

   Recoll remembers the last few searches that you performed. You can use the

   Recoll remembers the last few searches that you performed. You can use the

   simple search text entry widget (a combobox) to recall them (click on the

   simple search text entry widget (a combobox) to recall them (click on the

   thing at the right of the text field). Please note, however, that only the

   thing at the right of the text field). Please note, however, that only the

   search texts are remembered, not the mode (all/any/filename).

   Hitting ^Tab (Ctrl + Tab) while entering a word in the simple search entry

   Hitting ^Tab (Ctrl + Tab) while entering a word in the simple search entry

   will open a window with possible completions for the word. The completions

   will open a window with possible completions for the word. The completions

   are extracted from the database.

   are extracted from the database.

     ----------------------------------------------------------------------

     ----------------------------------------------------------------------

  3.2.1. The result list right-click menu

  3.2.1. The result list right-click menu

   Apart from the preview and edit links, you can display a popup menu by

   right-clicking over a paragraph in the result list. This menu has the

   right-clicking over a paragraph in the result list. This menu has the

   following entries:

   following entries:

     * Preview

     * Preview

     * Copy Url

     * Copy Url

     * Find similar

     * Find similar

   The Preview and Edit entries do the same thing as the corresponding links.

   The Preview and Edit entries do the same thing as the corresponding links.

   clipboard, for pasting into another application.

   clipboard, for pasting into another application.

   The Find similar entry will select a number of relevant term from the

   The Find similar entry will select a number of relevant term from the

   current document and enter them into the simple search field. You can then

   current document and enter them into the simple search field. You can then

   start a simple search, with a good chance of finding documents related to

   start a simple search, with a good chance of finding documents related to

   The preview tabs have an internal incremental search function. You

   The preview tabs have an internal incremental search function. You

   initiate the search either by typing a / (slash) inside the text area or

   initiate the search either by typing a / (slash) inside the text area or

   by clicking into the Search for: text field and entering the search

   by clicking into the Search for: text field and entering the search

   string. You can then use the Next and Previous buttons to find the

   string. You can then use the Next and Previous buttons to find the

   next/previous occurence. You can also type F3 inside the text area to get

   to the next occurrence.

   to the next occurrence.

   If you have a search string entered and you use ^Up/^Down to browse the

   If you have a search string entered and you use ^Up/^Down to browse the

   results, the search is initiated for each successive document. If the

   results, the search is initiated for each successive document. If the

   string is found, the cursor will be positioned at the first occurrence of

   the search string.

   the search string.

     ----------------------------------------------------------------------

     ----------------------------------------------------------------------

3.4. Complex/advanced search

3.4. Complex/advanced search

   expansion). All relevant fields will be combined by an implicit AND

   expansion). All relevant fields will be combined by an implicit AND

   clause. All fields except "Exact phrase" can accept a mix of single words

   clause. All fields except "Exact phrase" can accept a mix of single words

   and phrases enclosed in double quotes.

   and phrases enclosed in double quotes.

   Advanced search will let you search for documents of specific mime types

   Advanced search will let you search for documents of specific mime types

   of the file type selection can be saved as the default (the file type

   of the file type selection can be saved as the default (the file type

   filter will not be activated at program startup, but the lists will be in

   the restored state).

   the restored state).

   instead, as the performance will be much better.

   Click on the Start Search button in the advanced search dialog, or type

   Click on the Start Search button in the advanced search dialog, or type

   Enter in any text field to start the search. The button in the main window

   Enter in any text field to start the search. The button in the main window

   always performs a simple search.

   always performs a simple search.

   The tool sorts a specified number of the most relevant documents in the

   The tool sorts a specified number of the most relevant documents in the

   result list, according to specified criteria. The currently available

   result list, according to specified criteria. The currently available

   criteria are date and mime type.

   criteria are date and mime type.

     ----------------------------------------------------------------------

     ----------------------------------------------------------------------

3.8. Search tips, shortcuts

3.8. Search tips, shortcuts

   Disabling stem expansion. Entering a capitalized word in any search field

   Disabling stem expansion. Entering a capitalized word in any search field

   will prevent stem expansion (no search for gardening if you enter Garden

   will prevent stem expansion (no search for gardening if you enter Garden

   instead of garden). This is the only case where character case should make

   instead of garden). This is the only case where character case should make

   Phrases. A phrase can be looked for by enclosing it in double quotes.

   Phrases. A phrase can be looked for by enclosing it in double quotes.

   Example: "user manual" will look only for occurrences of user immediately

   Example: "user manual" will look only for occurrences of user immediately

   followed by manual. You can use the This exact phrase field of the

   followed by manual. You can use the This exact phrase field of the

   advanced search dialog to the same effect. Phrases can be entered along

   advanced search dialog to the same effect. Phrases can be entered along

   set, a phrase will be automatically built and added to simple searches

   when looking for Any terms. This will not change radically the results,

   when looking for Any terms. This will not change radically the results,

   but will give a relevance boost to the results where the search terms

   but will give a relevance boost to the results where the search terms

   appear as a phrase. Ie: searching for virtual reality will still find all

   appear as a phrase. Ie: searching for virtual reality will still find all

   documents where either virtual or reality or both appear, but those which

   documents where either virtual or reality or both appear, but those which

   contain virtual reality should appear sooner in the list.

   contain virtual reality should appear sooner in the list.

   Finding related documents. Selecting the Find similar documents entry in

   Finding related documents. Selecting the Find similar documents entry in

   the result list paragraph right-click menu will select a set of

   the result list paragraph right-click menu will select a set of

   "interesting" terms from the current result, and insert them into the

   "interesting" terms from the current result, and insert them into the

   simple search entry field. You can then possibly edit the list and start a

   simple search entry field. You can then possibly edit the list and start a

   search to find documents which may be apparented to the current result.

   search to find documents which may be apparented to the current result.

   File names. File names are added as terms during indexing, and you can

   File names. File names are added as terms during indexing, and you can

   specify them as ordinary terms in normal search fields (Recoll used to

   specify them as ordinary terms in normal search fields (Recoll used to

   index all directories in the file path as terms. This has been abandoned

   as it did not seem really useful). Alternatively, you can use the specific

   as it did not seem really useful). Alternatively, you can use the specific

   file name search which will only look for file names and can use wildcard

   file name search which will only look for file names and can use wildcard

   expansion.

   expansion.

   Quitting. Entering ^Q almost anywhere will close the application.

   Quitting. Entering ^Q almost anywhere will close the application.

     ----------------------------------------------------------------------

     ----------------------------------------------------------------------

   Query configuration entry in the Preferences menu.

   Query configuration entry in the Preferences menu.

   There are two tabs in the dialog, dealing with the interface itself, and

   There are two tabs in the dialog, dealing with the interface itself, and

   with the parameters used for searching and returning results.

   with the parameters used for searching and returning results.

   User interface parameters:

   User interface parameters:

     * Number of results in a result page

     * Number of results in a result page

     * Result list font: There is quite a lot of information shown in the

     * Result list font: There is quite a lot of information shown in the

       The rest of the fonts used by Recoll are determined by your generic QT

       The rest of the fonts used by Recoll are determined by your generic QT

       config (try the qtconfig command.

       config (try the qtconfig command.

       which will be started from the Help menu to read the user manual. You

       which will be started from the Help menu to read the user manual. You

       can enter a simple name if the command is in your PATH, or browse for

       can enter a simple name if the command is in your PATH, or browse for

       a full pathname.

       a full pathname.

     * Show document type icons in result list: icons in the result list can

     * Show document type icons in result list: icons in the result list can

       be turned off. They take quite a lot of space and convey relatively

       be turned off. They take quite a lot of space and convey relatively

       little useful information.

       little useful information.

     * Auto-start simple search on whitespace entry: if this is checked, a

       search will be executed each time you enter a space in the simple

       search will be executed each time you enter a space in the simple

       search input field. This lets you look at the result list as you enter

       search input field. This lets you look at the result list as you enter

       new terms. This is off by default, you may like it or not...

       new terms. This is off by default, you may like it or not...

   Search parameters:

   Search parameters:

       document abstracts when displaying the result list. Abstracts are

       document abstracts when displaying the result list. Abstracts are

       constructed by taking context from the document information, around

       constructed by taking context from the document information, around

       the search terms. This can slow down result list display significantly

       the search terms. This can slow down result list display significantly

       for big documents, and you may want to turn it off.

       for big documents, and you may want to turn it off.

       and display an abstract in place of an explicit abstract found within

       and display an abstract in place of an explicit abstract found within

       the document itself.

       the document itself.

     * Synthetic abstract size: adjust to taste...

     * Synthetic abstract size: adjust to taste...

   that you may want to search. External indexes are designated by their

   that you may want to search. External indexes are designated by their

   database directory (ie: /home/someothergui/.recoll/xapiandb,

   database directory (ie: /home/someothergui/.recoll/xapiandb,

   /usr/local/recollglobal/xapiandb).

   /usr/local/recollglobal/xapiandb).

   Once entered, the indexes will appear in the All indexes list, and you can

   Once entered, the indexes will appear in the All indexes list, and you can

   the Active indexes list.

   Your main database (the one the current configuration indexes to), is

   Your main database (the one the current configuration indexes to), is

   always implicitly active. If this is not desirable, you can set up your

   configuration so that it indexes, for example, an empty directory.

   configuration so that it indexes, for example, an empty directory.

     ----------------------------------------------------------------------

     ----------------------------------------------------------------------

                            Chapter 4. Installation

                            Chapter 4. Installation

4.1. Installing a prebuilt copy

4.1. Installing a prebuilt copy

   Recoll binary installations are always linked statically to the xapian

   Recoll binary installations are always linked statically to the xapian

   libraries, and have no other dependencies. You will only have to check or

   libraries, and have no other dependencies. You will only have to check or

   install supporting applications for the file types that you want to index

   install supporting applications for the file types that you want to index

     ----------------------------------------------------------------------

     ----------------------------------------------------------------------

  4.1.1. Installing through a package system

  4.1.1. Installing through a package system

     * dvi: dvips

     * dvi: dvips

     * djvu: DjVuLibre

     * djvu: DjVuLibre

     * MP3: Recoll will use the id3info command from the id3lib package to

     * MP3: Recoll will use the id3info command from the id3lib package to

       extract tag information. Without it, only the filenames will be

       indexed.

       indexed.

     ----------------------------------------------------------------------

     ----------------------------------------------------------------------

4.3. Building from source

4.3. Building from source

  4.3.1. Prerequisites

  4.3.1. Prerequisites

   At the very least, you will need to download and install the xapian core

   At the very least, you will need to download and install the xapian core

   package (Recoll development currently uses version 0.9.5), and the qt

   package (Recoll development currently uses version 0.9.5), and the qt

   runtime and development packages (Recoll development currently uses

   You will most probably be able to find a binary package for qt for your

   You will most probably be able to find a binary package for qt for your

   system. You may have to compile Xapian but this is not difficult (if you

   system. You may have to compile Xapian but this is not difficult (if you

   are using FreeBSD, there is a port).

   are using FreeBSD, there is a port).

     * QTDIR should point to the directory above the one that holds the qt

     * QTDIR should point to the directory above the one that holds the qt

       include files (ie: qt.h).

       include files (ie: qt.h).

     * QMAKESPECS should be set to the name of one of the qt mkspecs

     * QMAKESPECS should be set to the name of one of the qt mkspecs

       subdirectories (ie: linux-g++).

   On many Linux systems, QTDIR is set by the login scripts, and QMAKESPECS

   On many Linux systems, QTDIR is set by the login scripts, and QMAKESPECS

   is not needed because there is a default link in mkspecs/.

   is not needed because there is a default link in mkspecs/.

   The Recoll configure script does a better job of checking these variables

   The Recoll configure script does a better job of checking these variables

   Normal procedure:

   Normal procedure:

         cd recoll-xxx

         cd recoll-xxx

         configure

         configure

         make

         make

   There little autoconfiguration. The configure script will mainly link one

   of the system-specific files in the mk directory to mk/sysconf. If your

   of the system-specific files in the mk directory to mk/sysconf. If your

   system is not known yet, it will tell you as much, and you may want to

   system is not known yet, it will tell you as much, and you may want to

   manually copy and modify one of the existing files (the new file name

   manually copy and modify one of the existing files (the new file name

   should be the output of uname -s).

   should be the output of uname -s).

   edit them by hand for now (there is still some hope for a GUI

   edit them by hand for now (there is still some hope for a GUI

   configuration tool in the future). The most accurate documentation for the

   configuration tool in the future). The most accurate documentation for the

   configuration parameters is given by comments inside the default files,

   configuration parameters is given by comments inside the default files,

   and we will just give a general overview here.

   and we will just give a general overview here.

   extract of the main configuration file might look as follows:

   extract of the main configuration file might look as follows:

         # Space-separated list of directories to index.

         # Space-separated list of directories to index.

         topdirs =  ~/docs /usr/share/doc

         topdirs =  ~/docs /usr/share/doc

     * Parameter affectation (name = value).

     * Parameter affectation (name = value).

     * Section definition ([somedirname]).

     * Section definition ([somedirname]).

   Section lines allow redefining some parameters for a directory subtree.

   Some of the parameters used for indexing are looked up hierarchically from

   Some of the parameters used for indexing are looked up hierarchically from

   the more to the less specific. Not all parameters can be meaningfully

   the more to the less specific. Not all parameters can be meaningfully

   redefined, this is specified for each in the next section.

   redefined, this is specified for each in the next section.

   The tilde character (~) is expanded in file names to the name of the

   The tilde character (~) is expanded in file names to the name of the

           directories that should be completely ignored. The list defined in

           directories that should be completely ignored. The list defined in

           the default file is:

           the default file is:

 *~ #* bin CVS  Cache caughtspam  tmp

 *~ #* bin CVS  Cache caughtspam  tmp

           changed for the top level ones in topdirs.

           The top-level directories are not affected by this list (that is,

           The top-level directories are not affected by this list (that is,

           a directory in topdirs might match and would still be indexed).

           a directory in topdirs might match and would still be indexed).

           The list in the default configuration does not exclude hidden

           The list in the default configuration does not exclude hidden

   filtersdir

   filtersdir

           A directory to search for the external filter scripts used to

           A directory to search for the external filter scripts used to

           index some types of files. The value should not be changed, except

           index some types of files. The value should not be changed, except

           if you want to modify one of the default scripts. The value can be

           if you want to modify one of the default scripts. The value can be

           redefined for any subdirectory.

   indexstemminglanguages

   indexstemminglanguages

           A list of languages for which the stem expansion databases will be

           A list of languages for which the stem expansion databases will be

           built. See recollindex(1) for possible values. You can add a stem

           built. See recollindex(1) for possible values. You can add a stem

   defaultcharset

   defaultcharset

           The name of the character set used for files that do not contain a

           The name of the character set used for files that do not contain a

           character set definition (ie: plain text files). This can be

           character set definition (ie: plain text files). This can be

           redefined for any subdirectory. If it is not set at all, the

           character set used is the one defined by the nls environment

           character set used is the one defined by the nls environment

           (LC_ALL, LC_CTYPE, LANG), or iso8859-1 if nothing is set.

           (LC_ALL, LC_CTYPE, LANG), or iso8859-1 if nothing is set.

   guesscharset

   guesscharset

   usesystemfilecommand

   usesystemfilecommand

           Decide if we use the file -i system command as a final step for

           Decide if we use the file -i system command as a final step for

           determining the mime type for a file (the main procedure uses

           determining the mime type for a file (the main procedure uses

           suffix associations as defined in the mimemap file). This can be

           suffix associations as defined in the mimemap file). This can be

           indexing of many bogus "text" files.

   indexallfilenames

   indexallfilenames

           Recoll indexes file names in a special section of the database to

           Recoll indexes file names in a special section of the database to

           allow specific file names searches using wild cards. This

           allow specific file names searches using wild cards. This

           parameter decides if file name indexing is performed only for

           parameter decides if file name indexing is performed only for

           files with mime types that would qualify them for full text

           files with mime types that would qualify them for full text

           indexing, or for all files inside the selected subtrees,

           indexing, or for all files inside the selected subtrees,

   idxabsmlen

   idxabsmlen

           Recoll stores an abstract for each indexed file inside the

           Recoll stores an abstract for each indexed file inside the

           database. This is so that they can be displayed inside the result

           database. This is so that they can be displayed inside the result

   mimemap also has a recoll_noindex variable which is a list of suffixes.

   mimemap also has a recoll_noindex variable which is a list of suffixes.

   Matching files will be skipped (avoids unnecessary decompressions or file

   Matching files will be skipped (avoids unnecessary decompressions or file

   executions). This is partially redundant with skippedNames in the main

   executions). This is partially redundant with skippedNames in the main

   configuration file, with two differences: it will not affect directories,

   configuration file, with two differences: it will not affect directories,

   and it can be changed for any subdirectory.

     ----------------------------------------------------------------------

     ----------------------------------------------------------------------

  4.4.3. The mimeconf file

  4.4.3. The mimeconf file

   mimeconf specifies how the different mime types are handled for indexing,

   mimeconf specifies how the different mime types are handled for indexing,

   and for display.

   and for display.

   Changing the indexing parameters is probably not a good idea except if you

   Changing the indexing parameters is probably not a good idea except if you

   previewed internally or displayed using firefox, but you may prefer

   previewed internally or displayed using firefox, but you may prefer

   mozilla, your openoffice.org program might be named oofice instead of

   mozilla, your openoffice.org program might be named oofice instead of

   openoffice ...). Look for the [view] section.

   openoffice ...). Look for the [view] section.

   You can also change the icons which are displayed by recoll in the result

   You can also change the icons which are displayed by recoll in the result