Open Source Projects Europe

   This document introduces full text search notions and describes the

   This document introduces full text search notions and describes the

   installation and use of the Recoll application. It currently describes

   installation and use of the Recoll application. It currently describes

   Recoll 1.18.

   Recoll 1.18.

     ----------------------------------------------------------------------

     ----------------------------------------------------------------------

   Table of Contents

   Table of Contents

   1. Introduction

   1. Introduction

                             2.3.2. Index case and diacritics sensitivity

                             2.3.2. Index case and diacritics sensitivity

                             2.3.3. The index configuration GUI

                             2.3.3. The index configuration GUI

                2.5. Periodic indexing

                2.5. Periodic indexing

                             2.5.1. Running indexing

                             2.5.1. Running indexing

                             3.1.2. The default result list

                             3.1.2. The default result list

                             3.1.3. The result table

                             3.1.3. The result table

                             duplicates

                             duplicates

                3.2. Searching with the KDE KIO slave

                3.2. Searching with the KDE KIO slave

                             3.2.1. What's this

                             3.2.1. What's this

                4.1. Writing a document filter

                4.1. Writing a document filter

                             4.1.1. Simple filters

                             4.1.1. Simple filters

                4.2. Field data processing

                4.2. Field data processing

                4.3. API

                4.3. API

                             5.4.5. The mimeview file

                             5.4.5. The mimeview file

                             5.4.6. Examples of configuration adjustments

                             5.4.6. Examples of configuration adjustments

1.1. Giving it a try

1.1. Giving it a try

   If you do not like reading manuals (who does?) and would like to give

   If you do not like reading manuals (who does?) and would like to give

   Recoll a try, just install the application and start the recoll graphical

   Recoll a try, just install the application and start the recoll graphical

   area.

   area.

   Also be aware that you may need to install the appropriate supporting

   Also be aware that you may need to install the appropriate supporting

   applications for document types that need them (for example antiword for

   applications for document types that need them (for example antiword for

   Microsoft Word files).

   Microsoft Word files).

1.2. Full text search

1.2. Full text search

   Recoll is a full text search application. Full text search applications

   Recoll is a full text search application. Full text search applications

   let you find your data by content rather than by external attributes (like

   let you find your data by content rather than by external attributes (like

   Stemming, by itself, does not accommodate for misspellings or phonetic

   Stemming, by itself, does not accommodate for misspellings or phonetic

   searches. Recoll supports these features through a specific tool (the term

   searches. Recoll supports these features through a specific tool (the term

   explorer) which will let you explore the set of index terms along

   explorer) which will let you explore the set of index terms along

   different modes.

   different modes.

1.3. Recoll overview

1.3. Recoll overview

   Recoll uses the Xapian information retrieval library as its storage and

   Recoll uses the Xapian information retrieval library as its storage and

   retrieval engine. Xapian is a very mature package using a sophisticated

   retrieval engine. Xapian is a very mature package using a sophisticated

   options to help you find what you are looking for. However, there are

   options to help you find what you are looking for. However, there are

   other ways to perform Recoll searches: mostly a command line interface, a

   other ways to perform Recoll searches: mostly a command line interface, a

   Python programming interface, a KDE KIO slave module, and a Ubuntu Unity

   Python programming interface, a KDE KIO slave module, and a Ubuntu Unity

   Lens module.

   Lens module.

2.1. Introduction

2.1. Introduction

   Indexing is the process by which the set of documents is analyzed and the

   Indexing is the process by which the set of documents is analyzed and the

   data entered into the database. Recoll indexing is normally incremental:

   data entered into the database. Recoll indexing is normally incremental:

   -z or -Z).

   -z or -Z).

   The following sections give an overview of different aspects of the

   The following sections give an overview of different aspects of the

   indexing processes and configuration, with links to detailed sections.

   indexing processes and configuration, with links to detailed sections.

  2.1.1. Indexing modes

  2.1.1. Indexing modes

   Recoll indexing can be performed along two different modes:

   Recoll indexing can be performed along two different modes:

       by executing the recollindex command. The typical usage is to have a

       by executing the recollindex command. The typical usage is to have a

       nightly indexing run programmed into your cron file.

       nightly indexing run programmed into your cron file.

       or changed. recollindex runs as a daemon and uses a file system

       or changed. recollindex runs as a daemon and uses a file system

       alteration monitor such as inotify, Fam or Gamin to detect file

       alteration monitor such as inotify, Fam or Gamin to detect file

       changes.

       changes.

   The choice between the two methods is mostly a matter of preference, and

   The choice between the two methods is mostly a matter of preference, and

   indexing on a big documentation directory, and real time indexing on a

   indexing on a big documentation directory, and real time indexing on a

   small home directory). Monitoring a big file system tree can consume

   small home directory). Monitoring a big file system tree can consume

   significant system resources.

   significant system resources.

   The choice of method and the parameters used can be configured from the

   The choice of method and the parameters used can be configured from the

   recoll GUI: Preferences->Indexing schedule

  2.1.2. Configurations, multiple indexes

  2.1.2. Configurations, multiple indexes

   The parameters describing what is to be indexed and local preferences are

   The parameters describing what is to be indexed and local preferences are

   defined in text files contained in a configuration directory.

   defined in text files contained in a configuration directory.

   For index generation, multiple configurations are totally independant from

   For index generation, multiple configurations are totally independant from

   each other. When multiple indexes need to be used for a single search,

   each other. When multiple indexes need to be used for a single search,

   some parameters should be consistent among the configurations.

   some parameters should be consistent among the configurations.

  2.1.3. Document types

  2.1.3. Document types

   Recoll knows about quite a few different document types. The parameters

   Recoll knows about quite a few different document types. The parameters

   for document types recognition and processing are set in configuration

   for document types recognition and processing are set in configuration

   files.

   files.

   Other file types (ie: postscript, pdf, ms-word, rtf ...) need external

   Other file types (ie: postscript, pdf, ms-word, rtf ...) need external

   applications for preprocessing. The list is in the installation section.

   applications for preprocessing. The list is in the installation section.

   After every indexing operation, Recoll updates a list of commands that

   After every indexing operation, Recoll updates a list of commands that

   would be needed for indexing existing files types. This list can be

   would be needed for indexing existing files types. This list can be

   displayed by selecting the menu option File->Show Missing Helpers in the

   recoll GUI. It is stored in the missing text file inside the configuration

   recoll GUI. It is stored in the missing text file inside the configuration

   directory.

   directory.

  2.1.4. Recovery

  2.1.4. Recovery

   In the rare case where the index becomes corrupted (which can signal

   In the rare case where the index becomes corrupted (which can signal

   itself by weird search results or crashes), the index files need to be

   itself by weird search results or crashes), the index files need to be

   erased before restarting a clean indexing pass. Just delete the xapiandb

   erased before restarting a clean indexing pass. Just delete the xapiandb

   directory (see next section), or, alternatively, start the next

   directory (see next section), or, alternatively, start the next

   recollindex with the -z option, which will reset the database before

   recollindex with the -z option, which will reset the database before

   indexing.

   indexing.

2.2. Index storage

2.2. Index storage

   The default location for the index data is the xapiandb subdirectory of

   The default location for the index data is the xapiandb subdirectory of

   the Recoll configuration directory, typically $HOME/.recoll/xapiandb/.

   the Recoll configuration directory, typically $HOME/.recoll/xapiandb/.

   This can be changed via two different methods (with different purposes):

   This can be changed via two different methods (with different purposes):

       RECOLL_CONFDIR environment variable, or using the -c option to the

       RECOLL_CONFDIR environment variable, or using the -c option to the

       Recoll commands. This method would typically be used to index

       Recoll commands. This method would typically be used to index

       different areas of the file system to different indexes. For example,

       different areas of the file system to different indexes. For example,

       if you were to issue the following commands:

       if you were to issue the following commands:

       Using multiple configuration directories and configuration options

       Using multiple configuration directories and configuration options

       allows you to tailor multiple configurations and indexes to handle

       allows you to tailor multiple configurations and indexes to handle

       whatever subset of the available data you wish to make searchable.

       whatever subset of the available data you wish to make searchable.

       storage location for the index by setting the dbdir parameter in the

       storage location for the index by setting the dbdir parameter in the

       configuration file (see the configuration section). This method would

       configuration file (see the configuration section). This method would

       mainly be of use if you wanted to keep the configuration directory in

       mainly be of use if you wanted to keep the configuration directory in

       its default location, but desired another location for the index,

       its default location, but desired another location for the index,

       typically out of disk occupation concerns.

       typically out of disk occupation concerns.

   The index data directory (xapiandb) only contains data that can be

   The index data directory (xapiandb) only contains data that can be

   completely rebuilt by an index run (as long as the original documents

   completely rebuilt by an index run (as long as the original documents

   exist), and it can always be destroyed safely.

   exist), and it can always be destroyed safely.

  2.2.1. Xapian index formats

  2.2.1. Xapian index formats

   Xapian versions usually support several formats for index storage. A given

   Xapian versions usually support several formats for index storage. A given

   major Xapian version will have a current format, used to create new

   major Xapian version will have a current format, used to create new

   indexes, and will also support the format from the previous major version.

   indexes, and will also support the format from the previous major version.

   Using the -z option to recollindex is not sufficient to change the format,

   Using the -z option to recollindex is not sufficient to change the format,

   you will have to delete all files inside the index directory (typically

   you will have to delete all files inside the index directory (typically

   ~/.recoll/xapiandb) before starting the indexing.

   ~/.recoll/xapiandb) before starting the indexing.

  2.2.2. Security aspects

  2.2.2. Security aspects

   The Recoll index does not hold copies of the indexed documents. But it

   The Recoll index does not hold copies of the indexed documents. But it

   does hold enough data to allow for an almost complete reconstruction. If

   does hold enough data to allow for an almost complete reconstruction. If

   confidential data is indexed, access to the database directory should be

   confidential data is indexed, access to the database directory should be

   in appropriate protection.

   in appropriate protection.

   If you use another setup, you should think of the kind of protection you

   If you use another setup, you should think of the kind of protection you

   need for your index, set the directory and files access modes

   need for your index, set the directory and files access modes

   appropriately, and also maybe adjust the umask used during index updates.

   appropriately, and also maybe adjust the umask used during index updates.

2.3. Index configuration

2.3. Index configuration

   Variables set inside the Recoll configuration files control which areas of

   Variables set inside the Recoll configuration files control which areas of

   the file system are indexed, and how files are processed. These variables

   the file system are indexed, and how files are processed. These variables

   section.

   section.

   As of Recoll 1.18 there are two incompatible types of Recoll indexes,

   As of Recoll 1.18 there are two incompatible types of Recoll indexes,

   depending on the treatment of character case and diacritics. The next

   depending on the treatment of character case and diacritics. The next

   section describes the two types in more detail.

   section describes the two types in more detail.

  2.3.1. Multiple indexes

  2.3.1. Multiple indexes

   Multiple Recoll indexes can be created by using several configuration

   Multiple Recoll indexes can be created by using several configuration

   directories which are usually set to index different areas of the file

   directories which are usually set to index different areas of the file

   Most importantly, all indexes to be queried concurrently must have the

   Most importantly, all indexes to be queried concurrently must have the

   same option concerning character case and diacritics stripping, but there

   same option concerning character case and diacritics stripping, but there

   are other constraints. Most of the relevant parameters are described in

   are other constraints. Most of the relevant parameters are described in

   the linked section.

   the linked section.

  2.3.2. Index case and diacritics sensitivity

  2.3.2. Index case and diacritics sensitivity

   As of Recoll version 1.18 you have a choice of building an index with

   As of Recoll version 1.18 you have a choice of building an index with

   terms stripped of character case and diacritics, or one with raw terms.

   terms stripped of character case and diacritics, or one with raw terms.

   For a source term of Resume, the former will store resume, the latter

   For a source term of Resume, the former will store resume, the latter

   As a cost for added capability, a raw index will be slightly bigger than a

   As a cost for added capability, a raw index will be slightly bigger than a

   stripped one (around 10%). Also, searches will be more complex, so

   stripped one (around 10%). Also, searches will be more complex, so

   probably slightly slower, and the feature is still young, so that a

   probably slightly slower, and the feature is still young, so that a

   certain amount of weirdness cannot be excluded.

   certain amount of weirdness cannot be excluded.

  2.3.3. The index configuration GUI

  2.3.3. The index configuration GUI

   Most parameters for a given index configuration can be set from a recoll

   Most parameters for a given index configuration can be set from a recoll

   GUI running on this configuration (either as default, or by setting

   GUI running on this configuration (either as default, or by setting

   RECOLL_CONFDIR or the -c option.)

   RECOLL_CONFDIR or the -c option.)

   The interface is started from the Preferences->Index Configuration menu

   entry. It is divided in four tabs, Global parameters, Local parameters,

   entry. It is divided in four tabs, Global parameters, Local parameters,

   parameters.

   parameters.

   The Global parameters tab allows setting global variables, like the lists

   The Global parameters tab allows setting global variables, like the lists

   of top directories, skipped paths, or stemming languages.

   of top directories, skipped paths, or stemming languages.

   The configuration tool normally respects the comments and most of the

   The configuration tool normally respects the comments and most of the

   formatting inside the configuration file, so that it is quite possible to

   formatting inside the configuration file, so that it is quite possible to

   use it on hand-edited files, which you might nevertheless want to backup

   use it on hand-edited files, which you might nevertheless want to backup

   first...

   first...

   This feature can be enabled in the GUI Index configuration panel, or by

   This feature can be enabled in the GUI Index configuration panel, or by

2.5. Periodic indexing

2.5. Periodic indexing

  2.5.1. Running indexing

  2.5.1. Running indexing

   start indexing (except if canceled).

   start indexing (except if canceled).

   The recollindex indexing process can be interrupted by sending an

   The recollindex indexing process can be interrupted by sending an

   interrupt (Ctrl-C, SIGINT) or terminate (SIGTERM) signal. Some time may

   interrupt (Ctrl-C, SIGINT) or terminate (SIGTERM) signal. Some time may

   elapse before the process exits, because it needs to properly flush and

   elapse before the process exits, because it needs to properly flush and

   close the index. This can also be done from the recoll GUI File->Stop

   Indexing menu entry.

   Indexing menu entry.

   After such an interruption, the index will be somewhat inconsistent

   After such an interruption, the index will be somewhat inconsistent

   because some operations which are normally performed at the end of the

   because some operations which are normally performed at the end of the

   indexing pass will have been skipped (for example, the stemming and

   indexing pass will have been skipped (for example, the stemming and

   file selection process for some area of the file system, by adding the top

   file selection process for some area of the file system, by adding the top

   directory to the skippedPaths list and using an appropriate file selection

   directory to the skippedPaths list and using an appropriate file selection

   method to build the file list to be fed to recollindex -if. Trivial

   method to build the file list to be fed to recollindex -if. Trivial

   example:

   example:

            find . -name indexable.txt -print | recollindex -if

   recollindex -i will not descend into subdirectories specified as

   recollindex -i will not descend into subdirectories specified as

   parameters, but just add them as index entries. It is up to the external

   parameters, but just add them as index entries. It is up to the external

   file selection method to build the complete file list.

   file selection method to build the complete file list.

  2.5.2. Using cron to automate indexing

  2.5.2. Using cron to automate indexing

   The most common way to set up indexing is to have a cron task execute it

   The most common way to set up indexing is to have a cron task execute it

   every night. For example the following crontab entry would do it every day

   every night. For example the following crontab entry would do it every day

   at 3:30AM (supposing recollindex is in your PATH):

   at 3:30AM (supposing recollindex is in your PATH):

   Or, using anacron:

   Or, using anacron:

 1  15  su mylogin -c "recollindex recollindex > /tmp/rcltraceme 2>&1"

 1  15  su mylogin -c "recollindex recollindex > /tmp/rcltraceme 2>&1"

   As of version 1.17 the Recoll GUI has dialogs to manage crontab entries

   As of version 1.17 the Recoll GUI has dialogs to manage crontab entries

   for recollindex. You can reach them from the Preferences->Indexing

   Schedule menu. They only work with the good old cron, and do not give

   Schedule menu. They only work with the good old cron, and do not give

   access to all features of cron scheduling.

   access to all features of cron scheduling.

   The usual command to edit your crontab is crontab -e (which will usually

   The usual command to edit your crontab is crontab -e (which will usually

   start the vi editor to edit the file). You may have more sophisticated

   start the vi editor to edit the file). You may have more sophisticated

   Please be aware that there may be differences between your usual

   Please be aware that there may be differences between your usual

   interactive command line environment and the one seen by crontab commands.

   interactive command line environment and the one seen by crontab commands.

   Especially the PATH variable may be of concern. Please check the crontab

   Especially the PATH variable may be of concern. Please check the crontab

   manual pages about possible issues.

   manual pages about possible issues.

2.6. Real time indexing

2.6. Real time indexing

   Real time monitoring/indexing is performed by starting the recollindex -m

   Real time monitoring/indexing is performed by starting the recollindex -m

   command. With this option, recollindex will detach from the terminal and

   command. With this option, recollindex will detach from the terminal and

   become a daemon, permanently monitoring file changes and updating the

   become a daemon, permanently monitoring file changes and updating the

 recollconf=$HOME/.recoll-home

 recollconf=$HOME/.recoll-home

 recolldata=/usr/local/share/recoll

 recolldata=/usr/local/share/recoll

 RECOLL_CONFDIR=$recollconf $recolldata/examples/rclmon.sh start

 RECOLL_CONFDIR=$recollconf $recolldata/examples/rclmon.sh start

 fvwm

 fvwm

   The indexing daemon gets started, then the window manager, for which the

   The indexing daemon gets started, then the window manager, for which the

   session waits.

   session waits.

   By default the indexing daemon will monitor the state of the X11 session,

   By default the indexing daemon will monitor the state of the X11 session,

   email folders change. Also, monitoring large file trees by itself

   email folders change. Also, monitoring large file trees by itself

   significantly taxes system resources. You probably do not want to enable

   significantly taxes system resources. You probably do not want to enable

   it if your system is short on resources. Periodic indexing is adequate in

   it if your system is short on resources. Periodic indexing is adequate in

   most cases.

   most cases.

  2.6.1. Slowing down the reindexing rate for fast changing files

  2.6.1. Slowing down the reindexing rate for fast changing files

   When using the real time monitor, it may happen that some files need to be

   When using the real time monitor, it may happen that some files need to be

   indexed, but change so often that they impose an excessive load for the

   indexed, but change so often that they impose an excessive load for the

   system.

   system.

   Recoll provides a configuration option to specify the minimum time before

   Recoll provides a configuration option to specify the minimum time before

   which a file, specified by a wildcard pattern, cannot be reindexed. See

   which a file, specified by a wildcard pattern, cannot be reindexed. See

   the mondelaypatterns parameter in the configuration section.

   the mondelaypatterns parameter in the configuration section.

3.1. Searching with the Qt graphical user interface

3.1. Searching with the Qt graphical user interface

   The recoll program provides the main user interface for searching. It is

   The recoll program provides the main user interface for searching. It is

   based on the Qt library.

   based on the Qt library.

   recoll has two search modes:

   recoll has two search modes:

       field where you can enter multiple words.

       field where you can enter multiple words.

       toolbox bar icon) has multiple entry fields, which you may use to

       toolbox bar icon) has multiple entry fields, which you may use to

       build a logical condition, with additional filtering on file type,

       build a logical condition, with additional filtering on file type,

       location in the file system, modification date, and size.

       location in the file system, modification date, and size.

   In most cases, you can enter the terms as you think them, even if they

   In most cases, you can enter the terms as you think them, even if they

   printed is for east-asian languages (Chinese, Japanese, Korean). Words

   printed is for east-asian languages (Chinese, Japanese, Korean). Words

   composed of single or multiple characters should be entered separated by

   composed of single or multiple characters should be entered separated by

   white space in this case (they would typically be printed without white

   white space in this case (they would typically be printed without white

   space).

   space).

  3.1.1. Simple search

  3.1.1. Simple search

    1. Start the recoll program.

    1. Start the recoll program.

    2. Possibly choose a search mode: Any term, All terms, File name or Query

    2. Possibly choose a search mode: Any term, All terms, File name or Query

   File name will specifically look for file names. The point of having a

   File name will specifically look for file names. The point of having a

   separate file name search is that wild card expansion can be performed

   separate file name search is that wild card expansion can be performed

   more efficiently on a small subset of the index (allowing wild cards on

   more efficiently on a small subset of the index (allowing wild cards on

   the left of terms without excessive penality). Things to know:

   the left of terms without excessive penality). Things to know:

       and is not treated specially.

       and is not treated specially.

       of the type of index.

       of the type of index.

       prepended and appended with '*' (ie: etc -> *etc*, but Etc -> etc).

       prepended and appended with '*' (ie: etc -> *etc*, but Etc -> etc).

       may result in inefficient searches.

       may result in inefficient searches.

   You can search for exact phrases (adjacent words in a given order) by

   You can search for exact phrases (adjacent words in a given order) by

   enclosing the input inside double quotes. Ex: "virtual reality".

   enclosing the input inside double quotes. Ex: "virtual reality".

   punctuation, newlines and all - except for wildcard characters (single ?

   punctuation, newlines and all - except for wildcard characters (single ?

   characters are ok). Recoll will process it and produce a meaningful

   characters are ok). Recoll will process it and produce a meaningful

   search. This is what most differentiates this mode from the Query Language

   search. This is what most differentiates this mode from the Query Language

   mode, where you have to care about the syntax.

   mode, where you have to care about the syntax.

   You can use the Tools->Advanced search dialog for more complex searches.

  3.1.2. The default result list

  3.1.2. The default result list

   After starting a search, a list of results will instantly be displayed in

   After starting a search, a list of results will instantly be displayed in

   the main list window.

   the main list window.

   open tabs in the existing preview window. You can use Shift+Click to force

   open tabs in the existing preview window. You can use Shift+Click to force

   the creation of another preview window, which may be useful to view the

   the creation of another preview window, which may be useful to view the

   documents side by side. (You can also browse successive results in a

   documents side by side. (You can also browse successive results in a

   single preview window by typing Shift+ArrowUp/Down in the window).

   single preview window by typing Shift+ArrowUp/Down in the window).

   The Preview and Open edit links may not be present for all entries,

   The Preview and Open edit links may not be present for all entries,

   meaning that Recoll has no configured way to preview a given file type

   meaning that Recoll has no configured way to preview a given file type

   (which was indexed by name only), or no configured external editor for the

   (which was indexed by name only), or no configured external editor for the

   file type. This can sometimes be adjusted simply by tweaking the mimemap

   file type. This can sometimes be adjusted simply by tweaking the mimemap

   The result list is divided into pages (the size of which you can change in

   The result list is divided into pages (the size of which you can change in

   the preferences). Use the arrow buttons in the toolbar or the links at the

   the preferences). Use the arrow buttons in the toolbar or the links at the

   bottom of the page to browse the results.

   bottom of the page to browse the results.

   Apart from the preview and edit links, you can display a pop-up menu by

   Apart from the preview and edit links, you can display a pop-up menu by

   right-clicking over a paragraph in the result list. This menu has the

   right-clicking over a paragraph in the result list. This menu has the

   following entries:

   following entries:

   The Preview and Open entries do the same thing as the corresponding links.

   The Preview and Open entries do the same thing as the corresponding links.

   The Copy File Name and Copy Url copy the relevant data to the clipboard,

   The Copy File Name and Copy Url copy the relevant data to the clipboard,

   for later pasting.

   for later pasting.

   lists extracts from the document, taken around search terms occurrences,

   lists extracts from the document, taken around search terms occurrences,

   along with the corresponding page number, as links which can be used to

   along with the corresponding page number, as links which can be used to

   start the native viewer on the appropriate page. If the viewer supports

   start the native viewer on the appropriate page. If the viewer supports

   it, its search function will also be primed with one of the search terms.

   it, its search function will also be primed with one of the search terms.

  3.1.3. The result table

  3.1.3. The result table

   In Recoll 1.15 and newer, the results can be displayed in spreadsheet-like

   In Recoll 1.15 and newer, the results can be displayed in spreadsheet-like

   fashion. You can switch to this presentation by clicking the table-like

   fashion. You can switch to this presentation by clicking the table-like

   icon in the toolbar (this is a toggle, click again to restore the list).

   icon in the toolbar (this is a toggle, click again to restore the list).

   window with the corresponding values. You can click the row to freeze the

   window with the corresponding values. You can click the row to freeze the

   display. The bottom area is equivalent to a result list paragraph, with

   display. The bottom area is equivalent to a result list paragraph, with

   links for starting a preview or a native application, and an equivalent

   links for starting a preview or a native application, and an equivalent

   right-click menu. Typing Esc (the Escape key) will unfreeze the display.

   right-click menu. Typing Esc (the Escape key) will unfreeze the display.

   The preview window opens when you first click a Preview link inside the

   The preview window opens when you first click a Preview link inside the

   result list.

   result list.

   Subsequent preview requests for a given search open new tabs in the

   Subsequent preview requests for a given search open new tabs in the

   metadata stored in the index.

   metadata stored in the index.

   You can print the current preview window contents by typing Ctrl-P (Ctrl +

   You can print the current preview window contents by typing Ctrl-P (Ctrl +

   P) in the window text.

   P) in the window text.

   The preview window has an internal search capability, mostly controlled by

   The preview window has an internal search capability, mostly controlled by

   the panel at the bottom of the window, which works in two modes: as a

   the panel at the bottom of the window, which works in two modes: as a

   classical editor incremental search, where we look for the text entered in

   classical editor incremental search, where we look for the text entered in

   the entry zone, or as a way to walk the matches between the document and

   the entry zone, or as a way to walk the matches between the document and

           list for this group will be walked. This is not the same as a text

           list for this group will be walked. This is not the same as a text

           search, because the occurences will include non-exact matches (as

           search, because the occurences will include non-exact matches (as

           caused by stemming or wildcards). The search will revert to the

           caused by stemming or wildcards). The search will revert to the

           text mode as soon as you edit the entry area.

           text mode as soon as you edit the entry area.

   The advanced search dialog helps you build more complex queries without

   The advanced search dialog helps you build more complex queries without

   memorizing the search language constructs. It can be opened through the

   memorizing the search language constructs. It can be opened through the

   Tools menu or through the main toolbar.

   Tools menu or through the main toolbar.

   always performs a simple search.

   always performs a simple search.

   Click on the Show query details link at the top of the result page to see

   Click on the Show query details link at the top of the result page to see

   the query expansion.

   the query expansion.

   This part of the dialog lets you constructc a query by combining multiple

   This part of the dialog lets you constructc a query by combining multiple

   clauses of different types. Each entry field is configurable for the

   clauses of different types. Each entry field is configurable for the

   following modes:

   following modes:

   Additional entry fields can be created by clicking the Add clause button.

   Additional entry fields can be created by clicking the Add clause button.

   When searching, the non-empty clauses will be combined either with an AND

   When searching, the non-empty clauses will be combined either with an AND

   or an OR conjunction, depending on the choice made on the left (All

   or an OR conjunction, depending on the choice made on the left (All

   quick fox with a slack of 0 will match quick fox but not quick brown fox.

   quick fox with a slack of 0 will match quick fox but not quick brown fox.

   With a slack of 1 it will match the latter, but not fox quick. A proximity

   With a slack of 1 it will match the latter, but not fox quick. A proximity

   search for quick fox with the default slack will match the latter, and

   search for quick fox with the default slack will match the latter, and

   also a fox is a cunning and quick animal.

   also a fox is a cunning and quick animal.

   This part of the dialog has several sections which allow filtering the

   This part of the dialog has several sections which allow filtering the

   results of a search according to a number of criteria

   results of a search according to a number of criteria

       can specify both a minimum and a maximum date. The initial values are

       can specify both a minimum and a maximum date. The initial values are

       set according to the oldest and newest documents found in the index.

       set according to the oldest and newest documents found in the index.

       two entries for minimum and maximum size. Enter decimal numbers. You

       two entries for minimum and maximum size. Enter decimal numbers. You

       can use suffix multipliers: k/K, m/M, g/G, t/T for 1E3, 1E6, 1E9, 1E12

       can use suffix multipliers: k/K, m/M, g/G, t/T for 1E3, 1E6, 1E9, 1E12

       respectively.

       respectively.

       mime categories (ie: media/text/message/etc.).

       mime categories (ie: media/text/message/etc.).

       You can transfer the types between two boxes, to define which will be

       You can transfer the types between two boxes, to define which will be

       included or excluded by the search.

       included or excluded by the search.

       The state of the file type selection can be saved as the default (the

       The state of the file type selection can be saved as the default (the

       file type filter will not be activated at program start-up, but the

       file type filter will not be activated at program start-up, but the

       lists will be in the restored state).

       lists will be in the restored state).

       of the indexed area. You can use the Invert checkbox to search for

       of the indexed area. You can use the Invert checkbox to search for

       files not in the sub-tree instead. If you use directory filtering

       files not in the sub-tree instead. If you use directory filtering

       often and on big subsets of the file system, you may think of setting

       often and on big subsets of the file system, you may think of setting

       up multiple indexes instead, as the performance may be better.

       up multiple indexes instead, as the performance may be better.

       You can use relative/partial paths for filtering. Ie, entering

       You can use relative/partial paths for filtering. Ie, entering

       dirA/dirB would match either /dir1/dirA/dirB/myfile1 or

       dirA/dirB would match either /dir1/dirA/dirB/myfile1 or

       /dir2/dirA/dirB/someother/myfile2.

       /dir2/dirA/dirB/someother/myfile2.

   The advanced search tool memorizes the last 100 searches performed. You

   The advanced search tool memorizes the last 100 searches performed. You

   can walk the saved searches by using the up and down arrow keys while the

   can walk the saved searches by using the up and down arrow keys while the

   keyboard focus belongs to the advanced search dialog.

   keyboard focus belongs to the advanced search dialog.

   The complex search history can be erased, along with the one for simple

   The complex search history can be erased, along with the one for simple

   search, by selecting the File->Erase Search History menu entry.

   Recoll automatically manages the expansion of search terms to their

   Recoll automatically manages the expansion of search terms to their

   derivatives (ie: plural/singular, verb inflections). But there are other

   derivatives (ie: plural/singular, verb inflections). But there are other

   cases where the exact search term is not known. For example, you may not

   cases where the exact search term is not known. For example, you may not

   remember the exact spelling, or only know the beginning of the name.

   remember the exact spelling, or only know the beginning of the name.

   Double-clicking on a term in the result list will insert it into the

   Double-clicking on a term in the result list will insert it into the

   simple search entry field. You can also cut/paste between the result list

   simple search entry field. You can also cut/paste between the result list

   and any entry field (the end of lines will be taken care of).

   and any entry field (the end of lines will be taken care of).

   See the section describing the use of multiple indexes for generalities.

   See the section describing the use of multiple indexes for generalities.

   Only the aspects concerning the recoll GUI are described here.

   Only the aspects concerning the recoll GUI are described here.

   A recoll program instance is always associated with a specific index,

   A recoll program instance is always associated with a specific index,

   RECOLL_ACTIVE_EXTRA_DBS is available for Recoll versions 1.17.2 and later.

   RECOLL_ACTIVE_EXTRA_DBS is available for Recoll versions 1.17.2 and later.

   A change was made in the same update so that recoll will automatically

   A change was made in the same update so that recoll will automatically

   deactivate unreachable indexes when starting up.

   deactivate unreachable indexes when starting up.

   Documents that you actually view (with the internal preview or an external

   Documents that you actually view (with the internal preview or an external

   tool) are entered into the document history, which is remembered.

   tool) are entered into the document history, which is remembered.

   You can display the history list by using the Tools/Doc History menu

   You can display the history list by using the Tools/Doc History menu

   entry.

   entry.

   You can erase the document history by using the Erase document history

   You can erase the document history by using the Erase document history

   entry in the File menu.

   entry in the File menu.

   The documents in a result list are normally sorted in order of relevance.

   The documents in a result list are normally sorted in order of relevance.

   It is possible to specify a different sort order, either by using the

   It is possible to specify a different sort order, either by using the

   vertical arrows in the GUI toolbox to sort by date, or switching to the

   vertical arrows in the GUI toolbox to sort by date, or switching to the

   result table display and clicking on any header. The sort order chosen

   result table display and clicking on any header. The sort order chosen

   identity is based on an MD5 hash of the document container, not only of

   identity is based on an MD5 hash of the document container, not only of

   the text contents (so that ie, a text document with an image added will

   the text contents (so that ie, a text document with an image added will

   not be a duplicate of the text only). Duplicates hiding is controlled by

   not be a duplicate of the text only). Duplicates hiding is controlled by

   an entry in the GUI configuration dialog, and is off by default.

   an entry in the GUI configuration dialog, and is off by default.

   Term completion. Typing Esc Space in the simple search entry field while

   Term completion. Typing Esc Space in the simple search entry field while

   entering a word will either complete the current word if its beginning

   entering a word will either complete the current word if its beginning

   matches a unique term in the index, or open a window to propose a list of

   matches a unique term in the index, or open a window to propose a list of

   completions.

   completions.

   index all directories in the file path as terms. This has been abandoned

   index all directories in the file path as terms. This has been abandoned

   as it did not seem really useful). Alternatively, you can use the specific

   as it did not seem really useful). Alternatively, you can use the specific

   file name search which will only look for file names, and may be faster

   file name search which will only look for file names, and may be faster

   than the generic search especially when using wildcards.

   than the generic search especially when using wildcards.

   Phrases and Proximity searches. A phrase can be looked for by enclosing it

   Phrases and Proximity searches. A phrase can be looked for by enclosing it

   in double quotes. Example: "user manual" will look only for occurrences of

   in double quotes. Example: "user manual" will look only for occurrences of

   user immediately followed by manual. You can use the This phrase field of

   user immediately followed by manual. You can use the This phrase field of

   the advanced search dialog to the same effect. Phrases can be entered

   the advanced search dialog to the same effect. Phrases can be entered

   IBM. Searching for the word inside a phrase (ie: "the IBM company") will

   IBM. Searching for the word inside a phrase (ie: "the IBM company") will

   only match the dotted abrreviation if you increase the phrase slack (using

   only match the dotted abrreviation if you increase the phrase slack (using

   the advanced search panel control, or the o query language modifier).

   the advanced search panel control, or the o query language modifier).

   Literal occurences of the word will be matched normally.

   Literal occurences of the word will be matched normally.

   Using fields. You can use the query language and field specifications to

   Using fields. You can use the query language and field specifications to

   only search certain parts of documents. This can be especially helpful

   only search certain parts of documents. This can be especially helpful

   with email, for example only searching emails from a specific originator:

   with email, for example only searching emails from a specific originator:

   search tips from:helpfulgui

   search tips from:helpfulgui

   Printing previews. Entering Ctrl-P in a preview window will print the

   Printing previews. Entering Ctrl-P in a preview window will print the

   currently displayed text.

   currently displayed text.

   Quitting. Entering Ctrl-Q almost anywhere will close the application.

   Quitting. Entering Ctrl-Q almost anywhere will close the application.

   You can customize some aspects of the search interface by using the GUI

   You can customize some aspects of the search interface by using the GUI

   configuration entry in the Preferences menu.

   configuration entry in the Preferences menu.

   There are several tabs in the dialog, dealing with the interface itself,

   There are several tabs in the dialog, dealing with the interface itself,

   the parameters used for searching and returning results, and what indexes

   the parameters used for searching and returning results, and what indexes

   are searched.

   are searched.

   User interface parameters:

       highlighted in the result list samples and the preview window. The

       highlighted in the result list samples and the preview window. The

       color can be chosen here. Any Qt color string should work (ie red,

       color can be chosen here. Any Qt color string should work (ie red,

       #ff0000). The default is blue.

       #ff0000). The default is blue.

       to the whole Recoll application on startup. The default value is

       to the whole Recoll application on startup. The default value is

       empty, but there is a skeleton style sheet (recoll.qss) inside the

       empty, but there is a skeleton style sheet (recoll.qss) inside the

       /usr/share/recoll/examples directory. Using a style sheet, you can

       /usr/share/recoll/examples directory. Using a style sheet, you can

       change most recoll graphical parameters: colors, fonts, etc. See the

       change most recoll graphical parameters: colors, fonts, etc. See the

       sample file for a few simple examples.

       sample file for a few simple examples.

       search term inside the text before inserting it in the preview window

       search term inside the text before inserting it in the preview window

       involves quite a lot of processing, and can be disabled over the given

       involves quite a lot of processing, and can be disabled over the given

       text size to speed up loading.

       text size to speed up loading.

       as such inside the preview window. If this causes problems with the Qt

       as such inside the preview window. If this causes problems with the Qt

       HTML display, you can uncheck it to display the plain text version

       HTML display, you can uncheck it to display the plain text version

       instead.

       instead.

       preview window, Recoll tries to preserve some of the original text

       preview window, Recoll tries to preserve some of the original text

       line breaks and indentation. It can either use PRE HTML tags, which

       line breaks and indentation. It can either use PRE HTML tags, which

       will well preserve the indentation but will force horizontal scrolling

       will well preserve the indentation but will force horizontal scrolling

       for long lines, or use BR tags to break at the original line breaks,

       for long lines, or use BR tags to break at the original line breaks,

       which will let the editor introduce other line breaks according to the

       which will let the editor introduce other line breaks according to the

       window width, but will lose some of the original indentation. The

       window width, but will lose some of the original indentation. The

       third option has been available in recent releases and is probably now

       third option has been available in recent releases and is probably now

       the best one: use PRE tags with line wrapping.

       the best one: use PRE tags with line wrapping.

       the xdg-open utility will be used to open files when you click the

       the xdg-open utility will be used to open files when you click the

       Open link in the result list, instead of the application defined in

       Open link in the result list, instead of the application defined in

       mimeview. xdg-open will in term use your desktop preferences to choose

       mimeview. xdg-open will in term use your desktop preferences to choose

       an appropriate application.

       an appropriate application.

       these are mime types that will still be opened according to Recoll

       these are mime types that will still be opened according to Recoll

       preferences. This is useful for passing parameters like page numbers

       preferences. This is useful for passing parameters like page numbers

       or search strings to applications that support them (e.g. evince).

       or search strings to applications that support them (e.g. evince).

       This cannot be done with xdg-open which only supports passing one

       This cannot be done with xdg-open which only supports passing one

       parameter.

       parameter.

       started by the Open links inside the result list, for specific

       started by the Open links inside the result list, for specific

       document types.

       document types.

       document categories are displayed as a list or a set of buttons.

       document categories are displayed as a list or a set of buttons.

       search will be executed each time you enter a space in the simple

       search will be executed each time you enter a space in the simple

       search input field. This lets you look at the result list as you enter

       search input field. This lets you look at the result list as you enter

       new terms. This is off by default, you may like it or not...

       new terms. This is off by default, you may like it or not...

       frequently, checking the entries will get it to open when recoll

       frequently, checking the entries will get it to open when recoll

       starts.

       starts.

       tool stat between invocations. It normally starts with sorting

       tool stat between invocations. It normally starts with sorting

       disabled.

       disabled.

   Result list parameters:

       result list, and you may want to customize the font and/or font size.

       result list, and you may want to customize the font and/or font size.

       The rest of the fonts used by Recoll are determined by your generic Qt

       The rest of the fonts used by Recoll are determined by your generic Qt

       config (try the qtconfig command).

       config (try the qtconfig command).

       presentation of each result list entry. See the result list

       presentation of each result list entry. See the result list

       customisation section.

       customisation section.

       inserted at the end of the result page HTML header. More detail in the

       inserted at the end of the result page HTML header. More detail in the

       result list customisation section.

       result list customisation section.

       inside the result list. This should be specified as an strftime()

       inside the result list. This should be specified as an strftime()

       string (man strftime).

       string (man strftime).

       data, which are usually made of several snippets from different parts

       data, which are usually made of several snippets from different parts

       of the document, this defines the snippet separator, an ellipsis by

       of the document, this defines the snippet separator, an ellipsis by

       default.

       default.

   Search parameters:

       identical documents found in different places.

       identical documents found in different places.

       language. This listbox will let you chose among the stemming databases

       language. This listbox will let you chose among the stemming databases

       which were built during indexing (this is set in the main

       which were built during indexing (this is set in the main

       configuration file), or later added with recollindex -s (See the

       configuration file), or later added with recollindex -s (See the

       recollindex manual). Stemming languages which are dynamically added

       recollindex manual). Stemming languages which are dynamically added

       will be deleted at the next indexing pass unless they are also added

       will be deleted at the next indexing pass unless they are also added

       in the configuration file.

       in the configuration file.

       automatically built and added to simple searches when looking for Any

       automatically built and added to simple searches when looking for Any

       terms. This will give a relevance boost to the results where the

       terms. This will give a relevance boost to the results where the

       search terms appear as a phrase (consecutive and in order).

       search terms appear as a phrase (consecutive and in order).

       should not be included in automatic phrase searches for performance

       should not be included in automatic phrase searches for performance

       reasons. The parameter defines the cutoff percentage (percentage of

       reasons. The parameter defines the cutoff percentage (percentage of

       the documents where the term appears).

       the documents where the term appears).

       and display an abstract in place of an explicit abstract found within

       and display an abstract in place of an explicit abstract found within

       the document itself.

       the document itself.

       document abstracts (lists of snippets) when displaying the result

       document abstracts (lists of snippets) when displaying the result

       list. Abstracts are constructed by taking context from the document

       list. Abstracts are constructed by taking context from the document

       information, around the search terms.

       information, around the search terms.

       around each term occurrence.

       around each term occurrence.

       automatically get turned into ext:xxx file name suffix clauses when

       automatically get turned into ext:xxx file name suffix clauses when

       starting a query language query (ie: doc xls xlsx...). This will save

       starting a query language query (ie: doc xls xlsx...). This will save

       some typing for people who use file types a lot when querying.

       some typing for people who use file types a lot when querying.

   External indexes: This panel will let you browse for additional indexes

   External indexes: This panel will let you browse for additional indexes

   always implicitly active. If this is not desirable, you can set up your

   always implicitly active. If this is not desirable, you can set up your

   configuration so that it indexes, for example, an empty directory. An

   configuration so that it indexes, for example, an empty directory. An

   alternative indexer may also need to implement a way of purging the index

   alternative indexer may also need to implement a way of purging the index

   from stale data,

   from stale data,

   The result list presentation can be exhaustively customized by adjusting

   The result list presentation can be exhaustively customized by adjusting

   two elements:

   two elements:

   These can be edited from the Result list tab of the GUI configuration.

   These can be edited from the Result list tab of the GUI configuration.

   Newer versions of Recoll (from 1.17) use a WebKit HTML object by default

   Newer versions of Recoll (from 1.17) use a WebKit HTML object by default

   (this may be disabled at build time), and total customisation is possible

   (this may be disabled at build time), and total customisation is possible

   WebKit build), if there are restrictions to what you can do, they are

   WebKit build), if there are restrictions to what you can do, they are

   beyond this author's HTML/CSS/Javascript abilities... There are a few

   beyond this author's HTML/CSS/Javascript abilities... There are a few

   examples on the page about customising the result list on the Recoll web

   examples on the page about customising the result list on the Recoll web

   site.

   site.

      The paragraph format

   This is an arbitrary HTML string where the following printf-like %

   This is an arbitrary HTML string where the following printf-like %

   substitutions will be performed:

   substitutions will be performed:

       The associations are defined inside the mimeconf configuration file.

       The associations are defined inside the mimeconf configuration file.

       If a thumbnail for the file is found at the standard Freedesktop

       If a thumbnail for the file is found at the standard Freedesktop

       location, this will be displayed instead.

       location, this will be displayed instead.

   The format of the Preview, Edit, and Snippets links is <a href="P%N">, <a

   The format of the Preview, Edit, and Snippets links is <a href="P%N">, <a

   href="E%N"> and <a href="A%N"> where docnum (%N) expands to the document

   href="E%N"> and <a href="A%N"> where docnum (%N) expands to the document

   number inside the result page).

   number inside the result page).

   how they look.

   how they look.

   It is also possible to define the value of the snippet separator inside

   It is also possible to define the value of the snippet separator inside

   the abstract section.

   the abstract section.

3.2. Searching with the KDE KIO slave

3.2. Searching with the KDE KIO slave

  3.2.1. What's this

  3.2.1. What's this

   The Recoll KIO slave allows performing a Recoll search by entering an

   The Recoll KIO slave allows performing a Recoll search by entering an

   The instructions for building this module are located in the source tree.

   The instructions for building this module are located in the source tree.

   See: kde/kio/recoll/00README.txt. Some Linux distributions do package the

   See: kde/kio/recoll/00README.txt. Some Linux distributions do package the

   kio-recoll module, so check before diving into the build process, maybe

   kio-recoll module, so check before diving into the build process, maybe

   it's already out there ready for one-click installation.

   it's already out there ready for one-click installation.

  3.2.2. Searchable documents

  3.2.2. Searchable documents

   As a sample application, the Recoll KIO slave could allow preparing a set

   As a sample application, the Recoll KIO slave could allow preparing a set

   of HTML documents (for example a manual) so that they become their own

   of HTML documents (for example a manual) so that they become their own

     }

     }

 </script>

 </script>

  ....

  ....

 <body ondblclick="recollsearch()">

 <body ondblclick="recollsearch()">

3.3. Searching on the command line

3.3. Searching on the command line

   There are several ways to obtain search results as a text stream, without

   There are several ways to obtain search results as a text stream, without

   a graphical interface:

   a graphical interface:

   The first two methods work in the same way and accept/need the same

   The first two methods work in the same way and accept/need the same

   arguments (except for the additional -t to recoll). The query to be

   arguments (except for the additional -t to recoll). The query to be

   executed is specified as command line arguments.

   executed is specified as command line arguments.

 text/html       [file:///Users/uncrypted-dockes/projets/bateaux/ilur/comptes.html]      [comptes.html]  18593   bytes  

 text/html       [file:///Users/uncrypted-dockes/projets/bateaux/ilur/comptes.html]      [comptes.html]  18593   bytes  

 text/html       [file:///Users/uncrypted-dockes/projets/nautique/webnautique/articles/ilur1/index.html] [Constructio...

 text/html       [file:///Users/uncrypted-dockes/projets/nautique/webnautique/articles/ilur1/index.html] [Constructio...

 text/html       [file:///Users/uncrypted-dockes/projets/pagepers/index.html]    [psxtcl/writemime/recoll]...

 text/html       [file:///Users/uncrypted-dockes/projets/pagepers/index.html]    [psxtcl/writemime/recoll]...

 text/html       [file:///Users/uncrypted-dockes/projets/bateaux/ilur/factEtCie/recu-chasse-maree....

 text/html       [file:///Users/uncrypted-dockes/projets/bateaux/ilur/factEtCie/recu-chasse-maree....

3.4. The query language

3.4. The query language