recoll / Code / Diff of /src/README

Diff of /src/README [34070e] .. [821409]

Switch to unified view


...
                            Chapter 1. Introduction

1.1. Giving it a try

   If you do not like reading manuals (who does?) and would like to give
   Recoll a try, just install the application and start the recoll graphical
   user interface (GUI), which will ask to index your home directory by
   default, allowing you to search immediately after indexing completes.

   Do not do this if your home directory contains a huge number of documents
   and you do not want to wait or are very short on disk space. In this case,
   you may first want to customize the configuration to restrict the indexed
   area.
...
   default configuration will index your home directory with default
   parameters and should be sufficient for giving Recoll a try, but you may
   want to adjust it later, which can be done either by editing the text
   files or by using configuration menus in the recoll GUI

   The indexing process is started automatically the first time you execute
   the recoll GUI. Indexing can also be performed by executing the
   recollindex command.

   Searches are usually performed inside the recoll GUI, which has many
   options to help you find what you are looking for. However, there are
   other ways to perform Recoll searches: mostly a command line interface, a
   Python programming interface, a KDE KIO slave module, and a Ubuntu Unity
   Lens module.

     ----------------------------------------------------------------------

                              Chapter 2. Indexing

...
   Recoll knows about quite a few different document types. The parameters
   for document types recognition and processing are set in configuration
   files.

   Most file types, like HTML or word processing files, only hold one
   document. Some file types, like email folders or zip archives, can hold
   many individually indexed documents, which may in turn be themselves
   compound ones. Such hierarchies can go quite deep, and Recoll can process,
   for example, an ms-word document stored as an attachment to an email
   message inside an email folder archived in a zip file...


   Recoll indexing processes plain text, HTML, OpenDocument
   (Open/LibreOffice), email formats, and a few others internally.

   Other file types (ie: postscript, pdf, ms-word, rtf ...) need external
   applications for preprocessing. The list is in the installation section.
   After every indexing operation, Recoll updates a list of commands that
   would be needed for indexing existing files types. This list can be
   displayed by selecting the menu option File->Show Missing Helpers in the
   recoll GUI. It is stored in the missing text file inside the configuration
   directory.

   Without further configuration, Recoll will index all appropriate files
   from your home directory, with a reasonable set of defaults.

   In some cases, it may be interesting to index different areas of the file
...
   the documents. It may also be much smaller if the documents contain a lot
   of images or other non-indexed data (an extreme example being a set of mp3
   files where only the tags would be indexed).

   Of course, images, sound and video do not increase the index size, which
   means that nowadays (2012), typically, even a big index will be negligible
   against the total amount of data on the computer.

   The index data directory (xapiandb) only contains data that can be
   completely rebuilt by an index run (as long as the original documents
   exist), and it can always be destroyed safely.

...

   Most parameters for a given indexing configuration can be set from a
   recoll GUI running on this configuration (either as default, or by setting
   RECOLL_CONFDIR or the -c option.)

   The interface is started from the Preferences->Indexing Configuration menu
   entry. It is divided in three tabs, Global parameters, Local parameters,
   and Beagle web history, which is explained in the next section.

   The first tab allows setting global variables, like the lists of top
   directories, skipped paths, or stemming languages.

   The second tab allows setting variables that can be redefined for
   subdirectories. This second tab has an initially empty list of
   customisation directories, to which you can add. The variables are then
   set for the currently selected directory (or at the top level if the empty
   line is selected).

   The meaning for most entries in the interface is self-evident and
   documented by a ToolTip popup on the text label. For more detail, you will
   need to refer to the configuration section of this guide.

...

   If the recoll program finds no index when it starts, it will automatically
   start indexing (except if canceled).

   The recollindex indexing process can be interrupted by sending an
   interrupt (Ctrl-C, SIGINT) or terminate (SIGTERM) signal. Some time may
   elapse before the process exits, because it needs to properly flush and
   close the index. This can also be done from the recoll GUI File->Stop
   Indexing menu entry.

   After such an interruption, the index will be somewhat inconsistent
   because some operations which are normally performed at the end of the
   indexing pass will have been skipped (for example, the stemming and
   spelling databases will be inexistant or out of date). You just need to
   restart indexing at a later time to restore consistency. The indexing will
   restart at the interruption point (the full file tree will be traversed,
   but files that were indexed up to the interruption and are still up to
   date will not need to be reindexed).
...
       toolbox bar icon) has multiple entry fields, which you may use to
       build a logical condition, with additional filtering on file type and
       location in the file system.

   In most cases, you can enter the terms as you think them, even if they
   contain embedded punctuation or other non-textual characters. For example,
   Recoll can handle things like email addresses, or arbitrary cut and paste
   from another text window, punctation and all.

   The main case where you should enter text differently from how it is
   printed is for east-asian languages (Chinese, Japanese, Korean). Words
   composed of single or multiple characters should be entered separated by
...
   This entry is mainly useful for email attachments and permits viewing the
   message to which the document is attached. Note that the entry will also
   appear for an email which is part of an mbox folder file, but that you
   can't actually visualize the folder (there will be an error dialog if you
   try). Recoll is unfortunately not yet smart enough to disable the entry in
   this case. In other cases, the Open option makes sense, for example to
   start a chm viewer on the parent document for a help page.

     ----------------------------------------------------------------------

  3.1.3. The result table
...
   will open a new window for side by side viewing).

   Starting another search and requesting a preview will create a new preview
   window. The old one stays open until you close it.

   You can close a preview tab by typing Ctrl-W (Ctrl + W) in the window.
   Closing the last tab for a window will also close the window.

   Of course you can also close a preview window by using the window manager
   button in the top of the frame.

   You can display successive or previous documents from the result list
...
   area or by clicking into the Search for: text field and entering the
   search string. You can then use the Next and Previous buttons to find the
   next/previous occurrence. You can also type F3 inside the text area to get
   to the next occurrence.

   If you have a search string entered and you use Ctrl-Up/Ctrl-Down to
   browse the results, the search is initiated for each successive document.
   If the string is found, the cursor will be positioned at the first
   occurrence of the search string.

   A right-click menu in the text area allows switching between displaying
   the main text or the contents of fields associated to the document (ie:
   author, abtract, etc.). This is especially useful in cases where the term
   match did not occur in the main text but in one of the fields.

   You can print the current preview window contents by typing Ctrl-P (Ctrl +
   P) in the window text.

     ----------------------------------------------------------------------

  3.1.5. Complex/advanced search

...

   Forced opening of a preview window. You can use Shift+Click on a result
   list Preview link to force the creation of a preview window instead of a
   new tab in the existing one.

   Closing previews. Entering Ctrl-W in a tab will close it (and, for the
   last tab, close the preview window). Entering Esc will close the preview
   window and all its tabs.

   Printing previews. Entering Ctrl-P in a preview window will print the
   currently displayed text.

   Quitting. Entering Ctrl-Q almost anywhere will close the application.

     ----------------------------------------------------------------------

  3.1.11. Customizing the search interface

...

     * Style sheet: The name of a Qt style sheet text file which is applied
       to the whole Recoll application on startup. The default value is
       empty, but there is a skeleton style sheet (recoll.qss) inside the
       /usr/share/recoll/examples directory. Using a style sheet, you can
       change most recoll graphical parameters: colors, fonts, etc. See the
       sample file for a few simple examples.

     * Maximum text size highlighted for preview Inserting highlights on
       search term inside the text before inserting it in the preview window
       involves quite a lot of processing, and can be disabled over the given
...
   displayed.

   No more detail will be given about the header part (only useful with the
   WebKit build), if there are restrictions to what you can do, they are
   beyond this author's HTML/CSS/Javascript abilities... There are a few
   examples on the page about customising the result list on the Recoll web
   site.

     ----------------------------------------------------------------------

      3.1.11.1.1. The paragraph format
...
   ie: the From: header, for an email message), and containing either beatles
   or lennon and either live or unplugged but not potatoes (in any part of
   the document).

   An element is composed of an optional field specification, and a value,
   separated by a colon. Example: Beatles, author:balzac, dc:title:grandet

   The colon, if present, means "contains". Xesam defines other relations,
   which are not supported for now.

   All elements in the search entry are normally combined with an implicit
...

   As usual, words inside quotes define a phrase (the order of words is
   significant), so that title:"prejudice pride" is not the same as
   title:prejudice title:pride, and is unlikely to find a result.

   Modifiers can be set on a phrase clause, for example to specify a
   proximity search (unordered). See the modifier section.

   Recoll currently manages the following default fields:

     * title, subject or caption are synonyms which specify data to be
...
       field and only one value makes sense in a query (you can't use
       dir:dir1 OR dir:dir2). Relative paths make sense, for example,
       dir:share/doc would match either /usr/share/doc or
       /usr/local/share/doc

     * size for filtering the results on file size. Example: size<10000. You
       can use <, > or = as operators. You can specify a range like the
       following: size>100 size<1000. The usual k/K, m/M, g/G, t/T can be
       used as (decimal) multipliers. Ex: size>1k to search for files bigger
       than 1000 bytes.

...
       time. Periods are specified as PnYnMnD. The n numbers are the
       respective numbers of years, months or days, any of which may be
       missing. Dates are specified as YYYY-MM-DD. The days and months parts
       may be missing. If the / is present but an element is missing, the
       missing element is interpreted as the lowest or highest date in the
       index. Examples:

          * 2001-03-01/2002-05-01 the basic syntax for an interval of dates.

          * 2001-03-01/P1Y2M the same specified with a period.

...
   the filter if the operation is for indexing or previewing. Some filters
   use this to output a slightly different format, for example stripping
   uninteresting repeated keywords (ie: Subject: for email) when indexing.
   This is not essential.

   You should look to one of the simple filters, for example rclps for a
   starting point.

   Don't forget to make your filter executable before testing !

     ----------------------------------------------------------------------
...
   In all cases, the strict software dependancies (ie on Xapian or iconv)
   will be automatically satisfied, you should not have to worry about them.

   You will only have to check or install supporting applications for the
   file types that you want to index beyond those that are natively processed
   by Recoll (text, HTML, email files, and a few others).

   You should also maybe have a look at the configuration section (but this
   may not be necessary for a quick test with default parameters). Most
   parameters can be more conveniently set from the GUI interface.

...

     * Midi karaoke files need Python and the Midi module

     * Konqueror webarchive format with Python (uses the Tarfile module).

     * mimehtml web archive format (support based on the email filter, which
       introduces some mild weirdness, but still usable).

   Text, HTML, email folders, and Scribus files are processed internally. Lyx
   is used to index Lyx files. Many filters need iconv and the standard sed
   and awk.

     ----------------------------------------------------------------------

...
   expanded to the name of the user's home directory, as a shell would do.

   White space is used for separation inside lists. List elements with
   embedded spaces can be quoted using double-quotes.

   Encoding issues. Most of the configuration parameters are plain ASCII. Two
   particular sets of values may cause encoding issues:

     * File path parameters may contain non-ascii characters and should use
       the exact same byte values as found in the file system directory.
       Usually, this means that the configuration file should use the system
       default locale encoding.

     * The unac_except_trans parameter should be encoded in UTF-8. If your
       system locale is not UTF-8, and you need to also specify non-ascii
       file paths, this poses a difficulty because common text editors cannot
       handle multiple encodings in a single file. In this relatively
       unlikely case, you can edit the configuration file as two separate
       text files with appropriate encodings, and concatenate them to create
       the complete configuration.

     ----------------------------------------------------------------------

  5.4.1. Main configuration file

   recoll.conf is the main configuration file. It defines things like what to
...
           a directory in topdirs might match and would still be indexed).

           The list in the default configuration does not exclude hidden
           directories (names beginning with a dot), which means that it may
           index quite a few things that you do not want. On the other hand,
           email user agents like thunderbird usually store messages in
           hidden directories, and you probably want this indexed. One
           possible solution is to have .* in skippedNames, and add things
           like ~/.thunderbird or ~/.evolution in topdirs.

           Not even the file names are indexed for patterns in this list. See
           the recoll_noindex variable in mimemap for an alternative approach
           which indexes the file names.

...
           character set definition (ie: plain text files). This can be
           redefined for any sub-directory. If it is not set at all, the
           character set used is the one defined by the nls environment
           (LC_ALL, LC_CTYPE, LANG), or iso8859-1 if nothing is set.

   unac_except_trans

           This is a list of characters, encoded in UTF-8, which should be
           handled specially when converting text to unaccented lowercase.
           For example, in Swedish, the letter a with diaeresis has full
           alphabet citizenship and should not be turned into an a. Each
           element in the space-separated list has the special character as
           first element and the translation following. The handling of both
           the lowercase and upper-case versions of a character should be
           specified, as appartenance to the list will turn-off both standard
           accent and case processing. Example for Swedish:

 unac_except_trans =  aaaa AAaa a:a: A:a: o:o: O:o:
            

           Note that the translation is not limited to a single character,
           you could very well have something like u:ue in the list.

           This parameter can't be defined for subdirectories, it is global,
           because there is no way to do otherwise when querying. If you have
           document sets which would need different values, you will have to
           index and query them separately.

   maildefcharset

           This can be used to define the default character set specifically
           for email messages which don't specify it. This is mainly useful
           for readpst (libpst) dumps, which are utf-8 but do not say so.

   localfields

           This allows setting fields for all documents under a given
...
           used inside the [prefixes] and [stored] sections

   filter-specific sections

           Some filters may need specific configuration for handling fields.
           Only the email message filter currently has such a section (named
           [mail]). It allows indexing arbitrary email headers in addition to
           the ones indexed by default. Other such sections may appear in the
           future.

   Here follows a small example of a personal fields file. This would extract
   a specific email header and use it as a searchable field, with data
   displayable inside result lists. (Side note: as the email filter does no
   decoding on the values, only plain ascii headers can be indexed, and only
   the first occurrence will be used for headers that occur several times).

 [prefixes]
 # Index mailmytag contents (with the given prefix)