Open Source Projects Europe

= Recoll WebUI Apache installation from scratch 

The https://github.com/koniu/recoll-webui[Recoll WebUI] offers an
alternative, WEB-based, interface for querying a Recoll index.

It can be quite useful to extend the use of a shared index to multiple
workstations, without the need for a local Recoll installation and shared
data storage.

The Recoll WebUI is based on the
http://bottlepy.org/docs/dev/index.html[Bottle Python framework], which has
a built-in WEB server, and the simplest deployment approach is to run it
standalone. However the built-in server is restricted to handling one
request at a time, which is problematic in multi-user situations,
especially because some requests, like extracting a result list into a CSV
file, can take a significant amount of time.

The Bottle framework can work with several multi-threading Python HTTP
server libraries, but, given the limitations of the Recoll Python module
and the Python interpreter itself, this will not yield optimal performance,
and, especially can't efficiently leverage the now ubiquitous
multiprocessors.

In multi-user situations, you can get better performance and ease of use
from the Recoll WebUI by running it under Apache rather than as a
standalone process. With this approach, a few requests per second can
easily be handled even in the presence of long-running ones.

Neither Recoll nor the WebUI are optimized for high multi-user load, and it
would be very unwise to use them as the search interface to a busy WEB
site.

The instructions about using the WebUI under Apache as given in the
repository README are a bit terse, and are missing a few details,
especially ones which impact performance.

Here follows the synopsis of two WebUI installations on initially
Apache-less Ubuntu (14.04) and DragonFly BSD systems. The first should
extend easily to other Debian-based systems, the second at least to
FreeBSD. rpm-based systems are left as an exercise to the reader, at least
for now...


CAUTION: THE CONFIGURATIONS DESCRIBED HAVE NO ACCESS CONTROL. ANYONE WITH
ACCESS TO THE NETWORK WHERE THE SERVER IS LOCATED CAN RETRIEVE ANY
DOCUMENT.

== On a Debian/Ubuntu system

=== Install recoll 

    sudo apt-get install recoll python-recoll

Configure the indexing and check that the normal search works (I spent
quite a lot of time trying to understand why the WebUI did not work, when
in fact it was the normal recoll configuration which was broken and the
regular search did not work either).

Take care to be logged in as the user you want to run the web search as
while you do this.


=== Install the WebUI

Clone the github repository, or extract the master tar installation, and
move it to '/var/www/recoll-webui-master/'. Take care that it is read/execute
accessible by your user.

=== Install Apache and mod-wsgi


    sudo apt-get install apache2 libapache2-mod-wsgi

I then got the following message:

    AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message

To clear it, I added a ServerName directive to the apache config, maybe you
won't need it. Edit '/etc/apache2/sites-available/000-default.conf' and add
the following at the top (globally). Things work without this fix anyway,
this is just to suppress the error message. You probably need to adjust the
address or use a real host name:

    ServerName 192.168.4.6


Edit '/etc/apache2/mods-enabled/wsgi.conf', add the following at the end of
the "IfModule" section.

Change the user ('dockes' in the example) taking care that he is the one who
owns the index ('.recoll' is in his home directory).

    WSGIDaemonProcess recoll user=dockes group=dockes \
        threads=1 processes=5 display-name=%{GROUP} \
        python-path=/var/www/recoll-webui-master
    WSGIScriptAlias /recoll /var/www/recoll-webui-master/webui-wsgi.py
    <Directory /var/www/recoll-webui-master>
            WSGIProcessGroup recoll
            Order allow,deny
            allow from all
    </Directory>

NOTE: the Recoll WebUI application is mostly single-threaded, so it is of
little use (and may actually be counter-productive in some cases) to
specify multiple threads on the WSGIDaemonProcess line. Specify multiple
processes instead to put multiple CPUs to work on simultaneous requests.


Then run the following to restart apache:

    sudo apachectl restart

The Recoll WebUI should now be accessible. on 'http://my.server.com/recoll/'

NOTE: Take care that you need a '/' at the end of the URL used to access
the search (use: 'http://my.server.com/recoll/', not
'http://my.server.com/recoll'), else files other than the script itself are
not found (the page looks weird and the search does not work).

CAUTION: THERE IS NO ACCESS CONTROL. ANYONE WITH ACCESS TO THE NETWORK
WHERE THE SERVER IS LOCATED CAN RETRIEVE ANY DOCUMENT.

== Variant for BSD/ports

=== Packages

As root:

    pkg install recoll


Do what you need to do to configure the indexing and check that the normal
search works.

Take care to be logged in as the user you want to run the web search as
while you do this.

    pkg install apache24

Add apache24_enable="YES" in /etc/rc.conf

    pkg install ap24-mod_wsgi4
    pkg install git

=== Clone the webui repository

    cd /usr/local/www/apache24/
    git clone https://github.com/koniu/recoll-webui.git recoll-webui-master

Important: most input handler helper applications (e.g. 'pdftotext') are
installed in '/usr/local/bin' which is not in the PATH as seen by Apache
(at least on DragonFly). The simplest way to fix this is to modify the
launcher module for the webui app so that it fixes the PATH.

Edit 'recoll-webui-master/webui-wsgi.py' and add the following line after
the 'import os' line:

    os.environ['PATH'] = os.environ['PATH'] + ':' + '/usr/local/bin'



=== Configure apache

Edit /usr/local/etc/apache24/modules.d/270_mod_wsgi.conf

Uncomment the LoadModule line, and add the directives to alias /recoll/ to
the webui script.

Change the user (dockes in the example) taking care that he is the one who
owns the index (.recoll is in his home directory).

Contents of the file:

    ## $FreeBSD$
    ## vim: set filetype=apache:
    ##
    ## module file for mod_wsgi
    ##
    ## PROVIDE: mod_wsgi
    ## REQUIRE:
    
    LoadModule wsgi_module        libexec/apache24/mod_wsgi.so
    
    WSGIDaemonProcess recoll user=dockes group=dockes \
        threads=1 processes=5 display-name=%{GROUP} \
        python-path=/usr/local/www/apache24/recoll-webui-master/
    WSGIScriptAlias /recoll /usr/local/www/apache24/recoll-webui-master/webui-wsgi.py
    
    <Directory /usr/local/www/apache24/recoll-webui-master>
            WSGIProcessGroup recoll
            Require all granted
    </Directory>

=== Restart apache

As root:

    apachectl restart