From 1e22966bd31726c5828242b732d4b08bb47bdf45 Mon Sep 17 00:00:00 2001 From: Jean-Francois Dockes Date: Thu, 12 Apr 2018 10:35:22 +0200 Subject: [PATCH] doc for partial incremental indexing --- src/doc/user/recoll.conf.xml | 7 +++ src/doc/user/usermanual.html | 95 ++++++++++++++++++++++++++---------- src/doc/user/usermanual.xml | 61 ++++++++++++++--------- src/sampleconf/recoll.conf | 7 +++ 4 files changed, 120 insertions(+), 50 deletions(-) diff --git a/src/doc/user/recoll.conf.xml b/src/doc/user/recoll.conf.xml index 13715592..7a06f7e7 100644 --- a/src/doc/user/recoll.conf.xml +++ b/src/doc/user/recoll.conf.xml @@ -9,6 +9,13 @@ directories to recursively index. Default to ~ (indexes $HOME). You can use symbolic links in the list, they will be followed, independantly of the value of the followLinks variable. + +monitordirs +(1.25) Space-separated list of +files or directories to monitor for updates. When running +the real-time indexer, this allows monitoring only a subset of the whole +indexed area. The elements must be included in the tree defined by the +'topdirs' members. skippedNames Files and directories which should be ignored. diff --git a/src/doc/user/usermanual.html b/src/doc/user/usermanual.html index 641d0f24..0905ee21 100644 --- a/src/doc/user/usermanual.html +++ b/src/doc/user/usermanual.html @@ -92,11 +92,11 @@ alink="#0000FF"> "#RCL.INDEXING.INTRODUCTION.CONFIG">Configurations, multiple indexes
2.1.3. Document types
+ "#idm223">Document types
2.1.4. Indexing failures
+ "#idm264">Indexing failures
2.1.5. Recovery
+ "#idm276">Recovery
2.2. saké, mate / maté).

-

Recoll versions 1.18 - and newer can optionally store the raw terms, without - accent stripping or case conversion. In this configuration, - default searches will behave as before, but it is possible - to perform searches sensitive to case and diacritics. This - is described in more detail in the Recoll can optionally + store the raw terms, without accent stripping or case + conversion. In this configuration, default searches will + behave as before, but it is possible to perform searches + sensitive to case and diacritics. This is described in more + detail in the section about index case and diacritics sensitivity.

@@ -783,7 +783,7 @@ alink="#0000FF">

Recoll indexing can - be performed along two different modes:

+ be performed along two main modes:

    @@ -807,10 +807,8 @@ alink="#0000FF"> as a file is created or changed. recollindex runs as a daemon and uses a file system alteration - monitor such as inotify, Fam or Gamin to detect file + monitor (e.g. inotify) to detect file changes.

@@ -821,6 +819,14 @@ alink="#0000FF"> documentation directory, and real time indexing on a small home directory). Monitoring a big file system tree can consume significant system resources.

+

With Recoll 1.25 and + newer, it is also possible to set up an index so that + only a subset of the tree will be monitored and the rest + will be covered by batch/incremental indexing. (See the + details in the Real time indexing + section.

The choice of method and the parameters used can be configured from the recoll GUI: @@ -834,12 +840,13 @@ alink="#0000FF"> later restart of indexing will mostly resume from where things stopped (the file tree walk has to be restarted from the beginning).

-

When the real time indexer is running, only a stop - operation is available from the menu. When no indexing is - running, you have a choice of updating the index or - rebuilding it (the first choice only processes changed - files, the second one zeroes the index before starting so - that all files are processed).

+

When the real time indexer is running, two operations + are available from the menu: 'Stop' and 'Trigger + incremental pass'. When no indexing is running, you have + a choice of updating the index or rebuilding it (the + first choice only processes changed files, the second one + zeroes the index before starting so that all files are + processed).

@@ -910,8 +917,8 @@ alink="#0000FF">
-

2.1.3. Document types

+

2.1.3. Document types

@@ -1008,8 +1015,8 @@ alink="#0000FF">
-

2.1.4. Indexing failures

+

2.1.4. Indexing failures

@@ -1044,8 +1051,8 @@ alink="#0000FF">
-

2.1.5. Recovery

+

2.1.5. Recovery

@@ -2111,7 +2118,7 @@ alink="#0000FF"> "application">X11 session monitoring (else the daemon will not start).

By default, the messages from the indexing daemon will - be setn to the same file as those from the interactive + be sent to the same file as those from the interactive commands (logfilename). You may want to change this by setting the daemlogfilename and system resources. You probably do not want to enable it if your system is short on resources. Periodic indexing is adequate in most cases.

+

As of Recoll 1.25, you + can set the monitordirs + configuration variable to specify that only a subset of + your indexed files will be monitored for instant indexing. + In this situation, an incremental pass on the full tree can + be triggered by either restarting the indexer, or just + running the recollindex, which will + notify the running process. The recoll GUI also has a + menu entry for this.

Increasing resources for inotify

@@ -7985,6 +8004,17 @@ for i in range(nres): of the followLinks variable.

monitordirs
+
+

(1.25) Space-separated list of files or + directories to monitor for updates. When running + the real-time indexer, this allows monitoring + only a subset of the whole indexed area. The + elements must be included in the tree defined by + the 'topdirs' members.

+
+
skippedNames
@@ -8931,6 +8961,17 @@ for i in range(nres): have custom fields.

idxtexttruncatelen
+
+

Truncation length for all document texts. Only + index the beginning of documents. This is not + recommended except if you are sure that the + interesting keywords are at the top and have + severe disk space issues.

+
+
aspellLanguage
diff --git a/src/doc/user/usermanual.xml b/src/doc/user/usermanual.xml index 89311c66..53949c25 100644 --- a/src/doc/user/usermanual.xml +++ b/src/doc/user/usermanual.xml @@ -226,13 +226,12 @@ diacritics (sake / saké, mate / maté). - &RCL; versions 1.18 and newer can optionally store the raw - terms, without accent stripping or case conversion. In this - configuration, default searches will behave as before, but it is - possible to perform searches sensitive to case and - diacritics. This is described in more detail - in the section about index - case and diacritics sensitivity. + &RCL; can optionally store the raw terms, without accent + stripping or case conversion. In this configuration, default searches + will behave as before, but it is possible to perform searches + sensitive to case and diacritics. This is described in more detail in + the section about index case + and diacritics sensitivity. &RCL; has many parameters which define exactly what to index, and how to classify and decode the source @@ -327,7 +326,7 @@ Indexing modes - &RCL; indexing can be performed along two different modes: + &RCL; indexing can be performed along two main modes: @@ -343,18 +342,16 @@ <link linkend="RCL.INDEXING.MONITOR">Real - time indexing:</link> - indexing takes place as soon as a file is created or - changed. recollindex runs as a daemon - and uses a file system alteration monitor such as - inotify, - Fam or - Gamin - to detect file changes. - + time indexing: indexing takes place as + soon as a file is created or + changed. recollindex runs as a daemon and + uses a file system alteration monitor + (e.g. inotify) to detect file + changes. + The choice between the two methods is mostly a matter of preference, and they can be combined by setting up multiple indexes (ie: use periodic indexing on a big documentation @@ -362,6 +359,12 @@ directory). Monitoring a big file system tree can consume significant system resources. + With &RCL; 1.25 and newer, it is also possible to set up an + index so that only a subset of the tree will be monitored and the + rest will be covered by batch/incremental indexing. (See the + details in the Real time + indexing section. + The choice of method and the parameters used can be configured from the recoll GUI: @@ -378,11 +381,12 @@ mostly resume from where things stopped (the file tree walk has to be restarted from the beginning). - When the real time indexer is running, only a stop operation - is available from the menu. When no indexing is running, you have - a choice of updating the index or rebuilding it (the first choice - only processes changed files, the second one zeroes the index - before starting so that all files are processed). + When the real time indexer is running, two operations are + available from the menu: 'Stop' and 'Trigger incremental pass'. + When no indexing is running, you have a choice of updating the + index or rebuilding it (the first choice only processes changed + files, the second one zeroes the index before starting so that all + files are processed). @@ -1456,7 +1460,7 @@ session monitoring (else the daemon will not start). By default, the messages from the indexing daemon will be - setn to the same file as those from the interactive commands + sent to the same file as those from the interactive commands (logfilename). You may want to change this by setting the daemlogfilename and daemloglevel configuration parameters. Also @@ -1482,6 +1486,17 @@ your system is short on resources. Periodic indexing is adequate in most cases. + As of &RCL; 1.25, you can set the monitordirs + configuration variable to specify that only a subset of your indexed + files will be monitored for instant indexing. In this situation, an + incremental pass on the full tree can be triggered by either + restarting the indexer, or just running the + recollindex, which will notify the running + process. The recoll GUI also has a menu entry for + this. + + Increasing resources for inotify On Linux systems, monitoring a big tree may need increasing the resources available to inotify, which are diff --git a/src/sampleconf/recoll.conf b/src/sampleconf/recoll.conf index 24fd2d62..e1072507 100644 --- a/src/sampleconf/recoll.conf +++ b/src/sampleconf/recoll.conf @@ -20,6 +20,13 @@ # independantly of the value of the followLinks variable. topdirs = ~ +# (1.25) Space-separated list of +# files or directories to monitor for updates.When running +# the real-time indexer, this allows monitoring only a subset of the whole +# indexed area. The elements must be included in the tree defined by the +# 'topdirs' members. +#monitordirs= + # # # Files and directories which should be ignored.