scalable changelog redesign

This patch provides a rewritten changelog system that is designed to run
efficiently on both small and large wikis. The patch includes a plugin to
convert changelogs from the current format. The conversion is
non-destructive and happens automatically. For more information on the new
changelog format see "http://wiki.splitbrain.org/wiki:changelog".

Structure
In short the changelog is now stored in per-page changelog files, with a
recent changes cache. The recent changes cache is kept in
"/data/meta/_dokuwiki.changes" and trimmed daily. The per-page changelogs
are kept in "/data/meta/<ns>/<page_id>.changes" files. To preserve
revision information for revisions stored in the attic, the "*.changes"
files are not removed when their page is deleted. This allows the full
life-cycle of page creation, deletion, and reversion to be tracked.

Format
The changelog line format now uses a general "line type" field in place of
the special "minor" change syntax. There is also an extra field that can
be used to store arbitrary data associated with special line types. The
reverted line type (R) is a good example. There the extra field holds the
revision date used as the source for reverting the page. See the wiki for
the complete syntax description.

Code Notes
The changelog functions have been rewritten to load the whole file only if
it is small. For larger files, the function loads only the relevant
chunk(s). Parsed changelog lines are cached in memory to speed future
function calls.

getRevisionInfo
A binary search is used to locate the chunk expected to contain the
requested revision. The whole chunk is parsed, and adjacent lines are
optimistically cached to speed consecutive calls.

getRevisions
Reads the changelog file backwards (newest first) in chunks until the
requested number of lines have been read. Parsed changelog lines are
cached for subsequent calls to getRevisionInfo. Because revisions are read
from the changelog they are no longer guaranteed to exist in the attic.

(Note: Even with lines of arbitrary length getRevisionInfo and
getRevisions never split changelog lines while reading. This is done by
sliding the "file pointer" forward to the end of a line after each blind
seek.)

isMinor
Removed. To detect a minor edit check the type as follows:
$parsed_logline['type']

darcs-hash:20060830182753-05dcb-1c5ea17f581197a33732a8d11da223d809c03506.gz

1 files changed, 68 insertions, 1 deletions

diff --git a/lib/exe/indexer.php b/lib/exe/indexer.php
index 2728e5665..d65707911 100644
--- a/lib/exe/indexer.php
+++ b/lib/exe/indexer.php
@@ -27,7 +27,7 @@ if(@ignore_user_abort()){
 if(!$_REQUEST['debug']) ob_start();
 
 // run one of the jobs
-runIndexer() or metaUpdate() or runSitemapper();
+runIndexer() or metaUpdate() or runSitemapper() or runTrimRecentChanges();
 if($defer) sendGIF();
 
 if(!$_REQUEST['debug']) ob_end_clean();
@@ -36,6 +36,73 @@ exit;
 // --------------------------------------------------------------------
 
 /**
+ * Trims the recent changes cache (or imports the old changelog) as needed.
+ *
+ * @author Ben Coburn <btcoburn@silicodon.net>
+ */
+function runTrimRecentChanges() {
+    global $conf;
+
+    // Import old changelog (if needed)
+    // Uses the imporoldchangelog plugin to upgrade the changelog automaticaly.
+    // FIXME: Remove this from runTrimRecentChanges when it is no longer needed.
+    if (isset($conf['changelog_old']) &&
+        file_exists($conf['changelog_old']) && !file_exists($conf['changelog']) &&
+        !file_exists($conf['changelog'].'_importing') && !file_exists($conf['changelog'].'_tmp')) {
+            $tmp = array(); // no event data
+            trigger_event('TEMPORARY_CHANGELOG_UPGRADE_EVENT', $tmp);
+            return true;
+    }
+
+    // Trim the Recent Changes
+    // Trims the recent changes cache to the last $conf['changes_days'] recent
+    // changes or $conf['recent'] items, which ever is larger.
+    // The trimming is only done once a day.
+    if (file_exists($conf['changelog']) &&
+        (filectime($conf['changelog'])+86400)<time() &&
+        !file_exists($conf['changelog'].'_tmp')) {
+            io_lock($conf['changelog']);
+            $lines = file($conf['changelog']);
+            if (count($lines)<$conf['recent']) {
+                // nothing to trim
+                io_unlock($conf['changelog']);
+                return true;
+            }
+            // trim changelog
+            io_saveFile($conf['changelog'].'_tmp', ''); // presave tmp as 2nd lock
+            $kept = 0;
+            $trim_time = time() - $conf['recent_days']*86400;
+            $out_lines = array();
+            // check lines from newest to oldest
+            for ($i = count($lines)-1; $i >= 0; $i--) {
+                $tmp = parseChangelogLine($lines[$i]);
+                if ($tmp===false) { continue; }
+                if ($tmp['date']>$trim_time || $kept<$conf['recent']) {
+                    array_push($out_lines, implode("\t", $tmp)."\n");
+                    $kept++;
+                } else {
+                    // no more lines worth keeping
+                    break;
+                }
+            }
+            io_saveFile($conf['changelog'].'_tmp', implode('', $out_lines));
+            unlink($conf['changelog']);
+            if (!rename($conf['changelog'].'_tmp', $conf['changelog'])) {
+                // rename failed so try another way...
+                io_unlock($conf['changelog']);
+                io_saveFile($conf['changelog'], implode('', $out_lines));
+                unlink($conf['changelog'].'_tmp');
+            } else {
+                io_unlock($conf['changelog']);
+            }
+            return true;
+    }
+
+    // nothing done
+    return false;
+}
+
+/**
  * Runs the indexer for the current page
  *
  * @author Andreas Gohr <andi@splitbrain.org>