Some fixes for Polish translation.

[deb/packages.git] / BACKEND

diff --git a/BACKEND b/BACKEND
index cde9ef4d4e4fd554057022d7eb7934a1e1d590c8..7c87b1c78f3855369b5275dfe70272b6cec1fa73 100644 (file)
--- a/BACKEND
+++ b/BACKEND
@@ -1,5 +1,3 @@
-$Id$
-
 Most package data is available via sleepycat databases. Those have been
 specifically crafted for efficient lookup, and reasonable times to generate
 them, so that both the daily cronjobs and the actual pagerequests take only a
@@ -20,14 +18,14 @@ Generated by means of Packages.gz files:
 |  value: \0 separated tuples of "archive suite arch component section priority version shortdescription"
 |          (so you can split on spaces in 8 pieces, but need to not split further
 |          because shortdescription can have spaces)
-|          arch can also be 'virtual', with c/s/p/v being undefined then, and
-|          shortdescription being a space-separated list of packages providing
-|          the package that is the key
   Notes: - maybe add did right before shortdescription?
-         - TODO: make sure for each (archive,suite), newest package is shown
-           first, and all newest versions for each such section is first, so
-           that one can efficiently lookup just the newest entry for a given
-           (archive,suite)
+                - for each suite, newest package is shown first, and (suite,
+                  architecture) is unique, the newest one is choosen. Once you find
+                  the right suite, you know you've got the newest, once you found
+                  your (suite,arch), you know you've found the only unique such entry
+                - The very first element is different (TODO: maybe should be
+                  different DB then?), a \01 separated hash of suite -> provided-by,
+                  like "suite1\01prov1 prov2\01suite2\01prov1"
 
 | package_postfixes.db:
 |  key: a postfix string of a package name
@@ -41,11 +39,6 @@ Generated by means of Packages.gz files:
 |  key: "packagename version arch"
 |  value: a unique description id, did
  
-| descriptions.txt:
-|  on each line:
-|    description with strange characters mangled for proper substring
-|    searching, linenumber being the did
-
 | descriptions.db:
 |  key: did
 |  value: description, first line being short, the rest being long [no
@@ -93,3 +86,24 @@ Generated by means of Sources.gz files:
 |         - files: \01 separated list of "md5 size filename"
   Note: different key from packages_all, is that needed?
 
+*********************************************************
+Generated by means of Contents-$arch.gz files:
+*********************************************************
+
+This one is tricky, because it deals with about 1G of raw uncompressed data
+per suite. Not all data is updated every day though, so dealing with that
+efficiently pays off.
+
+Each sourcefile will create a filelists_$suite_$arch.db, with prefix
+compression. The last updated one will have a symlink from _all.db to it, to
+help filelist queries for 'all' packages.
+
+reverse_$suite_$arch.txt will be the reversed pathnames for that file,
+lowercased, sorted, with packagename:arch following it.
+
+For each suite, the suite-wide indices can then be updated by reading the 11
+or so reverse_$suite_$arch.txt in sorted order with sort -m. Same pathnames
+can be put together, and stored in reverse_$suite.db; filenames are then also
+incidently coming by grouped uniquely (but reverse sorted, not normal sorted),
+and can be written out linearly to filenames_$suite.txt
+