[email protected]
[Top] [All Lists]

Bug#303154: marked as forwarded (leafnode: Should checkpoint progress ev

Subject: Bug#303154: marked as forwarded leafnode: Should checkpoint progress every 1000 articles
From: Debian Bug Tracking System
Date: Tue, 05 Apr 2005 01:48:15 -0700
Your message dated Tue, 5 Apr 2005 09:31:41 +0100
with message-id <[email protected]>
has caused the Debian Bug report #303154,
regarding leafnode: Should checkpoint progress every 1000 articles
to be marked as having been forwarded to the upstream software
author(s) [email protected]

(NB: If you are a system administrator and have no idea what I am
talking about this indicates a serious mail system misconfiguration
somewhere.  Please contact me immediately.)

Debian bug tracking system administrator
(administrator, Debian Bugs database)

---------------------------------------
Received: (at 303154-forwarded) by bugs.debian.org; 5 Apr 2005 08:31:43 +0000
>From [email protected] Tue Apr 05 01:31:43 2005
Return-path: <[email protected]>
Received: from kerouac.projectcolo.org.uk [80.71.3.114] 
        by spohr.debian.org with esmtp (Exim 3.35 1 (Debian))
        id 1DIjTK-00088Z-00; Tue, 05 Apr 2005 01:31:43 -0700
Received: from localhost (localhost [127.0.0.1])
        by kerouac.projectcolo.org.uk (Postfix) with ESMTP id CFE0B79D81;
        Tue,  5 Apr 2005 09:31:41 +0100 (BST)
Received: from kerouac.projectcolo.org.uk ([127.0.0.1])
        by localhost (kerouac [127.0.0.1]) (amavisd-new, port 10024)
        with ESMTP id 10187-06; Tue, 5 Apr 2005 09:31:41 +0100 (BST)
Received: by kerouac.projectcolo.org.uk (Postfix, from userid 10003)
        id 7DABF79D66; Tue,  5 Apr 2005 09:31:41 +0100 (BST)
Date: Tue, 5 Apr 2005 09:31:41 +0100
From: Mark Brown <[email protected]>
To: [email protected]
Cc: [email protected]
Subject: [[email protected]: Bug#303154: leafnode: Should checkpoint progress 
every 1000 articles]
Message-ID: <[email protected]>
Mime-Version: 1.0
Content-Type: multipart/mixed; boundary="98e8jtXdkpgskNou"
Content-Disposition: inline
User-Agent: Mutt/1.3.28i
X-Cookie: Programming is an unnatural act.
X-Virus-Scanned: by amavisd-new-20030616-p10 (Debian) at projectcolo.org.uk
Delivered-To: [email protected]
X-Spam-Checker-Version: SpamAssassin 2.60-bugs.debian.org_2005_01_02 
        (1.212-2003-09-23-exp) on spohr.debian.org
X-Spam-Status: No, hits=-11.0 required=4.0 tests=BAYES_00,HAS_BUG_NUMBER,
        HAS_PACKAGE autolearn=ham version=2.60-bugs.debian.org_2005_01_02
X-Spam-Level: 


--98e8jtXdkpgskNou
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

The enclosed bug was submitted against the Debian Leafnode package.  

-- 
"You grabbed my hand and we fell into it, like a daydream - or a fever."

--98e8jtXdkpgskNou
Content-Type: message/rfc822
Content-Disposition: inline

Return-Path: <[email protected]>
X-Original-To: [email protected]
Delivered-To: [email protected]
Received: from localhost (localhost [127.0.0.1])
        by kerouac.projectcolo.org.uk (Postfix) with ESMTP id 93F6C79D81
        for <[email protected]>; Tue,  5 Apr 2005 07:25:06 +0100 (BST)
Received: from kerouac.projectcolo.org.uk ([127.0.0.1])
        by localhost (kerouac [127.0.0.1]) (amavisd-new, port 10024)
        with ESMTP id 03121-07 for <[email protected]>;
        Tue, 5 Apr 2005 07:25:06 +0100 (BST)
Received: from spohr.debian.org (spohr.debian.org [140.211.166.43])
        by kerouac.projectcolo.org.uk (Postfix) with ESMTP id 0F3CC79D66
        for <[email protected]>; Tue,  5 Apr 2005 07:25:06 +0100 (BST)
Received: from debbugs by spohr.debian.org with local (Exim 3.35 1 (Debian))
        id 1DIhO4-0007KX-00; Mon, 04 Apr 2005 23:18:08 -0700
X-Loop: [email protected]
Subject: Bug#303154: leafnode: Should checkpoint progress every 1000 articles
Reply-To: Don Geddis <[email protected]>, [email protected]
Resent-From: Don Geddis <[email protected]>
Resent-To: [email protected]
Resent-Cc: Mark Brown <[email protected]>
Resent-Date: Tue, 05 Apr 2005 06:18:07 UTC
Resent-Message-ID: <[email protected]>
X-Debian-PR-Message: report 303154
X-Debian-PR-Package: leafnode
X-Debian-PR-Keywords: 
Received: via spool by [email protected] id=B.111268122925438
          (code B ref -1); Tue, 05 Apr 2005 06:18:07 UTC
Received: (at submit) by bugs.debian.org; 5 Apr 2005 06:07:09 +0000
Received: from c-24-5-179-151.hsd1.ca.comcast.net (yoda) [24.5.179.151] 
        by spohr.debian.org with esmtp (Exim 3.35 1 (Debian))
        id 1DIhDR-0006bS-00; Mon, 04 Apr 2005 23:07:09 -0700
Received: from geddis by yoda with local (Exim 4.50)
        id 1DIc7I-0008U0-T3
        for [email protected]; Mon, 04 Apr 2005 17:40:28 -0700
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
From: Don Geddis <[email protected]>
To: Debian Bug Tracking System <[email protected]>
X-Mailer: reportbug 3.8
Date: Mon, 04 Apr 2005 17:40:28 -0700
Message-Id: <[email protected]>
Delivered-To: [email protected]
Resent-Sender: Debian BTS <[email protected]>
X-Virus-Scanned: by amavisd-new-20030616-p10 (Debian) at projectcolo.org.uk
X-Spam-Checker-Version: SpamAssassin 3.0.2 (2004-11-16) on 
        kerouac.projectcolo.org.uk
X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00 autolearn=ham 
        version=3.0.2
X-Spam-Level: 

Package: leafnode
Version: 1.10.8.rel-2
Severity: wishlist

Leafnode works poorly when given a mix of small text groups and huge
binary groups.  Imagine a group list like this:
        comp.latest-language
        alt.binaries.huge
where the first gets 10 postings/day, but the second gets 10,000 postings/day.

The problem is that leafnode runs in an all-or-nothing style.  Until it has
completed an entire run, everything in the sequence has halted.  This means
that if "alt.binaries.huge" requires 48 hours (or a week) to download
completely, then the entire local news server is wedged during that time.
Thus local users are unable to post, no new articles in the small
"comp.latest-language" ever appear, etc.

A much better architecture would allow leafnode to checkpoint itself and
re-enter its whole loop after a sufficient amount of effort, say 1000
articles (per group? per server?) or something.  After doing that much work,
it ought to pause and cleanup.  Generate the XOVER databases.  See if any
new articles have been posted (and if so send them to the upstream servers).
Download a few new articles from the small groups.  Etc.

Right now there is no good way to handle such a mix of large and small groups.
If a user requests one of the large groups, that immediately becomes an
effective denial-of-service for days, as fetchnews will cease performing any
user-visible work until an entire run has been completed.

-- System Information:
Debian Release: 3.1
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'unstable')
Architecture: i386 (i686)
Kernel: Linux 2.6.8-2-686-smp
Locale: LANG=en_US, LC_CTYPE=en_US (charmap=ISO-8859-1)

Versions of packages leafnode depends on:
ii  debconf                     1.4.30.11    Debian configuration management sy
ii  libc6                       2.3.2.ds1-20 GNU C Library: Shared libraries an
ii  libpcre3                    4.5-1.1      Perl 5 Compatible Regular Expressi
ii  logrotate                   3.7-2        Log rotation utility
ii  netbase                     4.21         Basic TCP/IP networking system
ii  tcpd                        7.6.dbs-8    Wietse Venema's TCP wrapper utilit

-- debconf information excluded


--98e8jtXdkpgskNou--


-- 
To UNSUBSCRIBE, email to [email protected]
with a subject of "unsubscribe". Trouble? Contact [email protected]

<Prev in Thread] Current Thread [Next in Thread>
  • Bug#303154: marked as forwarded (leafnode: Should checkpoint progress every 1000 articles), Debian Bug Tracking System <=