WORDHORDE

I have far too much incoming information, all the time. This torrent is only likely to increase. How best to organize it? The WORDHORDE, inspired by William S. Burroughs's Word Hoard, is my ongoing attempt. Requirements, or at least desired properties, of WORDHORDE include:

• spooling to a personal store, using NNTP or perhaps Maildir or even IMAP
  • whatever backend is used, it must support speedy ncurses-based clients (like slrn and mutt) and scripts
  • further UI must address Bayesian searching, clustering-based searching, etc...
  • an actual database might be desirable, as opposed to flat files ala an NNTP spool
• permanent personal archival of all incoming data (including spam, phishing, malware etc for later/long-term analysis)
• being a good netizen -- event-driven mechanisms when possible, optimal bulk transfers, etc
• sources/dests must include RSS feeds, NNTP, mailing lists, personal mail, IM history
• full bidirectional gateway capabilites -- post NNTP and have it turned into an IM, a mail, a phone call voiced by festival(1), a blog comment, etc...
  • needs address protected poll-based feeds: LJ flist, LWN subscription, etc...

Contents 1 HTTP 1.1 RSS 1.1.1 Mail Gateways 1.1.2 Console Readers 2 NNTP 2.1 Standards 2.2 Servers 2.3 Utilities 2.4 Web Gateways

HTTP

RSS

Mail Gateways

• rss2email: r2e binary seems pretty simple for turning RSS into SMTP.
  • r2e new defaultmail: create a new r2e database, with default email target defaultmail
  • r2e add url optmail: add a feed to the database by URL, with an optional email target optmail
  • r2e run: scan for new articles; mail them. designed for cronjob use, it seems.

Console Readers

• olive (Olive r1 01b7): attractive reader, imports OPML directly
  • Add a new feed with ^A, edit with ^E, options with ^O
  • Not very configurable, kind of slow
• nrss
• snownews
• rsstail
• newsbeuter

NNTP

Standards

• RFC 977 (1986), "Network News Transfer Protocol: A Proposed Standard for the Stream-Based Transmission of News"
• RFC 1036 (1987), "Standard for Interchange of USENET Messages"
• RFC 2980 (2000), "Common NNTP Extensions"
• RFC 3977 (2006), "Network News Transfer Protocol (NNTP)"
• This tin page seems to be kept up-to-date

Servers

• leadnode - http://leafnode.sourceforge.net/
  • Perhaps the only actively-developed open, sane NNTP server?
  • Last stable release 2007-03-17 (1.11.6), release candidate 2008-01-25 (1.11.7.rc1)
• sn - http://patrik.infa.fi/sn/
  • Last release 2004-04-10
• papercut - http://pessoal.org/papercut/
  • Last release 2005-04-21, primarily an NNTP wrapper around web forums

Utilities

• nglister - list newsgroups / stats from an NNTP server
• postnews - post an article from stdin to an NNTP server

Web Gateways

• W3news (NNTP/HTTP) (no debian package)
• WING (NNTP/IMAP) (no debian package)
• NewsPortal (NNTP/HTTP) (no debian package)