Status of this memo:
This document is a pre-internet-draft of an
experimental protocol on a possible internet standards track.
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
RFCs can be obtained at
http://www.rfc-editor.org/
The World Wide Web Consortium (W3C) is at
http://www.w3.org/
Copyright notice:
Copyright © 2000 by Mike Inman, All Rights Reserved.
Current intent is to take this specification GPL, if there's any reason for it.
Abstract
This document captures current thinking on the Anne's Ark distributed real-time
information sharing protocol from a technical and implementation perspective. It
describes possible features and implementations while discussing and analyzing the
technical and end-user ramifications of deploying such a system on the Internet today.
The protocol is currently (5-23-00 thru 7-16-00) a work in progress, and can be expected
to change before actual implementation or deployment.
Direct user to user file sharing has experienced a great deal of attention lately due to the "Napster music sharing community", its lawsuits, and its clones. User to user file sharing is nothing new to the Internet, HTTP, FTP, Kermit, et. al. are long standing file transfer mechanisims, while Archie through Yahoo have served to index these files.
If the Napster generation has done anything new and unique, it has provided a real-time database of files that are online and available right now. Previous search engines slowly slogged through the millions of available files and compiled massive database tables that are days or weeks out of date before they are available. In 1993, having any kind of accessible database was useful and impressive. Today, the out-of-date databases are still useful, but we can do better.
The other Napster contribution is a relatively easy to use package that integrates the file search and transfer mechanisms, including automatic sharing of files on the user's PC.
This document describes the Anne's Ark protocol suite, which acceses the transient "home server" phenomenon, while using the widely available capabilities of more reliably available servers to augment the system. Much like the Internet itself, Anne's Ark protocol does not rely on any strong central servers to function, instead keeping the information distributed widely, but with a cohesive interconnection that allows access to all currently available data.
The compelling feature of an Anne's Ark client is the ease with which information is shared - both in realtime as with Napster and Gnutella, and after the user signs off with automatic NETPOSTing of selected files - as with traditional websites. The protocol makes sharing, searching and retrieving a seamless process.
Real-time information sharing has applications beyond file transfer. Chat and newsgroup type of information may also be shared through this protocol. More advanced applications, such as garage sales, auctions, video conferencing, musician's jam sessions on sufficiently fast networks, multi-player video game pick-up rooms, or any other real-time information sharing situation may be developed using Anne's Ark protocol.
The HTML based World Wide Web is entering its tenth year, reliable servers are abundant and cheap - often free, but usually limited in storage space and / or bandwidth availability - especially when they are free or cheap.
Home PCs are largely multi-media capable, MP3 audio, MPEG video, etc. are all readily accessible technologies. The files are large, but hard drives are larger, and cheap. PC users have these large files, sometimes large collections of these large files, which they desire to share. Placing a home PC "on the web" as a general purpose server is a somewhat scary proposition, opening the possibility for attack from any corner of the globe.
High speed access lines (DSL, Cable, etc.) are the double edged sword that makes a "trustable" home server the killer app of today - they allow rapid transfer of these multi-media files, while simultaneously opening the door for rapid hacker-attacks. Just opening up the native operating system file sharing services is perceived (rightly) as a dangerous act.
The ideal home server sofware would provide functionality even after the home PC is safely disconnected from the net, and even shut down, since many (most?) home PCs cannot be dedicated to 24 hour service. This returns to the "upload to a dedicated server" scenario, though integrating this function seamlessly with the realtime file sharing will go a long way toward increasing acceptance and usage.
Gnutella is architecturally similar to Anne's Ark (in a simple way, and only the real-time net), though there are other parallels including net-news (NNTP), and its non-internet precursors including FidoNet and other BBS networks. While Anne's Ark does not reach as radically far into the future "lifestream" of data concept as The Second Coming - A Manifesto by David Gelernter.html, it is headding in that direction. Robin Raskin offers up some more down to earth advice in her article Being Smart - Internet Style, which can serve as general guidance as to what Anne's Ark is really all about - finding stuff, stuff you want, easily.
At its most basic level, the level 1 real-time Anne's Ark protocol might be considered "binary XML." It allows infinite extensibility with easy bypassing of unknown sub-elements. This begs the question: Why not just use XML? While XML is certainly in fashion these days, I have written XML parsers, and I have written binary parsers. Take a guess which one is easier to program and debug? In my experience, ASCII based XML is roughly twice as difficult to deal with (turn 5,000 lines of code into 10,000 lines of code) when starting from scratch with a standard ANSI C/C++ function library.
I just recently became aware of WBXML - which is a rather fancy way of encoding XML into binary format for use over WAP (see: http://www.w3.org/Submission/1999/07/). WBXML is quite elegant, and accomplishes the purpose of nicely compressing XML for use over WAP, but I don't believe it applies as well in Anne'sArk, where a brutally simple protocol can get the job done.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119] .
NETPOST - a webpage on a permanently available server containing information that might otherwise be retrieved directly from the AANET. A collection of NETPOSTs is an implementation of Anne's Ark level 2 protocol.
AANET - the collection of computers actively participating in Anne's Ark level 1 protocol.
NODE - a single computer at a particular IP address/port
VINE - a node which MAY route network messages
LEAF - a node which SHALL NOT route network messages
AACIM - AANET Connection Initiation Messages
INITIATOR - NODE which originates an AACIM
TRUSTED - NODE which has somehow proven itself to be a "good citizen" in the AANET, note that TRUSTED status may be directly earned by 1 to 1 experience, or personal contact, or it may be inferred by trusting a node who is trusted by a node whom you trust. Note that inferred trust should degrade very quickly, much beyond 2 contacts is effectively useless.
HOP - traversal between connected NODEs, usually used in terms of "number of HOPs" to get from NODE A to NODE B
SEARCH REQUEST - a request initiated by a NODE, intended to flood out to an entire AANET, or just specific SIG portions. Includes searches for network topology information, searches for files, and broadcast announcements (searches for someone who gives a ....)
BAN - to forbid a specific IP address from acting as a NODE
AALEVEL2 - a collection of NETPOSTs which mirror information from an active AANET, as well as (old) information from participants in the AANET who are not currently active (online).
SIG - Special Interest Group
FSP - File Sharing Protocol, encompassing both search and transfer functions
DRT - Distributed Real-Time, a group of computers with no central kingpin or cornerstone which exchange information that has validity right now. Distributed really needs another word, since it so often connotates systems with a strong central server (Decentralized?).
AADRTFSP - Anne's Ark Distributed Real-Time File Sharing Protocol, just "Anne's Ark" for English speakers.
SERVENT - a client/server in a peer to peer network relationship (gnutella-speak).
AANET - the collection of computers actively participating in Anne's Ark level 1 protocol. The level 1 protocol is the real-time network, providing up to the minute information about:
NODEs may serve as VINE or LEAF. VINE - a node which MAY route network messages. LEAF - a node which SHALL NOT route network messages. In practical terms, computers on a slow (56Kbps and lower) modem connection should serve as LEAF, while computers on a faster connection should serve as VINE. VINE nodes MUST pass on all SEARCH REQUEST messages, unless they are specifically performing net management on the messages.
The future may bring other node types (maybe ROOT nodes which serve as "always on" anchor points....), but for now the network is assumed to work with only VINE and LEAF. A minimal network implementation would use only VINE nodes, though the most lightweight servant sofware would be a LEAF only implementation.
Automatic net management techniques SHALL be used to cause the fastest VINEs to connect to one another forming a "backbone," with progressively slower connections being made on the periphery, leading to minimized lag for all NODEs. Direct observed lag and throughput measurements SHALL be used to determine speed for backbone formation purposes. NODEs MUST determine for themselves what their connection speed is with other NODEs and use the relative speed of all connections to determine which connections to sever for net topology optimization. Further consideration for backbone formation comes from minimum spanning tree requirements - causing NODEs to place a higher "value" on connections that are distant in terms of HOPs on the net.
Further automatic net management techniques SHALL throttle excessive requests, and BAN NODEs performing excessively badly (as in, malicious intent) from the AANET. Each NODE SHALL make its own determination whether or not to BAN a specific IP address, although a NODE MAY consider information from a TRUSTED NODE regarding BANning, it is more important to be skeptical when establishing TRUSTED status than to act on BAN requests from other nodes.
SEARCH REQUESTs are propogated throughout the entire net as a "network flood", except when limited to the scope of a specific SIG. LEAFs MAY request their VINE connection that they not receive certain types of SEARCH REQUESTs. VINEs are MUST pass on all SEARCH REQUESTs that are not being specifically filtered for net management.
NETPOSTing is OPTIONAL, though NODEs which perform this valuable service will make the AANET more easily accessible by providing connection information in a known, fixed place on the world wide web. In theory, a NODE could endeavor to mirror the entire contents of the AANET to a NETPOST. In practice, only the most important information should be placed in a NETPOST due to the typically limited storage capacity of WWW servers.
LEAF and VINE NODEs MAY establish NETPOST mirrors of their entire current AANET information, or any subset thereof. This information MAY include:
NETPOST requirements take into consideration the W3C XML standard reccomendations, but above all, serve up as HTML pages with human readable and browsable content - thus, they SHALL be implemented in strict XHTML.
The packet format is intended to be as "lightweight" as possible, while
providing as much flexibility and extensibility as possible. Current standards
such as MIB were examined, but considered to be too complex for the task at hand.
XML is structurally similar to Anne'sArk level 1, but it is implemented in rather
An upper limit on packet sizes of 4GB is arbitrarily placed - reflecting practical
limits of current transmission technology, though this does not mean that files of
longer lengths cannot be transmitted in multiple packets. Upon connection, NODEs
SHALL exchange maximum accepted packet size information. Too large packets MAY be
dealt with in two ways, 1) drop them, or 2) break them up. Dropping is probably
preferred, due to the complexity involved in breaking up complex packets.
Packet and block formats
SHOULD keep to 4 byte word alignment, and multiples of 12 bytes for each block
size*,
though this is not REQUIRED. Also, numeric values SHOULD be stored in network
byte order, MSB first. When an order is not specified, it MUST be MSB first.
To keep the protocol extensible, node software MUST ignore unrecognized block
types within a packet, but pass these unrecognized blocks along with the rest of
the packet, when it is appropriate to pass the packet along. For the most part,
block order within a packet SHOULD not matter. Further on this point, if a
node is considering stripping an unrecognized (presumably large) block from
a packet as a matter of traffic control, it would usually be preferred to drop
the packet entirely (unless the NODE really knows what it is doing,
which is unlikely if the block type is unrecognized.).
* This uniform block size recomendation
comes from 10 years of experience "living and dying" with a similar block based
extensible binary data encoding scheme. Ultimately, when something goes wrong,
you will end up looking at the binary data stream with a binary editor,
and if it has universal uniformity to the packet size, you will be much
happier. The program shouldn't care if the uniformity is there or not.
** For all the same reasons that uniform
block sizes are a good idea, nice, readable, mnemonic packet headers makes life
in the trenches much easier. It doesn't matter to the computer, but ultimately,
it will make development of new client / server software go more quickly and
easily.
It is RECOMMENDED that all packets include an epilogue block:
The purpose of the epilogue block is just a redundant check to ensure that
the whole data stream arrived in tact. An over-ambitious spec writer might
define a "CHK1" type of epilogue block with a 32 bit checksum of the whole
packet in it. And, in fact, this MAY be added in the future. However,
I believe that would be overkill within the anticipated TCP/IP communication
medium. On the other hand, in a scenario where a stream gets corrupted near
the end and filled with garbage - simply looking for this end marker will
provide a check for that.
Note that the epilogue block is counted in the header's count of blocks,
so for systems which follow the recomendation to include an epilogue block,
a minimal packet would look like:
* Reading this spec in its entirety, you
may notice a constant harping on this point of keeping the block sizes in uniform
12 byte multiples. This is also from 10 years of experience dealing with project
after project where the alignment was not included in the beginning, and later on
wishing dearly that it was.
Specific Design
AnnesArk packet format
Philosophy:
All nodes in the Anne's Ark level 1 realtime network communicate directly to
each other using packetized messages. It is assumed that these packets will
be transmitted via TCP/IP, though any transmission media which can carry
binary data reliably back and forth between "nodes" may be used. For example,
people might send paper punched tape via the postal service - it wouldn't be
particularly efficient, but it would still implement the network. In any
event, all nodes within a particular Anne's Ark net will use the same
initial underlying communication layer, usually TCP/IP.
All values are binary, but ASCII encoding SHOULD be used where possible to make
the stream easier to debug "by eye"**.Packet Header:
An Anne's Ark packet always starts with the header:
+----+----+----+
|Anne|size|blks|
+----+----+----+
Anne is literally the ASCII characters 'A','n','n','e'.
size describes the total size of this packet in bytes, including this 12 byte header.
blks describes the number of blocks in this packet.
Nodes may send minimal packets, containing only this 12 byte header (blks=0, size=12),
presumably as a "heartbeat" indication that the connection is still alive, though
most "heartbeat" packets would likely contain additional status information.
+----+----+----+
|sArk|type|data|
+----+----+----+
sArk is literally the ASCII characters 's','A','r','k'.
type defines the type of epilogue.
data contains a single data word, or just 4 bytes of padding.
The only type of epilogue defined so far is " END", with a traditional data
word of "pack", although the data word for " END" is ignored by definition,
so programmers with big egos may choose to replace the data word with four
bytes of their own choosing.
+----+----+----+----+----+----+
|Anne| 24 | 1 |sArk| END|pack|
+----+----+----+----+----+----+
Data Blocks:
When present, data blocks start immediately following the 12 byte header, each data
block has its own 8 byte header:
+----+----+
|Type|size|
+----+----+
Where Type is a unique 4 byte identifier of this block type. Note that multiple blocks
of the same Type MAY appear in a single packet.
size is the size of this block, in bytes, including this 8 byte header.
A minimal block would have a size of 8, though such a block SHOULD pad itself with
4 bytes of dummy data to achieve 12 byte alignment*. Note that
these dummy bytes are still counted in the size. This means that a block which
wishes to keep track of the size of an internal item down to the byte, yet still wishes
to acheive 12 byte alignment using padding MUST have another internal data element to
specify the size of the internal item.
+----+----+----+
|Type|size|blks|
+----+----+----+
Where a block will contain sub-blocks within its size, the number of sub-blocks,
not counting sub-sub blocks is indicated in blks. Note how this matches with
the structure of the initial 12 byte packet header.Vstring format
+----+--------
|size|data....
+----+--------
I was going to reference Net Strings, a passing fancy in RFQ circles,
and a concept quite similar to this one, but Net Strings are out of date, and a
bit overkill in this situation, anyway.
+----+----+----+ |CNCT| 12 |RQST| +----+----+----+ +----+----+----+ |CNCT| 12 |ACK | +----+----+----+ +----+----+----+ |CNCT| 12 |CHAL| +----+----+----+ +----+----+----+ |CNCT| 12 |RESP| +----+----+----+ +----+----+----+ |CNCT| 12 |DENY| +----+----+----+RQST is a request for connection.
Nodes serving a simple, open network will use connection of the form RQST ACK , where the node seeking connection sends a packet containing a CNCT..RQST block, and the receiving node responds with a packet containing a CNCT..ACK block. If the receiving node does not wish to connect to the sending node, it MAY respond with a CNCT..DENY block, or it MAY choose not to respond at all.
A more advanced node would also respond with a SVCS block in the packet, announcing the services it offers, and possibly a MSG block for extra flavor.
Nodes in a more secure network will answer a CNCT..RQST with a CNCT..CHAL, asking the sending node to do something that identifies itself. This will be responded to with the CNCT..RESP, which will then be answered with a CNCT..ACK , or a CNCT..DENY, or silence.
+----+----+----+ |SVCS| 12 |RQST| +----+----+----+ +----+----+----+----+----+----+----+-- |SVCS|size|SVCS|CNCT|TOPO|OPER|PROX|etc. +----+----+----+----+----+----+----+-- +----+----+----+ |SVCS| 12 |TYPE| +----+----+----+ +----+----+----+----+----+-- |SVCS|size|TYPE|SUB1|SUB2|etc. +----+----+----+----+----+--A SVCS..RQST block may be targeted for delivery to a specific node, or flooded out as a general request that all nodes respond with their services description. Note that it is RECOMMENDED that nodes respond to a specific SVCS..RQST, while general (flooded) requests SHOULD be throttled for traffic control. Also, nodes may choose to passively monitor SVCS..SVCS blocks and build their network information without specifically requesting it.
The SVCS..SVCS block lists all top level services offered by the responding node. Some top level services may have multiple implementations, in which case, a SVCS..TYPE request (where TYPE is a service with multiple implementations) may be issued to obtain further detail on that specific services are offered.
As always, respondants may add MSG for flavor.
Note that to achieve 12 byte alignment, a SVCS block MAY be padded with NULL services, where NULL is 'N','U','L','L'. The size will include the zero or more NULLs, but then size isn't a count of services provided, so noone should be confused - whether NULLs are present or not.
Also note that it is anticipated that Anne'sArk will follow a non-linear evolution, that is, some clients will support some services, while others will support others, and as such, declaring a "rev level" compliance doesn't really do much. The SVCS block gets down and dirty, telling what specific services are supported, and by exclusion, those which are not, either because the software is not capable, or the operator has chosen to turn them off: permanently or temporarily, for all or part of the net.
+----+----+----+----+----+----+ |THRO| 24 |incs|inct|oucs|ouct| +----+----+----+----+----+----+Where incs is the incomming chunk size inct is the incoming chunk time (in milliseconds), oucs is the outgoing chunk size, and ouct is the outgoing chunk time (in milliseconds), all from the perspective of the sending node. Zero in either of the time fields indicates unlimited throughput for that direction.
A THRO block is typically sent as part of a connection negotiation sequence, or by itself in a packet. If a THRO block is sent as part of a routed packet, it MUST be removed (or turned into a NULL block) before passing the packet on, since the THRO information only applies to the channel the block was sent through.
What the THRO block means is that the sender of the block wishes to throttle traffic on that connection to the level specified. The outgoing throttle rate will be implemented by the sending node - but it is the responsibility of the receiving node to implement the incoming data throttling. This may be accomplished simply by creating a queue for the data to be sent, then sending incs bytes, and waiting inct milliseconds before sending the next incs bytes obviously, the inct millisecond timer may start counting when the first byte is sent. Also, incs indicates a maximum chunk size, there is no reason to wait until incs bytes are available to start transmitting. In the absence of any THRO blocks, the connection may be assumed to be unthrottled (i.e. set your inct timer to 0, and incs to 1.) The most recently received THRO block information will be used until the connection is disconnected.
When a node receives a THRO block limiting inbound traffic to the connection, it MUST limit its outbound traffic to that connection to a value with the same or lower product of incs*inct. (e.g. incs=1024 bytes, inct=100ms, for a product of 10.24 Kbytes per second. The sending node must choose a oucs and ouct that transmits 10.24 Kbytes per second or less.) If the recipient of a THRO block does not properly throttle their transmissions, the sender of the THRO block may choose to terminate the connection, but should notify with a MSG block stating why the connection is being terminated, possibly identifying the version of the software which is upset with the data rate being transmitted. Also, a reasonable leeway should be given by both sides on the data rate - in other words, the sender should probably throttle to 98% or less of the stated desired data rate, and the recipient should probably accept sustained rates up to 110% of their requested rate before getting bent out of shape. Obviously, the recipient should average total bytes recieved over an interval of 10 seconds or more when calculating actual throughput observed, since data may be "backing up" somewhere on the net before being received.
+----+----+----+----+----+----+ |TOPO| 24 | 1 |RQST| 12 |ALL | +----+----+----+----+----+----+ +----+----+----+----+----+----+----+----+-------+----+----+-------+-- |TOPO|size|blks|RESP| 12 |ALL |SID |size|data...| ID |size|data...|etc. +----+----+----+----+----+----+----+----+-------+----+----+-------+-- +----+----+----+----+----+----+ |TOPO| 24 | 1 |RQST| 12 |VINE| +----+----+----+----+----+----+ +----+----+----+----+----+----+----+----+-------+-- |TOPO|size|blks|RESP| 12 |VINE|SID |size|data...|etc. +----+----+----+----+----+----+----+----+-------+--The first is a request for topology information, like SVCS, it may be target or flooded, with all the usual concerns and opportunities for eavesdropping.
The response contains embedded ID blocks which list all the nodes that this node has connection to, and one and only one SID block, usually at the front but not required, that identifies the server which is returning this information.
The VINE request requests that the returned TOPO block contain only VINE nodes. The response shown contains a RESP..VINE block to indicate that this TOPO information ignores LEAF and other nodes which are not serving VINE functions.
I don't think that MSG would enhance a TOPO block.
+----+----+----+-- |xID |size|blks|sub blocks.... +----+----+----+--CID is a "client ID" used to identify a service requestor.
The format is pretty loose, it contains sub blocks with, at a minimum, the node's address (e.g. an IP block.) Additionaly, it is RECOMMENDED that a HAND block be included for more unique identification, and nodes MAY choose to include one or more SVCS description blocks, an OPER block for further description of the system / operator, and MSG blocks for extra flavor, plus whatever other blocks are deemed appropriate as part of the unique identity of the node.
Nodes MAY choose to strip down the content in an xID block before passing them along, though it is RECOMMENDED to keep HAND blocks at all times, and REQUIRED to keep the address (e.g. an IP block.)
It is easy to see how system operators might get carried away overloading their ID block, and then how a TOPO block might become quite huge. As a matter of general practice, it is RECOMMENDED to keep the ID block limited to a size of 1000 bytes or less, and save the other information for when it is specifically requested. On the other hand, it is also beneficial to put information like SVCS into the ID block, so that these things do not need to be requested.
Since all sub-blocks SHOULD be held to 12 byte alignment, there shouldn't be a problem with keeping 12 byte alignment in the overall xID block.
+----+----+----+-- |xxxx|size|size|data... +----+----+----+-- +----+----+----+----+----+----+ |QERY|size|xxx1|www1|xxx2|www2| +----+----+----+----+----+----+These blocks contain "freetext", implemented as a Vstring. Note that freetext may be ASCII, double byte encoded text, kanjii, sanskrit, binary, etc.
The QERY block is sent to request a reply containing the specific freetext type - this is not intended for use with MSG but can be used with other types, example:
+----+----+----+----+----+----+ |QERY| 24 |ICON|MTYP|NULL|NULL| +----+----+----+----+----+----+might be used to request a MTYP (Music Type preference), icon for the node. Note that a QERY block might ask for one block, or many. NULL indicates that this is a space filler, to acheive 12 byte alignment.
+----+----+----+----+----+----+ |QERY| 24 |SIG |ALL |NULL|NULL| +----+----+----+----+----+----+is asking for all SIGs supported by this node. In the 4 byte description space, the word "ALL " indicates a wildcard match for everything. Nodes responding to this should exercise good judgement, for instance, if you have 1000 FACEs, you might choose to not send them all in response to an "ALL " query.
MSG - a general purpose message, for added flavor.
HAND - a unique identifier of the person operating the node, a handle, in practice a HAND block SHOULD be limited to 64 bytes, meaning that the handle itself is limited to 48 bytes, or 47 characters if it is a null terminated ASCII string. Also note that in an anarchy, uniqueness is not guaranteed, so combine the handle with the address to avoid accidental duplicates.
OPER - this is where the operator can tell you his life story, street address, telephone number, credit card number and expiration date, whatever he cares to share with the wide, faceless world of the Internet. In practice a OPER block should be limited to about 1000 bytes or less, but this still leaves room for a small .JPG image, if desired. While it would be possible to encode a .JPG image in the OPER block, it would be preferred to do that in a FACE block.
ICON - for systems which support them, ICON blocks define a small image (32x32 pixel .GIF RECOMMENDED) that somehow represents the node. Many types of ICONs may be defined, each with their own unique 4 byte type.
FACE - to put a face on the faceless Internet, now you can scan a picture out of a magazine to show the world what you really don't look like. RECOMMEND a portrait oriented .JPG file with an aspect ratio of 4 horizontal by 5 vertical, and minimum resolution of 100x125, with a maximum file size of 10Kbytes. Many types of FACEs may be defined, each with their own unique 4 byte type.
SIG - a fairly serious block, this one contains a description of a Special Interest Group, as ASCII text, lower case letters and numbers, separated by periods, and when terminated with a '*' representing a wildcard match to all strings that start that way.
SIGM is a type of query block looking for VINEs which support
matching SIGs.
+----+----+----+--
|SIGM|size|size|data...
+----+----+----+--
This may be flooded out to the general network to find gateways to
SIG sub-nets. Wildcards are allowed.
FILE is also rather serious, it contains the "meat" of an entire file. It would be used in conjunction with FNAM filename, and other similar blocks. Note that more sophisticated file transfer mechanisms may be developed, and that transfer of large files through the general network is a no-no, and banishment will soon follow....
FNAM as mentioned above, contains a filename, usually associated with a FILE block in the same packet.
FTIM for use with FILE and FNAM, to preserve the file creation time. Specific format of the time within this block is yet to be established.
Note: The above FILE, FNAM and FTIM blocks are intended as a "quick start" protocol, making something fast, useful and easy to implement. For "serious" servents, the LDAP / X.500 directory access protocols should be studied: RFC2251, RFC2589, RFC2307, RFC2649, RFC2829, RFC2830 and adapted to the Anne's Ark extensible block format. For interoperability with the more lightweight servents, heavyweights SHOULD also replicate the simple FNAM blocks already defined here.
Other friviolous freetext blocks may be invented at will.....
It is RECOMMENDED to send any unsolicited fretext blocks in the context of (inside) an ID block, identifying the sender.
As always, remember to 12 byte align the block by padding zero bytes after the Vstring, if required.
+----+----+----+ |VINE| 12 |RQST| +----+----+----+ +----+----+----+----+----+----+-- |SID |size|blks|VINE| 12 |INFO|etc... +----+----+----+----+----+----+-- +----+----+----+ |VINE| 12 |CHAL| +----+----+----+ +----+----+----+ |VINE| 12 |RESP| +----+----+----+ +----+----+----+ |VINE| 12 |DENY| +----+----+----+ +----+----+----+ |VINE| 12 |ONLY| +----+----+----+A VINE..RQST block requests the node to send its VINE information in the form of a SID block.
The response SHOULD be a SID block containing VINE..INFO, followed by an unabridged TOPO block, a top level SVCS block for the node, and all SIG blocks indicating which SIGs the VINE supports. If no SIG blocks are present, it is assumed that the node supports the open network. A SIG with a single * character also represents the open net. The usual IP address, HAND unique id and other information should also be present.
Note that order of blocks within a SID block is unrestricted, no information is stored in the ordering of the blocks. However, the above order is RECOMMENDED to keep human readers of the data stream from going too bonkers.
CHAL, RESP and DENY MAY be used by secure nodes in a similar fashion to the CNCT block.
The VINE..ONLY block is a special case block used in TOPO blocks to indicate that this topology tidbit only contains VINE nodes.
+----+----+----+----+----+----+ |GUID| 24 |THIS|TYPE|data|data| +----+----+----+----+----+----+ +----+----+----+----+----+----+ |GUID| 24 |WANT|TYPE|data|data| +----+----+----+----+----+----+ +----+----+----+----+----+----+ |GUID| 24 |RQST|TYPE|data|data| +----+----+----+----+----+----+ +----+----+----+----+----+----+ |GUID| 24 |DENY|TYPE|data|data| +----+----+----+----+----+----+THIS identifies the GUID of this packet.
Placed in a packet to uniquely identify this packet as different from any other, the 8 data bytes are hashed up from the node address, handle, time of creation of the packet, and whatever else is appropriate. The 4 bytes of (TYPE) identify the hash algorithm used, and are also compared for uniqueness. Note that all nodes using GUID MUST compare all 12 bytes for uniqueness. Uniqueness checkers will likely start looking at the end of the packet for uniqueness, and hopefully get an earlier hit than comparing a bunch of identical TYPEs.
If a packet arrives at a NODE which needs GUID for routing, and the packet does not yet have a GUID, that NODE will add a GUID block to the packet.
+----+----+----+ |TTL | 12 |cnt | +----+----+----+This is where the rubber hits the road for network flooded packets. A TTL block contains a count (usually less than 10), that is decremented every time the packet is passed from node to node. When the count hits 0, the packet is passed no more.
Gnutella weenies may take a potshot at TTL requiring 12 bytes where 1 would do, I ask them to look at the overhead of a TCP/IP connection before preaching any more about efficiency. Also, fitting TTL in as an optional component of the extensible block format (the Ark in Anne's Ark) just makes life that much easier when extending the protocol, as in a total no brainer for backward compatibility with old servants.
+----+----+----+----+-- |ROUT|size|blks| ID |etc. +----+----+----+----+--The ROUT block MAY be used to trace a packet through the net. It is filled with minimal (IP and HAND only) ID blocks, and in an exception to the rule that order does not matter, the newest ID block is added to the end of the list.
It is RECOMMENDED that a VINE node which handles ROUT blocks check and make sure that the node it recieved the packet from has already placed its ID as well as this node's ID in the ROUT block, if present. If this has not been done, or done incompletely, this node should complete that information. Then this node should add the ID of the node it is sending the packet to before passing the packet on. This way, a ROUT supporting VINE can make up for neighboring VINE nodes which do not support, or incompletely support ROUT information.
Obviously, ROUT information can be used to more efficiently flood the network without relying on GUID as heavily, as well as assisting in returning responses through the network without requiring a total flood.
Have I harped on the 12 byte alignment thing lately? Good.
+----+----+----+----+-- |RTS |size|blks|ROUT|etc. +----+----+----+----+--RTS can be used as a simple routing strategy for replying to messages. The recipient can copy the ROUT block out of a packet they just received, wrap it up in a RTS block and away it goes. Note that the GUID must change for the return message.
blks is anticipated to always be 1 in an RTS block, since the ROUT block contains IDs as sub-blocks within itself.
Assuming that the two firewalled nodes have found each other through an AnnesArk network, and they now wish to establish efficient communication with each other, not making multiple hops through the net, one may send the other (the long way) a packet that includes a list of known good proxy servers. The other may then compare that list to their list of known good proxy servers, find a match, and initiate direct connection (if they have not already for the general net) with the proxy, while also sending a return message (the long way) to the first so they may also initiate direct connection (if they have not already for the general net) with the proxy. Now we have a "two hop" link.
Proxy blocks are of the forms:
+----+----+----+ |PROX| 12 |RQST| +----+----+----+ +----+----+----+ |PROX| 12 |ACK | +----+----+----+ +----+----+----+ |PROX| 12 |CHAL| +----+----+----+ +----+----+----+ |PROX| 12 |RESP| +----+----+----+ +----+----+----+ |PROX| 12 |DENY| +----+----+----+ +----+----+----+----+-----+----+-----+ |PROX|size|FWD |SID |.. |CID |.. | +----+----+----+----+-----+----+-----+ +----+----+----+----+-----+ |PROX|size|USE | ID |.. | +----+----+----+----+-----+ +----+----+----+----+-----+----+-----+-- |PXLS|size|blks| ID |.. | ID |.. |etc. +----+----+----+----+-----+----+-----+--Where RQST is requesting establishment of proxy service, which may not always be granted, even if an ordinary connection has already been established.
The PROX..FWD block is the real meat of proxy work, where SID is the address of the server/sender, and CID is the address of the client/consumer. A proxy server node will simply forward this packet immediately, unmodified from SID to CID. A proxy server MAY choose to check if SID and the true sender match up properly, and act accordingly if they do not. Note that the PROX..FWD and PROX..USE blocks are somewhat unusual in that they contain a fixed number (2 and 1) of sub-blocks.
It is anticipated that a PROX block would commonly be used transfer a file using the FILE, FNAM and related blocks, or another hopefully more sophisticated scheme.
PXLS is a "list of good proxy servers" as mentioned above. The PROX..USE block is a response to such a list, telling which node was chosen - in the single ID block which immediately follows, and is in fact included in this PROX block and its size.
+----+----+----+----+----+----+----+-- |SRCH|size|blks|QERY| 12 |flag|FNAM|etc.. +----+----+----+----+----+----+----+-- +----+----+----+----+----+----+----+-- |SRCH|size|blks|RSLT| 12 |flag|FNAM|etc.. +----+----+----+----+----+----+----+--Where the presence of a QERY block is a query looking for matches, and the presence of a RSLT block indicates that these are search results, with a series of filenames, or other, better, file descriptions which match the QERY which was sent out.
Nodes which support searches will respond with SRCH in their top level SVCS..RQST response, and in response to a SVCS..SRCH request, the node will respond with what types of searches are supported. As a start, searches by filename are supported by including a FNAM block inside the SRCH block. Wildcard characters * and ? SHOULD be supported. Partial matches MAY be supported using additional notation which is not yet developed. Similarly, matches to MP3 artist and title fields, search for matches to MD5 hash codes, search on file content, etc. may be developed using sub-blocks within the SRCH block. It is anticipated that all of the above examples will be developed as a pattern for future expansion to follow before this spec is released, but it's far too late at night to do such an important thing now. We'll find a use for those flags later, too.
Responses to a search block are sent directly to the node who requested the search.
+----+----+----+----+-- |ANNC|size|blks|MSG |etc.. +----+----+----+----+-- +----+----+----+----+-- |CHAT|size|blks|MSG |etc.. +----+----+----+----+--ANNC and CHAT blocks are identically handled, but serve two different intended purposes, and making them different enables users to set filters to display or ignore them selectively. They may contain MSG blocks, other special blocks which modify the text color or font, other fancy stuff like ICONs and FACEs, whatever. It is not recomended that clients accept FILEs and store them on the local hard drive, as this would constitute a security hole allowing outsiders to plant unsolicited programs on the user's computer. On the other hand, in a low bandwidth chat environment, it may be desired to transfer "heavy" items like sound clips and photographs as files to your "friends," so a simple play or display by reference command can trigger the media without having to transfer it every time. Just be careful out there.
ANNC and CHAT blocks are propogated identically to SRCH searches.
+----+----+-- |SKIP|size|etc.. +----+----+--This is the poor man's way of deleting a block. Instead of removing the block from the chain, overwrite the block type with SKIP. At the earliest convenience, the block SHOULD be physically removed from the packet. Note that all sub-blocks are also removed from the packet by overwriting the "container" block's header type with SKIP.
+----+----+----+----+------+----+------+----+----+-- |PRIV|size|blks|CID |etc.. |KEY |etc.. |DATA|size|encrypted data.. +----+----+----+----+------+----+------+----+----+--Private blocks MAY be targeted to a specific node, in which case they SHOULD contain the CID of the consumer / client. Private blocks MAY also be targeted to a group, which MAY be identified by SIG, or a list of CIDs.
A KEY sub-block MAY be included as a sort of "password reminder" type of information, telling the client which key, or algorithm, should be used to decode the DATA. Decryption MAY also be pre-arranged so that sender and consumer do not need a KEY block to communicate privately. Specific encryption schemes will be outlined elsewhere.
The DATA sub-block contains the private information. It SHOULD contain another block or blocks that are processed by the consumer / client exactly as if they were not encrypted. This may be FILE, or CHAT, or any other block that the two nodes wish to communicate privately. Unless there is a specific reason TOPO and similar network information blocks SHOULD NOT be encrypted, to allow eavesdropping. Obviously, routing information cannot be encrypted, unless the packet is traveling through a secure SIG where all member nodes understand the encryption - in which case, the entire contents of a packet may be put in a PRIV wrapper block.
The size in the DATA sub-block may differ from the actual size of the decrypted blocks, it is an indication of the size of the encrypted data. Please remember to pad for 12 byte alignment, and include any specific size information the decryption algorithm may need in another field, not using the size sub-block field, unless you don't care about the 0-11 bytes that will be added on.
+----+----+----+----+----+----+--- |FSEG|size|from|from| to | to |data.. +----+----+----+----+----+----+---There are two purposes to the FSEG partial file block, as opposed to the FILE total file block. First, to enable transfer of reasonably sized chunks. Second, to enable transfer of files >4GB in total size. from and to are MSB first 64 byte numbers. Lightweight implementers MAY choose to ignore the upper 4 bytes, at the peril of being unable to handle files >4GB in size, but they MUST still recognize the placeholders.
Applications of FSEG are many and varied - from enabling partial downloads to be picked up where they were left off, to enabling multi-point downloads - getting the same file from several sources at once.
size should be padded to a 12 byte boundary, but from and to directly imply how many bytes of true file data reside in the block. Just to say it explicitly: from is the byte number (in the source file) of the first byte of data in this block, and to is the byte number of the last.
This is not to say that LEAF nodes may not connect to each other for direct communication (e.g. for file transfers). Furthermore, a node may be connected into the net as a VINE node, serving packet routing functions, and simultaneously open a LEAF service level connection to another node. This newly connected node must recognize the LEAF connection as a dead-end, and only route packets through it which are intended for that node.
TTL routing is based on the notion that a packet is created with a limited "Time To Live." This means that when a packet is routed into the network, it will make a limited number of "hops" from node to node before it is simply deleted. The TTL block contains a count of the number of "hops" remaining for this packet, and this count is decremented by 1 each time the packet is routed to another node. A TTL of 7, in a network where each node is connected to four others, would result in up to 4 x 3^6, or 2916 nodes receiving this packet. If each node is connected to 6 others, 6 x 5^6 = 93750. Of course, this assumes an ideal connection topology with no near neighbor redundancy.
GUID is a "Global Unique Identifier," which VINE nodes use to identify packets and keep them from needlessly looping around the net. Note that the TTL count keeps them from endlessly looping. Each VINE builds a "routing table" which consists of the most recently observed GUIDs, and which neighbors have seen each one already. Obviously, if a packet wanders back in to the VINE node via a local loop, it will be recognized and not sent out again. The GUID packet also provides provision for asking a node if they have "heard this one before," which will likely be a useful protocol for avoiding transfer of large packets needlessly.
Extremely lightweight LEAF nodes MAY be excused from creating GUID blocks for
their network packets. In fact, it may be preferable that the VINE nodes handle assignment
of the GUIDs, just to avoid flaky LEAF clients that might choose not so unique
GUIDs. In the event that a LEAF node sends a packet to a VINE with a TTL, but
no GUID, the VINE SHOULD assign an appropriate GUID before passing the packet
along, and obviously mark the sending LEAF as having sent (and therefore seen) that GUID.
SIG - Special Interest Group filtering
There is an obvious problem when considering the idea of every computer in the world searching
every other. That is, if everyone initiates just one search a day, the communication network
cannot handle all the traffic, and the CPUs cannot perform all the searches. In a rough sense,
the TTL limit can reduce one's "scope", but this is an extremely rough tool, as
demonstrated above, if each node makes 4 to 6 connections, a TTL of 7 can reach anywhere from
less than 3000 machines to nearly 100,000. While everyone might like to be able to search
millions of other machines, they certainly do not want their machine and communication line
serving search requests from a million others.
In order to reach "the right" group of machines through the network, SIG filtering is used. Although the network in general may reach a self-crushing number of nodes, a SIG contains only those nodes who are interested in the SIG's topic. Packets which contain a SIG block are only routed to VINE nodes which serve a matching SIG. The * character is a wildcard which means to match all strings which start with the characters preceeding the *. Characters following the * are ignored, and other "standard" wildcards (like ?) are not supported. Matches are non-case sensitive, it is RECOMMENDED that names be stored and displayed in all lower case. Only alpha, numeric, and ".'&+()-=/ " are permissible standard characters in a name. Commas and/or semicolons are used to separate multiple names in a list, and spaces are only permitted within the name, not as leading or trailing characters. An @ in the first character of the name indicates a non-wildcard scope. Examples of SIG matches and mis-matches follow:
It is anticipated that hierarchies like the alt. newsgroups will more or less naturally form themselves. Groups which become too popular will fragment, and those which are no longer served by VINEs will simply cease to function. Groups may be searched for through the general net, or through specific sub-trees.
It is also anticipated that the "general net" will become somewhat limited in its usefulness, perpetually bogged down with traffic, or choked down to a limited scope where the average user might to reach a couple of thousand of the nearest (unpredictable) nodes at the most. It might be possible to keep the "general net" a useful entity if undirected flood search requests were somehow foucsed to their targets using something like (CIP) Common Indexing Protocol, while VINEs serving the general net refused to serve network flood requests. On the other hand, as long as the "general net" does not grow too large, the great grinding halt will not come.
VINEs may choose to respond to a SVCS..SIG request by listing the SIGs they support, making these "public SIGs"; or not, making those "private SIGs." Note that private SIGs may increase their privacy through the use of PRIV wrappers which all members of the SIG know how to decode. Implementors wishing to create a kind of VPN using Anne's Ark protocol should study RFC2764 - A Framework for IP Based Virtual Private Networks, for a background on what a VPN can be. It is doubtful that Anne's Ark would efficiently serve as a full network transport protocol, but many useful services may be implemented, especially when the members of the private net may be implicitly trusted.
One wild thought is the idea of a "floating SIG" where all members support a group of SIGs like: yesterday.something, today.somethingelse, tomorrow.yetanother. Then, at each new day the group somehow decides on what tomorrow's new group name will be, distributes it to the group, and stops supporting the oldest SIG in the chain. This would have the effect of making the group hard to find, unless you keep up with it on a daily basis. Interesting? Perhaps. Useful? Perhaps not.
Regarding anarchy and name administration, well, in anarchy, there is no name administration. This won't necessarily guarantee chaos, however. Groups of people will choose names to identify their SIG. If two SIGs form with the same name, perhaps by happy coincidence they will share similar interests and be thrilled to find each other. In the more likely event that one group is "South Park" the gardening club from Manchester - founded in 1683, and the other group is "South Park" the cartoon enthusiasts, it is likely that "South Park" of Manchester will choose to change their name to "South Park of Manchester", eliminating the conflict. While this would seem to be the preferable solution, some other options exist:
When a packet is hopping back along the RTS route, it may encounter a network outage. The node holding the packet at that point may then choose to add a TTL and GUID block to the packet and broadcast it a short distance, hoping that it will reach its destination somehow. Or, a more sophisticated VINE may have been eavesdropping on network routing information and may know another route by which the packet may be sent, replacing the RTS with the new route. Any VINE which is directly connected to the intended recipient SHOULD be sophisticated enough to direct hop the packet to its destination, rather than following any intermediate hops thay might be called for in the RTS route.
All in all, reducing search result traffic to a single path through the network, instead of letting it flood out, is a very good thing.
If a proper ROUT block is included, then the SID block is redundant, and may be removed.
Nodes implementing CHAT networks SHOULD consider using SIG sub-nets to implement private rooms, instead of transmitting multiple CIDs on flooded packets.
Note the naming convention, extending the parent class' name to form the child class, and starting all Anne'sArk protocol related classes with CAnne, to avoid the inevitable collisions with common names. CAnneBlock - the most basic object is a "block." This includes the type, size, pointer to the remaining data and a notation whether or not this object "owns" the remaining data, or if the data even exists as a contiguous chunk.
CAnneBlockMulti - complex blocks are constructed as collections (a list) of other blocks. The collection may contain a mix of simple and complex blocks, creating a kind of tree structure. The CAnneMultiBlock is a daughter class of CAnneBlock, adding the number of contained blocks as another data item, as well as an array of pointers to those block objects. Note that an Anne'sArk packet is a form of CAnneMultiBlock.
CAnneVstring - as described above, a Vstring is simply a long integer telling the length of the data contained, then the data. The Vstring object contains the length, a pointer to the data, and a notation whether or not this object "owns" the data, which determines whether or not the data will be "freed" when the object is destroyed; and yes, the data would be malloc'ed, to get a nice meaty block of raw memory without type checking.
CAnneBlockFreetext - a multitude of useful information chunks will incarnate as freetext blocks, which are simple blocks, with a Vstring in them. As such, the simple block is the parent class, and this child class includes a Vstring object.
Within the program itself, any relevant information which may be stored in a block or multi-block object, should be stored and used that way. This will enable easy communication of the information to other nodes, and reduce or eliminate the need for translation layers.
O Search Engine Back End For a Real-Time Filesharing Infrastructure: ideas on high-powered searching, good ones mostly, but missing the point that media file sharing implementations do not need super search engines to make them work. Good ideas for scaling up to serious text retrieval systems.
O An alternative approach to routing good thoughts on keeping a traceroute as the packet floods out - can be used for many topology maintenance things, and possibly simplify overall implementation requirements...
O Caching and Mirroring within the GnutellaNet more thoughts on caching, not attractive, I think.
O Stormrose Proposal - interesting ideas about multiple services and extensibility, also insight into the current gnutella ping/pong problem.
O William Wong Proposals: more cache talk, some weak self-organization ideas for the network topology, a simple backward compatible semi-private network, chat (yes, but why don't they just get an IRC client and use that?!?), dangerous (security hole) ideas about sharing sub-directories, MD5..., and a good thought about forwarding unknown packet types to enable legacy servant software to work with new features.
O
Jukka Santala Proposals
Gnutella protocol negotiation speaking
toward the same point as Monaco above, making the higher bandwidth nodes into
a backbone for the network - taking communication load off of the modem connected
nodes. Also interesting insight into the GnutellaNG development politics...
Gnutella HTTP file-transfer good point
about the current protocol deviating from HTTP standards
and how this makes extra
work in using off the shelf libraries... Lots more that will be good reference
for implementing HTTP file transfer (wonder about using HTTPS: for secure connect).
Gnutella routing good thoughts on net
toplogy, disconnection protocol, etc.
O Bowser Proposal - good basic treatment of the gnutella scalability problems, and some solid solution suggestions. I especially like: "Cache the ENTIRE DATABASE of offered files for all connected "slow" links, and handle ALL search requests from the network on behalf of the slow clients."
Michael Carmack shared some ideas about a music distribution model. I think his business model is lacking, but the social commentary about music, MP3.com, and similar issues is right on target with what I have observed at various points on the web.