dGroups README -- Seth Lakowske (lakowsse at uwplatt.edu) (C) 2001 -- under the GPL This README is meant to be used as a starting point for a possible opensource p2p network. Additions and recomendations are welcomed. dGroups definitions: Group: A group consists of a list of nodes(computers) which have a simularity. A group is also required to have 2 hard=links to other groups. Group examples - 1) If three computers all had a copy of the same file they would form a group. 2) If ten computers all wanted to chat about linux they would form a group. Link: A list of network(IP) addresses representing the next or previous groups. Hard=Link: a link between groups which is defined by the group. notated by a_group <===> z_group Soft-Link: a link between groups which differs from the actual link defined by the group. notated by a_group <---> z_group Metadata: indexing data, not the data itself. the date used to tell you were to find data. analogous to the index in the back of books How it Works: dGroups uses a sorted network linked list of groups. Each group is arranged in the list by alphabetic order from greatest (a...) to least (z...). Every group is linked to the previous and next group alphabetically by name. Using these preconditions a node can navigate up and down the whole linked list of groups by knowing just one group. -- A side note: though only one group is needed to navigate the network a node is not limited to just one group and in fact is encouraged to know about additional groups to speed up searching, which will be explained later. In reality a node might know of thousands of differnt groups. Searching: Searching Requirements: A node(client) must have a local cache of a least one group in the group list. Additional groups can be placed in the list by using soft-links local group cache example: | | \*/ \*/ gtk_doc.ps <===> linux_chat <===> mesa3d.tgz <---> QT_doc.tgz <===> radiohead.mp3 <===> stp.mp3 As you can see we know about the groups linux_chat and radiohead.mp3. By definition a group is linked to two other groups seen above as gtk_doc.ps, mes3d.tgz and QT_doc.tgz, stp.mp3. We link this all together with a soft-link between mesa3d.tgz and QT_doc.tgz which represents our lack of knowledge about what is in between these two groups. Searching: How it all works search string: microsuck.jpg First a search string is input by the user. In this case "microsuck.jpg" is used. A search is started by the local node by searching his/her local group cache. The node searches his local list of groups for the closest match to the search string. In this case "mesa3d.tar.gz". Because mesa3d.tar.gz has a hard link to the next group below it we can forward the search on to the nodes in the "mesa3d.tgz" group. We know we are getting closer by forwarding it to mesa3d.tgz because microsuck.jpg < mesa3d.tgz alphabetically. The process of forwarding is continued by the mesa3d.tgz group until one of two conditions occurs. either a) the search gets forwarded to "microsuck.jpg" or b) "microsuck.jpg" is determined not to be on the network. the search is sped up considerably if members of the mesa3d.tgz group know of many different groups or already know of the microsuck.jpg group. If a group is popular many other groups will know about it and therfore the search will return quicker. Less popular groups will not be returned as fast but are gaurenteed to be returned eventually with the worst case being n, n being the number of groups on the network. Another important note, when a search is forwarded to a group it is forwarded to all members of the group. These members will then if necissary forward the search to the members of the next group. During this process you can see that the search will always get relayed as fast as possible because the group members with the lowest latency will recieve the search before other group members and will start relaying it before other members. The speed up is self perpetuating. these numbers represent the lowest latency between members of the different groups. a < 50ms > b < 68ms > c < 110ms > d Support for keywords: Keywords are supported but, I need to type it up yet. What dGroups is and what it isn't: dGroups is a great new way to search for metadata. dGroups isn't a network tool swiss army knife. instead modules(ftp, chat, IM) will use dGroups` metadata to carry out there desired services. v0.0.1 -- Being the first version it will be very minimal. It will consist of text interface that is proof of concept, however that is not to say it will not be useful. one will find it easy to find groups and hosts of interest on the network. The text based client will also be very usefull in debugging the network code. v0.0.5 a graphical interface created. By this point the network bugs should be worked out and we should be getting ready to role out module support. v0.1.0 Module support added. a user will be able to quickly add and remove new modules such as advanced file transport protocol and robust chat modules. v0.2.0 bugs worked out of modules and dGroups module code. vX.X.X development continues Modules already planned: -- I haven't started work on any modules yet but these are the planned modules. dHTTP -- an advanced HTTP client/server module. Allows resuming, starts stop at any point in the file. allows downloads to be distributed amongst group. For Example, if i wanted to download the file associated with the "mesa3d.tgz" group i would split the file into however many nodes there were in the group. lets say 175. I would then connect to each node telling it what chunk of the file i wanted. All of this would take place concurrently. 175 56k modems can really pound your t1 line :) I plan to help a little with development of this module however i hope that someone else who has more experience with HTTP would step forward for the cheif development of this module. dIRC -- a replacement for the IRC client/server. try to port as many features over from the original and add a few more. I hope someone intensily intrested in IRC development picks up this chief development of this module.