Technical information for the development of the Trust-forum project

Main project page

See in this folder http://spoirier.lautre.net/tforum/ the list of documents.  I'm very sorry that the documentation is not well-ordered: there are very big files with very little interest and that may be obsolete (all what was discussed with previous programmers about what had been done or was not well done, the many bugs, some diverse thoughts...), and there are small files that present all the main ideas that you should focus on :

For an introductory presentation from the user's viewpoint,see the user documentation.

Unfortunately, it does not describe everything. For example, it does not describe the admin panel.

Ideas for wiki system
Incomplete List of things to change

Incomplete draft of functions list of the present system

Last remarks and explanations, in connection to the Google Wave concept

History of the project development and discussions

I first tried to have it done by some students. They made this, but their code was not reused by the next programmers.

Then I had correspondance with 2 programming teams in Minsk (Belarus),

Finally, after many errors and waste of time, the Webdevs team abandoned the work and released the last version as it was, Trust-forum-1.4.0, with remaining bugs. 

There may be an interest to previous release (Trust-forum-1.3.1, made in 2006) because the programmers had so badly organised their work that they did developments from a previous obsolete version, then maybe not all correction made for 2006 have been reintegrated in 2007.

After this, the email conversation I had with previous programmers excluding attachments, was put into a big table. (I still have the full list of at least the meaningful excerpts of messages of that conversation from which this table was made).

Attachments have been put as other files of this folder mentioned above .

The rest of the conversation had been done in a private forum inside their installation of the program (it was at trust-forum.php.by, which no more exists). Here is a copy of its content.

The only technically meaningful output from all the messages she sent me, is her remarks (is that true ?) that:

List of user statuses (session types)

About global login system

The GLS aims to be a new open protocol for internet, to authentify users, working on top of other protocols.

It had been done twice, one by Frozen team, then it was redone independently by the webdevs for being integrated with the forum, I don't know why.
It had been initially tested with 2 virtual servers, except that they are only virtually separate, so that Laure suspected that it would work much less good with truly different servers.
I do not know exactly how the GLS now works. I fear it's not very good since Laure said she had it completely redone. But I wonder why, I have no
details. She had first criticised PHP as not a good language for it, then finally she announced a new GLS still entirely in PHP.

The "email user" functionality is a temporary feature to make life easier during the expansion of the use of the program, until it will be of universal use. Indeed, as long as it is not universal use, there needs to be a way to communicate between users and not-yet-users who only have email, that is called, an email user.
Such users can access forum just like a remote user by GLS, by using a key in received email messages; and receive the contents of the messages written by others, to their email address.

It would be interesting to call hosts by their own new names we would invent in our network, to avoid lengthy and uninteresting domains or URLs. (not necessary in the short term).

Trusted network and PKI : about invitation of new hosts to the network

Each site will have a pgp key pair, and the list of all other hosts of the network, with parameters
- Name of the site
- Public key
- A symmetric key just for this pair of hosts, to let secure communication go faster
- addresses to exchange information between hosts (may depend on what kind  of information; for example, authentication requests from a user as below)
- Identity (host and pseudo) of who invited this host, date and signature of the declaration by which this host with its public key was invited.

Here is the method to develop the network and maintain this list, though invitations of new servers into the network by users of existing servers :

The software to install to a new hosting will contain some public keys of existing servers already running the software, to initiate the secure communication with them.

Every declaration once done should be forwarded to all other sites of the network

Regularly, each site will produce a confirmed updated version of all the declarations of its users (either separately or globally signed ??) and send it again to all other sites or leave them available upon request (what is best ??). The certification of each server B will be available from any given trusted server A in the form of a chain of certifications by existing servers from A to B if it exists.

At the start of the above invitation procedure between servers, the new server C will request from A or B a certification of the key of the server B hosting the account of C's admin's friend, through such a chain of certifications from A to B where A is one of the well-known server whose key came together with the software.

Therefore, the new server C will know correctly the key of B, and can send a crypted message to his friend in B, that will recognize it according to content and invite host C to the network.

Later in the development of the project, we could replace the invitation by the following recognition in two parts:

- Certification of authenticity of the public key, or equivalently, the authenticity of a message received from a user of the other site, if the message was received crypted and signed: "I certify that this message was really written by this user", therefore his key is  authentic because as it is a crypted message, third parties could not have read it to know what it says and sent another message with similar content with another key. So this should be easy to establish.

- The trust to the root of another site, as a trust to the fact that this other site runs the program correctly and the root makes a fair use of his admin right.

Global login system and bookmarks

Each user can put on his account a list of bookmarks. Some are simple bookmarks. Others bookmarks are those of sites that share the following protocol, and are associated to one of his pseudos (so for each pseudo he has a list of such bookmarks). Once logged in in his usual host1 that contains his main account, he can click there on one of his bookmarks to host2, so that host2 getting in the request the previous URL containing the name of host1, establishes with host1 an authentification procedure of the user.

Technically, the link will be to host1 that redirects to host2 with crypted authentication request. So host1 will be contacted first in an authenticated way and establishes the authentification with host2. Then he connects to host2, being authenticated there as "pseudo@host1".

More precisely, to relate to the rest of the project: there are two different kinds of login: the fundamental one is login as a user, at the user's main account at host1 (in general, each user has only one main account at one particular host), which gives access on this host to all functionalities of the project including the use of one's different pseudos. Then, taking one of one's pseudos, be authenticated to another site host2, only as this pseudo "pseudo@host1" and not as a main user, without using the keyboard, by a request from host1, that gives access to some restricted set of functionalities that is not defined yet, and that would anyway remain open to development by anybody that wants to open a new site that offers new services. These functionalities may depend on information given by the first site. So here is a tool by which the second site can recognize visitors as authenticated as a pseudo from another site.

The method proposed now goes along the following lines:

Denote A = host 1, B = host2, U= user.

1) U logs into A
2) U requests A his decision to visit B.
3) A crypts the authentication information (maybe U's IP, U's pseudo, time of request, prefered language and what page one wants to access, and possibly more info in the future) in a compact form to send to B with its symmetric key with B
4) A includes this crypted information in the redirection request for U to connect to B
5) B receives the request from U and decrypts the information, knowing that it is from A to know which key to use thanks to the info of the previous URL that goes with the request.
6) If all is OK then U is logged in in B, with session variables containting info from the crypted data.

Another idea for the future, that would probably require a plugin to the browser: when the user logs in to A, he creates a private and public key pair, sends the public key to A, then A makes a certificate and sends it to U, so that U can log into any site he wants by signed automatic login requests with this new key and this certificate. When U wants to log out, the browser will send logout requests to all the sites which were logged into by this certificate, then destroys the private key and certificate.

After this, a next development of the project would be to make a secondary login form: once authenticated at host2 by the procedure above, one can create at host2 a new password to be able to log in directly next time at host2 with no need to request host1, by typing pseudo + new password. Host1 need even not be aware that such a secondary pass at host2 is made.

Remark about folders and subscription to a forum

A user can have in his folders links to forums hosted elsewhere, and let this work the same as if inside the same site:  if a user is subscribed and there is a new message in forum then the user sees the link to forum in his main box.
To do this, the forums must send info to the hosts of every "subscribed" user every time a message is posted or made invalid again by another user, so that the info of what to display in each folder (how many new messages) is locally available in the home host of the user.
Problem: if a message is posted but then made invalid again, then the news of presence of a new message, that had been sent to the hosts of subscribed users, should be cancelled.
Problem: if those user were logged in, they probably already saw the signal of the presence of a new message. Then, if it was made invalid, those user would come and not find anything...

Initial ideas for folders, that were not fully implemented yet. Maybe we should do something like that now !

Everyone can reorganise as he wishes the visual appearance and structure of the forums and threads he can access, including classification of messages (by date or threads), with folders and subfolders.
More precisely, we can imagine that once logged in we see a list of boxes in the left margin, like

Inbox
box 1 (429, 2 new)
box 2 (231, 10 new)
Folder 1
Folder 2
all forums

"Inbox" is what is immediately displayed in the main fraim.
"box x" is in fact renamed anyway, the same for "Folder  x"
Each box is like a forum organised in threads.
Each Folder is a list of more boxes.

But the description above is just the user interface, that each user can organise as he likes. Underlying this is the "real set of forums", each of which has its parameters, and the "reading rights" relation with the users. Each user builds his configuration on the set of forum he has the right to read, and whose list appears in his "all forums" folder. He can configure each forum he as access to, to appear in the box he wants. He can also configure each new forum that others will invite him to, to appear in one or another of his boxes according to his public identity involved and the state of the trusting relation he has with the inviter.

We could also study the possibility to link conveniently a reference to another forum or message inside one message.

Later, if there is nothing more important to do, a possible way to improve the features (but may be difficult) would be to allow anyone to choose the editing conventions and list of smilies he wants, whether it be for forum messages or for wikis (admittedly, a given wiki page could hardly be edited in other conventions but...).

From a former folder implementation, I had described the following modification to do, but finally they interpreted and made it another way:


The tree structure in forums is not useful, so it may be kept or dropped, according to what is more convenient for programming. There will be the following folders:
- Main box
- Possible other folders, created by user
- One folder of all public (and semi-public) forums of this host.
- List of remote sites, = bookmarks, whose target is the list of public and semi-public forums in each.

As it already works, private forums can be put to one of the user created folders, in which case they will always appear there; moreover they will appear in Main Box, if (they are not in another folder) or ((user subscribed to it) and (there is new message)).
But, which is new, public and semi-public forums can be put in any user folder including mainbox; they will also appear in mainbox if ((user subscribed to it) and (there is new message)). Anyway, they will always appear in the Public forums folder (public forums above, semi-public below, with separation).

User has possibility to display folders with, below each forum, the forum's comment. So on top of folder is the link "Show comments" or "hide comments".
With folders will be displayed the nature of every forum (public or private) but maybe not yet now because it would be perhaps too heavy on the screen. We will discuss it together with design, how to show it in a light way.

About email users

How I think things should be (how are they now ?):
When a posted message is sent to an email address, on bottom of email message will be a link to the forum with: forum name or id; invitation id of that email in the forum's invitation table; authentication code. This code is changed every time an email is received. If an older or wrong code is used, a new email message is sent with a new code.
When the email user finally registers, he remains user of this forum and sees it in his folder.

The headers of the email message sent when one posts message to a forum, is like in a mailing list:
Sender email address is the author pseudo@host (does not work to reply);
Sender name is defined by Comment of this pseudo;
Subject is: [Forum custom name] Forum message subject.

 (There was considered to be a webmail but it had to be restricted and replaced by an added functionality to contacts system, that can send messages but not receive nor store, because if I understood well from what programmers said, php programs would not be able to access the unix function of opening new mailboxes).

Some details about forum invitation

If things are as they should be, they are this way:

There are 2 conditions:

This way, if a user is invited to a right and then invited to a strictly higher right, the last inviter name would overwrite the previous one.

Function of calculation of effective right of a user, that is called every time the forum functionality depends on this right:

  1. take default
  2. increase by invitation
  3. decrease by denyance.

Other aspect of rights system is that there is ONE owner of the forum that is "the head" of all admins (this has been discussed by mail...) : that is the only admin that can deny other admins except himself; he can abandon his title in favor of another admin; this is necessary first if he wants to resign his invitation, unless it is a private forum and he is the only user, so that the forum is deleted).

Timeout and posting messages

A problem was to prevent the situation when user takes a long time writing a long message, but then, when user wants to post the message, as it was so long, session was closed by timeout so that posted message is lost.
The idea that should have been implemented (is it ?) was that message would anyway be saved as invalid, and user would have to login again to validate it.
Another idea, that I think would be preferable, is to make regular automatic saving (and why not previewing ? of course in a WYSIWYG editor it makes no difference) of the message, like in gmail for example.

About dating system

A first version has been done in the previous release (Trust-forum-1.3.1), that you may re-use to save some time, but it was very incomplete and unsatisfactory. 

The last full specification for what should be done about it is here, knowing that users exist before creating their dating file, and that the personal
web pages/wiki and the public and private forums that are involved there, are just involvements of the other parts of the project into this one, so that these other parts must be conceived in such a way that such involvements will be possible.

(This file is a copy of the big table where all that does not concern dating until Feb 27 2005 has been deleted - in 2006, the corrections and development of the dating system has been discussed but not done).

Problem: what exact list of parameters and their values should be included in this system; in particular, develop a list of possible interests...

Some technical wonders from an amateur's viewpoint

Well, I recognize that there is indeed a large mixing in a sense, by this superficial remark : whether I browse folders, forums, a list of contacts or configuration, urls are all of the form

http://trust-forum.php.by/index.php?action=...

Therefore it is always the same program file that is executed.
I look at the code and I find it is 89 kb. Apart from that there
dbaccesses.class.old.php (75 kb)
gls.class.php (100 kb)
dbaccesses.class.php (65.03 ko)
datingaccess.class.php (33 kb)
all the 20 other files in the main directory (64 kb), which seems like an imbalance ...
It may be necessary to reorganize everything otherwise?
Well, lots of other heavy files in the "action" folder, so it may not be a relevant remark.
One reason that everything is in the same file, may be the need for the same header and margins. Yes, but the forums will be accessed through GLS, with different headers and margins, so?


Well, apart from that, a technical issue which I fear has not been addressed seriously by programmers previous:

Presenty there are, it seems, one mysql table for the entire forums posts (public or private). I am afraid that this is a bad choice, since there will have to be on every server several thousand accounts, and consequently, several tens of thousands of forums, so many hundreds of thousands of messages. To view the contents of a forum, it might take significant cpu resources, right?
To them I had expressed my fear at a time, and they responded considering that there was no problem because SQL servers are fast enough, and that there are also forums with a single messages table.
Yes, but it's only one public forum theme with no more than a few hundred discussion threads. I want a system that averagely encompasses all messages written by thousands of users in their entire web experience, so it's a different range of quantities.

And if we ever we want to add a search function in the body of personal messages, it might unnecessarily slow in searching throughout all recorded messages!

Drafts of ideas about technical strategies and skills (by programmers who finally did not join)


Programming langages : As a software for web servers and also to let people the possibility to run copies in their websites, the programming language used until now was PHP/Mysql. It is open to possible development in other languages if it is not difficult to find hostings and if it interfaces well with the rest of the project.
Later will be the trust system, which will have an aspect of web interface (now in php), and an aspect of internal computation of the transitive relation (or preorder) generated by a given relation (in database) between thousands of points. What language would be good for this ? 

One programmer (Andrew, that finally did not join) told his thought about required skilled as follows:

1. Project leader

Required skills: analysing, cooperating, investigating, listening, written communication, creative problem solving, numeracy, organising, client-handling skills.

Languages/technologies:
- PHP - Javascript - MySql - familiar with other technologies like XHTML or AJAX

2. Programmer
Skills: Teamworking, analysing, investigating. attention to detail. Discovering new facts or concepts. Problem solving. Logical reasoning. Creativity. Good technical knowledge. Ability to work to deadlines. Determination.

Languages/technologies: Javascript - C++ - PHP or C#, ASP.NET - Mysql, Sql Server - AJAX

3. Designer, flash theme programmer (may be later)

Flash - Adobe Photoshop - HTML

Later we need:

4. Database Analyst/Designer: analysing, cooperating, planning & organising, problem solving, logical thinking. 



Another programmer considered helping but does not have
time now. His view of the work was that it can be divided into 2 parts,
with just one programmers in each; himself considered to work on the back-end part,
until he decided to not work full-time:


Work can be divided in two parts:
Back-end
Front-end
This will be constituted by the database and the trust forum server.
Products:
- PostgreSQL: "I know it fairly well and I think that it is powerful enough. I have used also MySQL but I think that PostgreSQL is more adapt. In any case I know it better."
- Perl or C. "I think that the server can be done in straight Perl. I have already some code already working. It will not too  difficult to adapt my code to your project."
  "Later in the project a conversion of some critical parts to C can be thought, but only if necessary."		 This will be constitued by the web server and the web application
Products:
- Apache: this is the "de facto" standard.
- Catalyst is a web application framework written in Perl, very, very powerful and efficient, and, after a somewhat long learning curve (one-two weeks), very easy to use.
- Template Pages: this is a product which allows the creation of dynamic pages, with a template language. Also in this case the product is very sophisticated and allows the creation of professional pages with little effort.
- CSS (Cascading Style Sheets). This is the standard used to make the
pages all look the same. Template pages use CSS

General links that might be useful (or not) (I don't know what to do with these links, could someone help me put order there ?)

http://php.resourceindex.com/
http://www.phpscriptsdirectory.com/
http://www.hotscripts.com/PHP/Scripts_and_Programs/index.html

http://www.hotscripts.com/PHP/Web_Sites/index.html
User Authentication
User Management
http://www.hotscripts.com/PHP/Scripts_and_Programs/Virtual_Communities/index.html

http://www.hotscripts.com/PHP/Scripts_and_Programs/Email_Systems/Web-based_Email/index.html http://www.hotscripts.com/PHP/Scripts_and_Programs/Email_Systems/Web-based_Email/index.html
http://px.sklar.com/section.html?id=58
http://php.resourceindex.com/Complete_Scripts/E_Mail_Utilities/
http://php.resourceindex.com/Complete_Scripts/E_Mail_Utilities/Web_Based_E_Mail/

dmoz:
http://dmoz.org/Computers/Programming/Languages/PHP/
http://dmoz.org/Computers/Software/Internet/Servers/Message_Boards/
http://dmoz.org/Computers/Software/Internet/Servers/Collaboration/
http://dmoz.org/Computers/Software/Internet/Servers/Mail/
http://dmoz.org/Computers/Software/Internet/Internet/Clients/Mail/Web-Based/

Classified ads:
http://php.resourceindex.com/Complete_Scripts/Classified_Ads/
http://php.resourceindex.com/Complete_Scripts/Classified_Ads/Matchmaking/
http://www.phpscripts-fr.net/scripts/scripts.php?cat=Petites+Annonces
http://www.hotscripts.com/PHP/Scripts_and_Programs/Classified_Ads/General/index.html

Other programs, projects or works that have some common subject with this project

DEM project: An open source micropayment system based on Java, JXTA, XML, OpenPGP and SSL - it list other projects of online payment
Central Authentication Service at Yale University; other single sing-on solutions
Qare
Ripple : project of a peer-to-peer money system
Alfarez Abdul Rahman's Research Documents on trust
(Poblano, a distributed trust model for p2p networks)
www.p2ptrust.org
Slashdot: Distributed trust metrics ? (for forums)
Python Trust Metrics. Implementation of Ford-Faulkersson Maximum Flow algorithm and mod_virgule's Trust Metrics code, in python
Open Source Applications Foundation and their Chandler project (of a user-friendly portable communications sytem in python)
Livejournal and its trust metrics community and Trustflow system (with FAQ).
http://www.identitycommons.net/
The PGP Web of Trust
Linkfilter.net
The PlaNetwork Consortium
One-of-us with its newest project One-of-us.org'ster
A wiki listing all trust metrics
In this list, there is Openprivacy.org that focuses on combining anonimity with reputation by special complex cryptographic algorithms when exchanging trust certificates to clients softwares, without any help of central servers.
IM2000 : proposal for a new protocol of mail system in which messages would be stored in the sender's box instead of the receiver's.
More email protocols here and there

The Free Haven project
The Augmented Social Network (html version of the whitepaper here)
thetransitioner: an interesting web site including TikiWiki

Cordance : creating the Dataweb
(The Chaordic Commons)

(Liberty Alliance Project : not a good partner according to what is explained at the Augmented Social Network project)
(ODP Passport identification stuff ??)

The FOAF project (Friend of a friend) points out the use of the RDF format for dating files. A discussion on this subject continued as a wiki page.

Recently suggested (July 2008)


http://listcultures.org/pipermail/p2presearch_listcultures.org/
http://groups.google.com/group/CooperationCommons
http://drupal.org/

Why not to use OpenId

Many people wanted me to have my project compatible with OpenId.
I think it is a very bad idea because it is incompatible in priciple with what I want to make:
I want :
I don't want:
Back to project page