Introduction

The new Forge is implemented as a collection of plugin applications on top of a robust and open platform. Some of the services provided by the platform include:

• Indexing and search
• Authentication and Authorization
• Email integration (every plugin application gets its own email address)
• Asynchronous processing via RabbitMQ
• Simple autolinking between different artifacts in the forge
• Attachment handling
• Plugin administration

Plugins, on the other hand, provide the actual user interface and logic to manipulate forge artifacts. Some of the plugins currently impemented include:

admin

This plugin is installed in all projects, and allows the administration of the project's plugins, authentication, and authorization

search

This plugin is installed in all projects, and provides the ability to search a project for various types of artifacts.

home

This plugin is installed in all projects, and provides the ability to customize the project landing page with "widgets" shared by other plugins.

SCM

This plugin allows you to host a version control system in the Forge. It also provides the ability to "fork" another SCM in order to provide your own extensions.

Wiki

This plugin provides a basic wiki with support for comments, attachments, and notifications.

Tracker

This plugin provides an extensible ticketing system for tracking feature requests, defects, or support requests.

Forum

This plugin provides a forum interface with full email integration as well. The forum also handles attachments to posts either via the web interface or via email.

Ming Databases and the Context Object

There are a few databases you need to be aware of when developing for the new Forge. The Forge maintains a 'main' database which contains the project index and user list as well as a project-local database for each project (sub-projects share databases with thier parent databases, however). Most of the time, you should not need to worry about these databases as they are automatically managed. The automatic management is based on various properties of the Pylons "context" object c:

project

The current project, used to determine which database to use for other objects

app

The current plugin application object.

user

The current user

The Forge platform provides the following functions to manage the context object:

Artifacts

We've mentioned artifacts a couple of times now without definition. An artifact, as used in the new Forge, is some object that a plugin needs to store in the forge. The platform provides facilities for controlling access to individual artifacts if that's what the plugin designer favors. For instance, the Forum plugin allows a user to edit or delete their own posts, but not to edit or delete others (unless the user has the 'moderate' permission on the forum itself). Some examples of artifacts in the current plugins:

• Forum: Forum, Thread, Post, Attachment
• Wiki: Page, Comment, Attachment
• Tracker: Issue, Comment, Attachment
• SCM: Repository, Commit, Patch

In order to implement your own artifact, you should override at least a few of the methods of the pyforge.model.artifact.Artifact class:

from ming.orm.property import FieldProperty
from pyforge.model import Artifact

class NewArtifact(Artifact):
    class __mongometa__:
        name='my_new_artifact' # collection where this artifact is stored
    type_s = 'My Artifact' # 'type' of the artifact used in search results

    # Add your own properties here (beyond those provided by Artifact)
    shortname = FieldProperty(str)

    def url(self):
        'Each artifact should have its own URL '
        return self.app.url + self.shortname + '/'

    def index(self):
        'Return the fields you want indexed on this artifact'
        result = Artifact.index(self)
        result.update(type_s=self.type_s,
                      name_s=self.shortname,
                      text=self.shortname)
        return result

    def shorthand_id(self):
        'Used in the generation of short links like [my_artifact]'
        return self.shortname

Platform services provided for artifacts

Whenever you create, modify, or delete an artifact, the platform does a couple of things for you:

• The artifact is added to the index and will appear in searches
• A shortlink is generated for the artifact (e.g. [MyWikiPage]). This allows you to reference the artifact from other artifacts. For instance, you might want to reference [Issue#151] from [Commit#abac332a]. Whenever the commit message is displayed in the SCM plugin, any references to [Issue#151] will be automatically linked to that Issue's page.

Shortlinks work only within a project hierarchy (in order to link to some other project's page, you'll have to use the full URL). Sometimes, a shortlink may need to be differentiated based on its location in a subproject or in one of many plugins of the same type within a project. In order to do this, shortlinks may be prefixed by either the plugin mount point or a project ID and plugin mount point.

For instance, suppose we have an issue tracker mounted at projects/test/tracker with Issue #42 in it. Further suppose that there is an SCM repository mounted at projects/test/subproject/repo. A user could push a commit to that repository with the commit message [projects/test:tracker:42] - Fix weird issue. If you then examined the commit in the SCM plugin, the shortlink would be clickable and would take you to the issue itself. The Tracker plugin would also list the commit message as a "related object" in a sidebar to allow for quick cross-referencing.

Asynchronous Processing

Much of the actual functionality of the new Forge comes from code that runs outside the context of a web request, in the reactor server (invoked by running paster reactor development.ini. Asynchronous processing is performed by two types of functions, auditors and reactors, differentiated as follows:

Auditor

Auditors listen to queues on the audit exchange. Messages sent to an auditor queue are interpreted imperatively ("do this"). Auditor-type messages should specify a project ID project_id, an application mount point mount_point, and a user ID user_id, which will be used by the platform to set the context before calling the registered callback, and all of which reference the recipient of the message. An auditor callback function is called once for each message received on its queue.

Reactor

Reactors listen to queues on the react exchange. Messages sent to a reactor queue are interpreted in an advisory manner ("this was done"). Reactor-type messages should specify a project ID project_id and a user ID user_id, which will be used by the platform to set the context before calling the registered callback, and all of which reference the source of the message. If the reactor callback is an instance method, it will be called once for each instance of the plugin that exists for the given project for each message received on its queue. If it is a class method, it will be called once for each message received on its queue. For instance, the Tracker plugin may be configured to react to SCM commit messages in order to generate links between SCM commits and Tracker tickets. All tracker instances in a project will be notified of SCM commits in such a case.

In order to create a callback function for an auditor or a reactor, simply add a method to the plugin application class that is decorated either with the @audit or the @react decorator. For instance, the forum plugin defines a reactor on the Forum.new_post message:

@react('Forum.new_post')
def notify_subscribers(self, routing_key, data):
    ....

If there are a large number of reactors, you can define them in a separate module and use the mixin_reactors() method as in the SCM plugin:

from .reactors import common_react, hg_react, git_react, svn_react
...
class ForgeSCMApp(Application):
    ...
mixin_reactors(ForgeSCMApp, common_react)
mixin_reactors(ForgeSCMApp, hg_react)
mixin_reactors(ForgeSCMApp, git_react)
mixin_reactors(ForgeSCMApp, svn_react)

$ paster reactor_setup development.ini

In order to actually send a message to either the audit or react exchange, a helper method is provided in the pylons global object g: