|
a/Allura/docs/platform.rst |
|
b/Allura/docs/platform.rst |
|
... |
|
... |
| 2 |
===================================
|
2 |
===================================
|
| 3 |
|
3 |
|
| 4 |
I'm told that the reason you build a platform is to "reduce the marginal cost
|
4 |
I'm told that the reason you build a platform is to "reduce the marginal cost
|
| 5 |
of developing applications." Sounds good. Well, actually it sounds a bit
|
5 |
of developing applications." Sounds good. Well, actually it sounds a bit
|
| 6 |
dry. But it's about right, we want to make creating new online development
|
6 |
dry. But it's about right, we want to make creating new online development
|
| 7 |
tools faster, easier, and more fun, which i guess is the "reduce the marginal
|
7 |
tools faster, easier, and more fun, which I guess is the "reduce the marginal
|
| 8 |
cost" thing.
|
8 |
cost" thing.
|
| 9 |
|
9 |
|
| 10 |
Platform building blocks
|
10 |
Platform building blocks
|
| 11 |
---------------------------------------------------------------------
|
11 |
---------------------------------------------------------------------
|
| 12 |
|
12 |
|
| 13 |
Before we get into the details of how to extend the allura platform, perhaps
|
13 |
Before we get into the details of how to extend the Allura platform, perhaps
|
| 14 |
it would be smart to explain some of the big pieces and why there are there.
|
14 |
it would be smart to explain some of the big pieces and why there are there.
|
| 15 |
|
15 |
|
| 16 |
We wanted PyForge tools to be fast, we needed them to scale, and we had some
|
16 |
We wanted PyForge tools to be fast, we needed them to scale, and we had some
|
| 17 |
complex requirements for data storage and extensibility. So, we needed a
|
17 |
complex requirements for data storage and extensibility. So, we needed a
|
| 18 |
**fast,** flexible, and easy to use data persistence system.
|
18 |
**fast,** flexible, and easy to use data persistence system.
|
|
... |
|
... |
| 32 |
because we wanted app tools to be able to:
|
32 |
because we wanted app tools to be able to:
|
| 33 |
|
33 |
|
| 34 |
* create and version their own document types,
|
34 |
* create and version their own document types,
|
| 35 |
* extend existing document structures,
|
35 |
* extend existing document structures,
|
| 36 |
* and to mange document revisions, access control lists, and other
|
36 |
* and to mange document revisions, access control lists, and other
|
| 37 |
platform level data.
|
37 |
platform level data.
|
| 38 |
|
38 |
|
| 39 |
In spite of the power and flexibility of the Roundup HyperTable
|
39 |
In spite of the power and flexibility of the Roundup HyperTable
|
| 40 |
implementation, we had some concerns about performance and scalability.
|
40 |
implementation, we had some concerns about performance and scalability.
|
| 41 |
|
41 |
|
| 42 |
Fortunately several of the PyForge authors (including me) used MongoDB
|
42 |
Fortunately several of the PyForge authors (including me) used MongoDB
|
| 43 |
in rewriting the download flow of SourceForge.net, and new that it could
|
43 |
in rewriting the download flow of SourceForge.net, and knew that it could
|
| 44 |
handle huge loads (we saturated a 2gb network connection on the server
|
44 |
handle huge loads (we saturated a 2gb network connection on the server
|
| 45 |
with 6% cpu utilization).
|
45 |
with 6% cpu utilization).
|
| 46 |
|
46 |
|
| 47 |
We also knew that MongoDB's flexible replication system would allow us
|
47 |
We also knew that MongoDB's flexible replication system would allow us
|
| 48 |
to build the forge in such a way that we could easily provide a
|
48 |
to build the forge in such a way that we could easily provide a
|
|
... |
|
... |
| 52 |
*Non*-Relational Mappers (ONRMs?) before, including one for MongoDB,
|
52 |
*Non*-Relational Mappers (ONRMs?) before, including one for MongoDB,
|
| 53 |
and he whipped up Ming, which backed on MongoDB and gave us exactly
|
53 |
and he whipped up Ming, which backed on MongoDB and gave us exactly
|
| 54 |
what we needed.
|
54 |
what we needed.
|
| 55 |
|
55 |
|
| 56 |
As I mentioned before we also needed a fast, flexible message bus and queuing
|
56 |
As I mentioned before we also needed a fast, flexible message bus and queuing
|
| 57 |
system. RabbitMQ was(lightning) fast, (shockingly) flexible, but not supper
|
57 |
system. RabbitMQ was (lightning) fast, (shockingly) flexible, but not supper
|
| 58 |
easy to use. Fortunately we didn't have to roll our own wrapper here, as
|
58 |
easy to use. Fortunately we didn't have to roll our own wrapper here, as
|
| 59 |
the python community already whipped up Carrot, and Celery, which made
|
59 |
the Python community already whipped up Carrot, and Celery, which made
|
| 60 |
working with the RabbitMQ based AMQP bus a LOT easer.
|
60 |
working with the RabbitMQ based AMQP bus a LOT easer.
|
| 61 |
|
61 |
|
| 62 |
|
62 |
|
| 63 |
Pluggable Event Listeners
|
63 |
Pluggable Event Listeners
|
| 64 |
---------------------------------------------------------------------
|
64 |
---------------------------------------------------------------------
|
|
... |
|
... |
| 70 |
|
70 |
|
| 71 |
**Auditors** are hooks that get called when events/messages come in,
|
71 |
**Auditors** are hooks that get called when events/messages come in,
|
| 72 |
they can modify the message before it is persisted to the document
|
72 |
they can modify the message before it is persisted to the document
|
| 73 |
store (via MongoDB).
|
73 |
store (via MongoDB).
|
| 74 |
|
74 |
|
| 75 |
Once the message is saved to the document store, it is then queued up for another set of hooks -- **reactors** -- that are not allowed to change the
|
75 |
Once the message is saved to the document store, it is then queued up for another
|
|
|
76 |
set of hooks -- **reactors** -- that are not allowed to change the
|
| 76 |
message, but can do things like send e-mail or push a new kind of event
|
77 |
message, but can do things like send e-mail or push a new kind of event
|
| 77 |
onto another queue.
|
78 |
onto another queue.
|
| 78 |
|
79 |
|
| 79 |
Nearly everything in roundup is implemented as either an auditor or a reactor, and PyForge definitely steals that idea and runs with it.
|
80 |
Nearly everything in Roundup is implemented as either an auditor or a reactor,
|
|
|
81 |
and PyForge definitely steals that idea and runs with it.
|
| 80 |
|
82 |
|
| 81 |
TODO: Finish reactor overview (after reactor code is written)
|
83 |
TODO: Finish reactor overview (after reactor code is written).
|
| 82 |
|
84 |
|
| 83 |
|
85 |
|
| 84 |
Application Tools
|
86 |
Application Tools
|
| 85 |
---------------------------------------------------------------------
|
87 |
---------------------------------------------------------------------
|
| 86 |
|
88 |
|