A diagram of the Pari-distributed architecture.
Pari-Distributed is a client-server framework, written mostly in Erlang, that lets you split up trivially parallelizable loops in Pari/GP across multiple nodes. Its goal is to provide a distributed, fault tolerant network of compute nodes, using multiple computers and/or CPU cores, providing a near-linear speedup in compute times for easily parallelizable math programs computed with Pari/GP.
Although Pari-Distributed is a very young project (launched March 2007), with much work needed before it will be as robust as it should be, it currently works quite well, at least with reasonably simple computations
A rough diagram of how Pari-Distributed works is shown below. A few minor details might change in the near future, such as having a separate node to handle only the output of node's computations.
A diagram of the Pari-distributed architecture.
See the page on detailed installation information for specifics on how to install and run Pari-Distributed.