Parallelism is:
This is the theory, however, parallelism is not guaranteed to work. Some algorithms do not parallelise well, others need to be rewritten in order to be suitable for parallel machines.
To make parallelism effective, need to:
Relatively speaking, communication time is much more expensive than computation time. When a message is sent there is always a `start-up' time (or delay) plus the time taken for the message to be delivered. Sometimes it may be advantageous to perform some extra computations in order to cut down the amount of communications.
Balancing the load ensures that there is little waste of resources due to processors sitting idle.