I. Definition of Parallel Computing (overview)
Parallel Computing has really close relationship with parallel processing and parallel programming.
Parallel Processing is a usage of more than one CPU to execute a program simultaneously. So it could be imagined how the result of our program using parallel processing. The job must be done in a short time because it has more than one worker. It means parallel processing help the program to run faster. But in reality, when we do the parallel processing, the process is hard to be separated to different CPUs as executors without dependencies between a CPU and another.
Parallel computing is one of the technique to do computations simultaneously using multiple computers. Parallel computing is a middleware that acts to control jobs distribution between nodes in a parallel machine. Therefore, users should create a parallel-designed program to use the machine.
Fig. 1.1. Single processing scheme.
Fig. 1.2. Single processing example.
Parallel Programming is a computer programming technique that make simultaneous operation execution (parallel computing) in a multiple single-processor computers or multiple multi-processor computers that connected in a network. The computer that is connected to network and used together to executing a single program is usually called distributed computing.
Fig. 1.3. Parallel processing scheme.
Fig. 1.4. Parallel processing example.
In communities, a computer usually has only a CPU, some others have more than one CPU. They could be two or four. So what makes a problem is, how a computer that only has one CPU could do the parallel processing jobs? As discribed before, parallel processing uses more than one CPU to execute a program simultaneously. The Answer is : The single core CPU is connected to a network, and the network is connected to a multiple single-processor computers cluster or a multiple multi-processor computers cluster.
II. Who Should Use Parallel Computing...