GPUs are co-processors to the CPU where the goal is to rapidly manipulate memory. GPUs use parallel processing to handle multiple tasks simultaneously.
This is a fundamentally different type of machine than the CPU, where GPUs are wide. GPU’s aren’t optimized for long sequential programs, but rather short computations that can be easily parallelized. They can’t run system calls, have I/O, networking, or OS access. Nvidia makes the most popular GPUs programmed with CUDA.