This is incorrect.

Certain applications - notably realtime 3D graphics and machine learning - need massive parallel processing capability which a GPU provides. If you only have a CPU, you can process in series faster, but you can't do millions of simple ops fast as you can on a parallel device.