With the development of computer hardware, it is more and more important to take hardware into consideration when trying to write super efficient code. The speed of a program is not only determined by the code, but also how well hardware supports. This blog is aimed at how to make the best use of your computer hardware.
GPU (Nvidia)
Architecture
kepler – maxwell
Speed, core, bandwidth, instructions per second,
SM = Streaming Multiprocessor
SP = Streaming Processor
SPU
A very comprehensive comparison of the performance of high-end graphic card in training state-of-the-art neural network could be found here.
Cudnn v5 got 3x speed up for 3d conv.
CPU (Intel)
Skylake
Quantum computation
Other issues
Solid State Drive (SSD) sometimes can greatly increase the speed when large disk I/O exists.
Welcome 