The Control Unit of the CPU is responsible for decoding instructions. It interprets the instructions and initiates the necessary actions to execute them. The primary memory holds the instructions and data, the program counter keeps track of the address of the next instruction to be executed, and the ALU (Arithmetic Logic Unit) performs arithmetic and logical operations.