CPUs are made of several highly important parts, and the ALU (arithmetic and logic unit) is definitely one of them. All the integer arithmetic and logical operations a computer does is done by two register values being passed to the ALU for computation. Everything from high-tech lab computers to simple microprocessors have an ALU somewhere inside or else they wouldn't even be able to maintain a timer. Of course, the former have much more powerful and complex ALUs.
An ALU bitsclice lying on a page of logic gate diagrams
Modern processors perform these logical and arithmetic operations using two's- complement binary number representation that the ALU reads and computer before spitting out the result for other CPU components to deal with. However, two's-complement is not the only method of representation possible for ALUs to process - some can run one's-complement, sign magnitude format and true decimal systems, though those were more common historically than in today's world.
Most processor operations in computers with any sort of complexity are performed by multiple ALUs loading register data input and returning results into an output register. The orders for what operation to perform is relayed from translated code, either automated or programmed in by a human, by an external control unit. This control unit also 'feeds' and tidies up after the ALU by moving processed data from memory to it, and back.