An exhaust manifold is normally the most important part of the exhaust to replace on a normally aspirated engine.
Your exhaust manifold is bolted directly to your engine's cylinder head and is responsible for the flow of exhaust gases away from the engine through the exhaust system. When seeking to increase power output, fitting an upgraded exhaust manifold should be considered. If you are increasing boost, fuel pressure or any other input to the engine, the output (exhaust fumes) need to be removed as efficiently as possible. A better designed manifold will increase scavenging effect which will, in turn, increase the flowrate of air into the pistons resulting in more power and torque.
On popular cars you may have a choice of manifold - A 4-1 exhaust manifold has the least restrictive design possible, enabling the biggest possible gains at high revs. Perfect for driving flat-out. A 4-2-1 manifold has an intermediate stage which improves low-end torque which can be better on road.
Either on its own or in combination with an exhaust system, a manifold is one of the biggest improvement you can make to the exhaust.