What you are talking about is primary imbalance, which creates main bearing loads - as long as all of the components are equal weight, those won't be a problem.
The crank damper is to damp out "secondary" resonances, which are "twisting" vibrations in the crank. What happens is that every two revolutions, each cylinder fires. This creates a twist as it fights against the compressing cylinder and the flywheel mass. it springs back quickly, but when the crank RPM matches up with these vibrations, the crank starts wringing back and forth. The damper adds an extra wobbling mass that keeps the resonance from amplifying. Without the damper, you can eventually cause the crank, or clutch/driveline parts to fail.
Quoting jepherz:
^^^ Can you explain that? I've never understood why you can't just static balance every crank component (flywheel, etc.) and put them together to create a balanced unit.