One factor by itself might be manageable. The danger is the combination. Lifting a heavy block (force) while bent and twisted (awkward posture), a hundred times a day (repetition, duration) is a different animal than lifting it once, close to your body, at waist height.
NIOSH's own risk-scoring approach adds exertion, posture, repetition, duration, and vibration together — because that's how the body experiences them. When you size up a task, don't ask 'is this one thing bad?' Ask 'how many of these am I doing at once, and for how long?'