Providing real-time ergonomic feedback to workers is essential towards preventing extreme postures that can cause work-related musculoskeletal disorders (WMSDs). In this work, we propose a novel methodology to assess in real-time the ergonomic risk of any work-related task using the REBA framework. The proposed methodology gets as input video sequences, extracts 3D skeletal features and processes these features to compute REBA scores. At the core of the methodology lies a novel multi-stream deep network that can process 3D skeletal joints regardless of the method used to acquire them and provides not only a total REBA score, but also partial REBA scores that correspond to individual body parts, thus assisting workers towards a better understanding of which body parts face the biggest strain during a task. Experimental results on two publicly available datasets demonstrate the generalization ability and accuracy of the proposed ergonomic risk assessment methodology.