Real-time encoding of live reconstructed mesh sequences for 3D Tele-Immersion

Abstract

3D tele-immersion systems based on live reconstructed geometry, 3D-based videoconferencing, require low-delay encoding and small bandwidth usage. Existing triangle mesh compression methods are not that useful, as they are generally optimized for "downloadable" 3D objects, with high encoding delays. In this paper we experiment with a real-time reconstruction system that outputs a sequence of triangular meshes. Explorative analysis on the mesh data shows that regularities happen in the connectivity and geometric data. Based on this, we develop a module that achieves a compact representation for each reconstructed mesh. Comparison with the SC3DMC standard on the live reconstructed mesh sequences reveals an encoding speedup of over 20 times at comparable rate/distortion levels. By exploiting the properties of the capture and reconstruction process, we provide an optimized encoding mechanism that enables real-time encoding. This paper discusses some potential directions to better handle the large byte size of live reconstructed 3D triangle mesh sequences with stringent constraints on bandwidth, delay and computational complexity. View