Author:

Keywords:

PSI_VISICS

Abstract:

3D reconstruction of dynamic scenes from monocular images poses various challenges. The unknown relative scale between the background and the foreground object is a subtle one, however it needs to be resolved properly for a realistic reconstruction. Our solutions are based on the fact that the foreground trajectory has components from the camera trajectory at wrong relative scales. This phenomenon is exploited in two ways: statistical approach computes the most independent object motion from the camera's and the geometric approach detects various regularities in the object motion which would hint the correct relative scale. Initially it was inherently assumed that the background object is labeled apriori. However, later it is shown that not only the correct relative scales result in the simplest motions but also the correct background labeling. Hence aforementioned relative scale resolution techniques are also applicable for that problem. However, it is also possible that the moving objects in a scene do not follow any motion simplicity constraint at all. One way to overcome this problem is by using another independently moving camera. In this setting, the relative scales for the foreground objects in both cameras are selected in a way which results in the identical foreground motion for both of the camera reference points. However, in order to achieve that not only the relative scales of the foreground objects but also the similarity transform between two reconstructions from both cameras and plus the time shift parameter need to be computed. The final result turns out to be a space-time-scale registration technique for video streams. Another basic expectation from a multi-body 3D reconstruction framework is proper segmentation of the moving objects. In contrast to many techniques which does feature tracking as a pre-processing step before segmentation, a new framework is presented where segmentation, tracking and reconstruction are done simultaneously. This requires online SfM, hence inevitably to be able to handle object appearance and merge-split operations. With this technique, 3D reconstruction of long and realistic sequences is achieved.