We propose a new stereo matching algorithm which computes the depth and surface orientation simultaneously. While area-based stereo matching is an essential technique in the recovery of the dense 3D shape, conventionally it uses square windows based on the implicit assumption that intensity patterns surrounding corresponding points have no deformations between images from different views. In practice, however, the local surface orientations deform the intensity patterns and such deformations often give rise to poor estimation of the 3D shape. To solve this problem, we formulate a new algorithm that allows a matching window to locally deform according to the surface orientation, which we propose to compute directly from intensity gradients within the window. Through experiments we demonstrate that our algorithm indeed realizes more precise recovery of the 3D shape than do conventional ones while being applicable to various images.
