Research into the perception of texture is fundamental to the understanding of biological visual systems and the development of general machine vision systems. However, even in the much more constrained area of inspection there are a wide range of tasks which require textures to be identified and categorised. This paper will use the exemplar of a hypothetical texture inspection task, where a textured surface will be imaged by a camera whose axis is normal to the texture plane. The textures present in the image will be classified as belonging to an a priori known class.

Perceived textures may be due to a projected illumination pattern, surface markings, the interaction of illumination with a rough surface or a combination of all three. This work is primarily concerned with the automated classification of textures belonging to the third group. While this type of texture is formed by the interaction of illumination and surface, most applications require the classification of surface type, regardless of illumination conditions. Consequently, work has been carried out on texture classification which is invariant to changes in the illuminant intensity [Thau90] and colour [Healey95]. However, the effect of variations in illuminant direction has been less frequently considered [Chantler94].

The direction of the illuminant with respect to the texture may defined by two polar co-ordinates: slant and tilt. Slant is the angle between the camera axis and the illuminant vector, in this work, it shall be held constant at 50° . Tilt refers to the polar angle of the illuminant on a plane normal to the camera axis. The aim of this work is to maintain consistently accurate classification of textures under changes in the illuminant tilt angle.

It has been shown that changes in illuminant tilt can induce classifier failure[Chantler95a]. For isotropic surfaces a change in illuminant tilt is effectively an instance of rotation, with the texture properties being preserved. However, for directional surfaces, the information content of the image will be altered by a change in the illuminant direction. While Chantler [Chantler95b] used a frequency domain technique, based on Kube's model [Kube89], to reduce the effect of tilt, non-linearities and artefacts in both the reflectance function and the imaging process present limitations to the practical application of this technique. In this section we propose a simulation-based technique which is defined in the spatial domain and circumvents many of the constraints of the frequency domain.

The simulation-based approach is designed to anticipate the feature space distributions by modelling the underlying physical and analytical processes of imaging and feature extraction (Figure 2.1). This technique forms a spatial model of the training surfaces in the recovery stage. The classification process proper, begins with secondary training, where the synthetic surface is rendered under the experimental conditions, and the resulting image forms the basis for training. If the model’s components, the reflectance function and the surface description, are sufficiently accurate we should be able to obtain classification rates approaching those of the ‘best case’ classification, i.e. classification based on training surfaces illuminated at the appropriate tilt angle.

The single image SFS problem is ill-posed and requires additional constraints, the smoothness constraint being almost universal. This clearly prevents the vast majority of single image algorithms being applied to the rough surfaces considered in this paper. Two algorithms which do not use this constraint are Knill's adaptive technique [Knill90] and Pentland's frequency-based technique [Pentland90]. Knill's technique uses prior knowledge of the surface and is therefore unsuitable for classification tasks, while Pentland's scheme is subject to the same problems of non-linearity as Chantler's tilt compensation scheme. The authors note that, if a single image scheme did form a consistent estimate of the surface, it would be possible to classify on the basis of the surface, rather than on the image, removing the need to resort to a model-based technique.

The technique of photometric stereo allows the formation of a surface description from several images of the same surface imaged under various illumination directions. The technique is entirely localised and does not require a smoothness constraint. It therefore seems ideally suited to our purposes and is intuitively satisfying since our problem is itself caused by an illuminant of variable direction. In fact the concept of recovering a surface for subsequent image prediction is not original, Russell used photometric techniques to acquire depth maps which could then be synthetically illuminated to simulate aerial images [Russell91], though the authors are unaware of any application of this approach to texture classification.

Since it was first advocated by Woodham [Woodham80], photometric stereo has been further developed to deal with specularities [Coleman82], accommodate more complex reflectance maps [Tagare91], use fewer images [Onn90], recover shape and the reflectance function simultaneously [Tagare90], estimate the degree of subpixel roughness [Solomon96] and has been implemented in various fashions [Woodham94], [Iwahori95]. In this paper, where the accuracy of surface recovery is less important than that of image prediction, and where the surfaces are near-Lambertian, we adopt the following simple and sub-optimal scheme.

Consider a facet with derivatives p,q, illuminated from a tilt of t and a slant of s . If we assume the reflectance function to be Lambertian, the perceived intensity will be given by

Now, dividing the t =0° and 90° facet images by (2) we can remove any albedo variation, and the non-linear denominator to form expressions for the surface derivatives.

The authors accept that the surface recovery performance, and generality of this technique, is inferior to any modern photometric technique. However, in the context of this application it offers an adequate prediction of the correct training image at the appropriate tilt angle for a minimum of implementation complexity, memory requirements and computational effort.

The simulation-based scheme seeks to reduce classification error by simulating the input image at a given tilt value. It is therefore natural that the system's classification accuracy should be compared with the accuracy of a classifier which has been retrained at each orientation on the actual data, and which consequently forms the ‘best case’ case classification for a given texture set and feature set.

Figure .5 Comparison of Model Based and Best Case Classifiers for Anaglypta Montage

· Shadowing. Shadowing in the recovery images presents a problem for photometric techniques in general, however, given the formulaic nature of the approach used here it is a particular problem with this scheme. We note that several researchers such as [Solomon96] use schemes which use more images to reduce the impact of shadowing on surface estimation. Shadowing also presents a problem at the rendering stage. Since the surface description is expressed in terms of surface derivatives, rather than absolute height, it is not possible to incorporate shadowing into the rendering algorithm.

· Reflectance assumptions. In the experimental work carried out here, all surfaces were of approximately Lambertian reflectance and uniform albedo. Suspension of the former condition will require more complex surface recovery and rendering algorithms, but will not of itself affect the validity of the simulation/training concept. The uniformity condition on the other hand is more serious: although the albedo variation may be isolated from the training data, it will be difficult to generalise the demodulation process to the classification data.

· Rotation-Invariance. This is probably the most fundamental and intractable limitation of the model-based scheme. The model-based system as discussed in this chapter is, almost by definition, unable to deal with the rotation of directional surfaces. However, simulation-based techniques may still have a significant role as a means of economically generating large volumes training data for the design of classifiers which are invariant both to the rotation of textured surfaces and the associated variation in relative illuminant tilt.

· Slant-variation. This paper has only considered the performance of the scheme for variations in illuminant tilt with the slant angle constant for recovery and classification images. It could therefore be argued that the scheme can only be described as an interpolation scheme since the generalisation properties of the scheme have not been assessed. If the recovery and rendering algorithms are sufficiently accurate, the scheme should be able to deal with slant variation and will form an extremely powerful support tool for texture analysis. The effect of slant variation will also have a significant effect on shape from texture techniques which are applied to rough surfaces, however, it is worth noting that the increased degree of shadowing associated with this task may require recovery of the actual height map, rather than just the surface derivatives. The required accuracy for height recovery is beyond the capabilities of the recovery technique discussed here.

The second conclusion is that a simulation-based system forms an effective approach to maintaining consistently good classification regardless of illuminant tilt. The misclassification rate is consistent and in most cases approaches the ‘best case’ level. The authors conclude that the concept of simulating training data does represent a novel and powerful tool for the development of systems robust to illuminant changes for relatively little extra effort at the training stage.