Shitsukan(質感)-“RECOGNITION OF MATERIAL CATEGORY AND ITS PROPERTIES”.

When we see objects in our daily life, we routinely and quickly recognize their material composition and properties; whether, for example, the cup in front of us is made of metal or ceramic, the sofa is upholstered with leather or fabric, or a shirt is wet or dry. In Japanese concept of “shitsukan” this perception is known as “RECOGNITION OF MATERIAL CATEGORY AND ITS PROPERTIES”. In this blog, I am going to discuss about about the neural mechanisms underlying material’s category and its properties perception, or shitsukan perception with special focus on “MATERIAL CATEGORY AND ITS PROPERTIES”.

What kind of information is involved in material perception?

The interaction of light with objects having specific surface meso-structures and optical properties produces various structures within the retinal images of objects, and these structures are the source of generation of a variety of surface qualities. In addition to this, other sensory modalities are also involved in material perception. For instance, when we see a sweater made of fine wool, we can perceive that it will be soft and warm, or we can sense that a metal cup will be cold and hard to the touch. Therefore, the information obtained through different sensory modalities is closely linked with each other in material perception. In this blog I will review cross-modal aspects of material perception including following.

  1. Neural processing of surface qualities, particularly the glossiness that can be named as “GLOSSINESS PERCEPTION”.
  2. Neural processing of surface meso-structures, that is another important property and is closely linked to material perception named as “NATURAL TEXTURE PERCEPTION”.
  3. Neural processes involved in categorization of materials and recognition of various material properties, such as roughness and hardness, based on information about surface glossiness, natural textures, and so on. It can be called as “RECOGNITION OF MATERIAL CATEGORY AND ITS PROPERTIES”.

In my previous Blogs I have already discussed 1 and 2. Therefore in this Blog I will elaborate “RECOGNITION OF MATERIAL CATEGORY AND ITS PROPERTIES”.

3. Recognition of material category and its properties

Material categorization

When we see objects in our daily life, we quickly recognize their material composition and properties; whether, for example, the cup in front of us is made of metal or ceramic, the sofa is upholstered with leather or fabric, or a shirt is wet or dry. According to research studies, human observers will correctly classify material categories with accuracy of 80% merely by viewing object images for a duration of 40 minutes, and with accuracy more than 90% when images are viewed for 120 minutes (tested with 10 material categories, as shown in the Figure 1. below).

Figure 1. Different materials in our daily life (Goda et al. 2014)

As described above, there are neurons along the ventral visual pathway that efficiently extract information about optical properties (e.g., gloss) and natural texture of surface that are diagnostic of materials. The visual system likely uses such information to estimate the material category and its properties. But in comparison to investigation into the neural mechanisms underlying object and scene category recognition, research on the mechanisms underlying material category recognition began only recently. It has been reported that material categories such as metal, stone, wood, and fabric can be distinguished (classified into the correct category) based on the pattern of fMRI (Functional magnetic resonance imaging) activity in the human visual cortex elicited by just viewing material images. This approach is called pattern classification analysis or decoding analysis. This is now widely used to determine whether a given cortical region carries information about categories. More recently, it has also been reported that viewed material categories can be decoded statistically significantly from the activities in early, middle, and higher order visual areas. This suggests information about material category is carried from early to higher visual areas rather than processed in specific areas. These human fMRI studies have also shown that early areas tend to show higher decoding performances than higher areas. In another study it is shown that wood and stone material categories could be decoded based on the patterns of ERPs (Event-Related Potential: is the measured brain response that is the direct result of a specific sensory, cognitive, or motor event) as early as 100 ms after stimulus onset. This temporal characteristic supports involvement of early visual areas in the material categorization. It is further suggested that decoding of object categories in object-selective higher areas also tends to be lower than in earlier visual areas probably because the spatial organization of object information in higher areas is not optimal for decoding with fMRI. It is therefore important to assess a material category selectivity using complementary methods before drawing a conclusion about it. In a later study adaptation paradigm, which is another method to examine material category selectivity was used, and a reliable material category adaptation not in the early visual areas but in the PHG (Parahippocampal Gyri ), a medial part of the ventral visual cortex was reported. 

Figure 2. Regions showing texture- and material-related activities in the human ventral visual cortex (Goda et al. 2014)

It has also been reported that activity in the ventral visual cortex around the CoS (blue circle) as shown in above Figure 2, is higher during discrimination of material category (wood vs rock) than during discrimination of a 3D shape using the same stimuli, while activities in earlier areas do not differ. Therefore, the results obtained using the approaches complementary to the decoding suggest that, although the decoding performance is not high, the higher visual areas have selectivity for the material category. It is likely that all areas along the ventral pathway play important, but different, roles during material categorization.

Recognition of material properties

Humans feel various material properties, including optical and various physical properties such as gloss, translucency, smoothness, coldness, hardness, and weight, just by looking at an object. These feelings or impressions about a material evoked by an object’s appearance called “perceptual material properties”. These can be measured with ratings on a Likert-type scale, which is a typically bipolar adjective scale such as hard-soft . This method has been employed to evaluate the perceptual material properties derived from the visual appearance of images of natural textures and materials, synthesized images and real objects, as well as those derived from touch and sound. These studies consistently show that various perceptual material properties tend to cluster for each material category; that is, exemplars of the same category generally give similar impressions about the properties. Such within-category clustering, as well as similarities between categories, cannot be attributed simply to low-level image features.

What brain region represents perceptual material properties? The studies on this question revealed that the neural similarities between material categories obtained from early visual areas (V1/V2) are well correlated with similarities in low-level image features, as was generally expected, whereas the neural similarity obtained from a higher order area in the human ventral visual cortex (a region encompassing the FG and CoS but not LO; Figure 3 below, upper left) reliably correlates with similarity of perceptual material properties evaluated using a 12 bipolar adjective scale. The representation in mid-level areas V3/V4 was between those in V1/V2 and FG/CoS (Figure 3 below). This suggests image-based representation of material categories in V1/V2 is gradually transformed through V3/V4 into a representation more reflective of the perceptual material properties in the higher area.

Fig. 3. Material representation in the human and macaque visual areas assessed using fMRI (Goda et al. 2014)

Visual versus nonvisual material properties

Some material properties, such as microscale roughness, hardness, coldness, and weight, are nonvisual and cannot be directly sensed visually. Nevertheless, interestingly, humans can accurately estimate such nonvisual properties from the visual appearance of materials, in a way that correlates with those haptically estimated through touching them. It is thought that recognition of such non-visual properties is based on the association between certain visual features and nonvisual properties learned through experience (see the “Crossmodal association through experience” section below). This implies that processing non-visual properties involves neural mechanisms different from those involved in processing visual properties, such as access to stored knowledge of learned associations. Consistent with this view, by analyzing the temporal characteristics of visual discrimination of material categories, studies suggested that it takes longer to process non-visual material properties than visual ones.

In that context, it is noteworthy that the ventral visual cortex in humans exhibits activity reflecting some non-visual material properties. Studies showed that the neural similarity in activities around the CoS and FG described above correlates with similarities in both the visual and non visual aspects of perceptual material properties. It has also been demonstrated that activity around the CoS increases when human observers judge the hardness of visually presented objects compared with 3D shape judgment (Figure 2. blue circles), and  showed that the weight of objects with natural textures can be decoded from activities around the CoS. Furthermore, as will be described later, the region around the CoS and even earlier visual areas exhibit activity during haptic texture judgment. These findings highlight the multi-modal nature of the ventral visual cortex for processing material property information. A study focused on three material properties – roughness, texturedness, and hardness – and asked whether they could be decoded from activity in the visual cortex while observers rated them in material images. The results showed that roughness and texturedness can be decoded from activity in the early visual areas (V1–V3), and their decoding is likely based to a considerable extent on low-level image features and PS statistics. This is consistent with several psycho-physical studies suggesting that roughness judgment is based on simple image features. On the other hand, hardness, a non visual property, could be decoded only from activity in the LG, the medial part of the ventral cortex (Figure 2. blue square in the right hemisphere). These findings, together with those summarized above, indicate representation of visual and non visual material properties emerge in early and higher ventral visual areas, respectively. This view is consistent with the finding that regions distributed from early to higher visual areas are involved in the material categorization, as described in the previous section.

Conclusion

This blog, reviewed what is currently known about the neural mechanisms underlying material perception, or shitsukan perception, which plays important roles in our perception of material properties, recognition of the conditions of objects, decision making about preference/avoidance of objects, and motor control of our actions toward objects.

Sources:

  1. Komatsu H, Goda N. Neural mechanisms of material perception: Quest on Shitsukan. Neuroscience. 2018 Nov 10;392:329-47.
  2. https://www.mub.eps.manchester.ac.uk/science-engineering/2018/07/12/shitsukan-how-we-make-sense-of-our-material-world/

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s