Computer Graphics (CG) animations of natural phenomena are currently widely used for movies and in video games. Granular materials occur widely in nature, and therefore it is necessary that CG animations represent ground surfaces composed of a granular material as well as model deformations when the granular material comes into contact with other physical rigid objects (called solid objects). In this paper, we propose a deformation algorithm for ground surfaces composed of granular material. The deformation algorithm is divided into three steps: (1) detection of the collision between a solid object and the ground surface, (2) displacement of the granular material and (3) erosion of the material at steep slopes. The proposed algorithm can handle solid objects of various shapes, including concave polyhedra by additionally using a layered data structure called the Height Span Map. Furthermore, a texture sliding technique is presented to render the motion of granular materials. In addition, our implementation of the deformation algorithm can be used at interactive frame rates.