A method is presented for recovering the 3D shape of objects from a sequence of stereo images, which integrates shape from stereo and deformable model . Scene point correspondence required for deriving the depth of the scene point is determined using Immune algorithm .The object shape is described by a deformable surface model, which successively fuses surface patches computed from each pair of stereo images through deformation. Aligned with the input surface path according to the object or camera motion, the model is deformed by attracting its corresponding surface towards the input surface patch. Object motion between successive pairs of stereo images is estimated by tracking a set of image features. Experimental results are also included to show the feasibility of our new method.