This paper introduces a completely new moving object detection algorithm from a static background scene that uses colour images to contain shadows. Tracking of Object is based on analysis and detection of object motion, removal of background and shadow. Initially, an organisation is used and thought of as background data. The foreground and background object data is recognized based on reference frame of model when a brand new object enters the frame. The object shadow of background details is combined with the foreground object and makes the tracking of object is somewhat difficult. Structural based activities are used within the strategy to identify and remove the shadow. With the suggested algorithm, video sequences are captured and tested. Experimental outcomes are shown, showing the performance of the system.