This paper focuses on the multi-agent safe control problem for stochastic systems. We propose a probabilistic certificate for safety and performance specifications and use it to construct a distributed control algorithm. The certificate integrates the reachability- and invariance-based (barrier function-based) approaches via a new notion of probabilistic invariance. The proposed method has three features. First, it can guarantee a long-term probability of safety and performance satisfaction using myopic evaluation. Second, each agent can collaboratively ensure system-wide specifications even if each does not have sufficient information to evaluate the specifications. Third, it can be easily integrated with learning-based controllers to provide safety and performance guarantee without requiring the convergence of the learning algorithm. The effectiveness of the proposed method is tested using numerical experiments.