Rapid and accurate detection of collision between virtual objects is crucial for many virtual reality based applications. In order to ensure a high-level of accuracy and to meet the real-time requirement, a fast collision detection algorithm between soft bodies is proposed. The developed algorithm is a combination of stochastic methods and particle swarm optimization with adaptive Cauchy mutation. The hierarchical bounding box method is used for a rough detection in order to filter out obvious disjoint space, and the problem is converted to a nonlinear optimization problem based on the distance of points. Then particle swarm optimization with adaptive Cauchy mutation is used to find the optimal solution. When it is updated iteratively, keeping some particles experience value and variation of other particles avoids particles trapped in local optimum, which further accelerates the speed of collision detection. The algorithm is implemented and evaluated through experiments and the results confirm the advantages of the developed algorithm.