Rickie Lambert struck once again from the penalty spot for the Saints, as the England international striker hammered home a penalty kick in the 89th minute.
Lambert, 31, scored with his first touch on his international debut for England against Scotland on Wednesday and continued his scoring run.
Teenage left back Luke Shaw went on another surging run as Southampton knocked on the door for a winner against the Baggies.
Shaw was taken down by Mulumbu as he cut into the box and the referee pointed to the spot.
With the large following of Southampton fans gathered behind the goal, Lambert smashed home his spot kick past Ben Foster to send the traveling fans wild.
Can Southampton hold on against West Brom? Watch here.