I have crushed coral or limestone in all of my African tanks, change water with straight tap, and have never lost a fish during a water change. The crushed coral provides "buffering capacity" to keep the water in a normal range, the pH can actually fluctuate more without it if nitrates or ammonia build up. Unless you're doing water changes of more than 50%, I doubt the overall pH changes much.