Landscape stores are a great source for rocks. Back in Cali, I had an old pick-up (73 GMC Stepside - I miss her) and would go to the landscape/gravel place and fill the bed with river rocks for like $10-15. Slate, Mexican Pot Rock (aka Texas holey rock), limestone, etc. were like 30-40 cents per pound, less in bulk.
With rock that's not from a waterway, I've always just rined in a bucket of tap water to get the dirt and small pieces off. Bleach is a good idea for water from waterways.
Regular white vinegar will test for calcium carbonate in rocks. Limestone will fizz (as will baking soda)...which means that it will buffer the water (good for some cichlids).
A fun science trick is to add some baking soda and vinegar together in a glass and cover the top with a platic bag. The CO2 produced will put out a match!
I use strictly flowerpots and PVC in my tanks. Much easier to keep clean
