Pyrite and Gold are formed under similar conditions, so they can often be found in the same location, I have a specimen piece of quartz at my mum's place containing both gold and pyrite. One thing is for sure though seeing them together you should immediately be able to tell them apart. If your eyes deceive you pyrite is hard and brittle and will leave a brown/black streak on a scratch test, a ceramic tile will do the job. Even taking a piece and smearing it in your pan with your fingernail will hint to it's real identity.
Is the creek/river you are working a known gold bearing one? Let us know your general location and I'm sure someone can point you in the right direction (was a bit slow to press send it seems! There certainly is gold around Dargo). Sounds like it is Mica in your pan though, 2.8-3.1 specific gravity, it will happily swirl around in the water. Pyrite a little heavier, 4.95-5.1 and Gold leading the pack at 19.32 specific gravity.
Good luck out there, once you find yourself in a gold bearing creek you should have little trouble finding some to toss in a vial, the real test is learning how to maximise your take for the effort put in. If you look at Dr.Duck's latest post on panning at Tuena, you can certainly see that all you need is a pan to bring home quality gold.
Garrett deluxe pack, already putting yourself in a good position to find some yellow.