New to Java

I just joined a group doing Android development so I could learn Android and Java, but right now I'm a total noob.

We were told that after installing the JDK we had to change our PC's PATH environment variable, to put the JDK ahead of System32 because supposedly System32 has a JRE that conflicts.

I didn't see a JRE in my System32, but I saw java.exe, which I'm told is a JVM. and when I looked around my PC I saw LOTS of JRE's, with java.exe's all over my PC! It seems like many apps that use Java have their own JRE's, often the same version! Why? Why isn't there just one JRE, or at least just one copy of each version?

Also, are JRE's backwards compatible? If an old Java app was written for an old JRE do we need to keep the old one around or should it work with the current JRE?

Could someone please explain to me what the canonically-correct java runtime architecture on a PC is supposed to be? And do I really need to change my PATH environment variable? (I don't like to put things in front od my system32 if I can help it)

Thanks in advance!