I have some notes for anyone that wants to open up their RCX. This task is non-trivial, as the battery holder is "clipped" to the PCB.
You can bend the clips back to release the battery holder. However, this can damage the PCB and the clips. Or,...
Find yourself a very small flathead screwdriver, a jeweller's screw- driver is preferable for this task. Carefully lift one side of the battery holder and look for two "slots". Now gently insert the screwdriver into these slots and "pop" the PCB off the clips holding it to the front cover. You probably want to repeat this process on the other side of the battery holder. The clips holding the PCB to the front cover are actually the connectors between the PCB and the "port pads". Hint: keep the front cover facing down; otherwise, when you remove the PCB the buttons and "port pads" will fall out.
Now you have the battery holder and PCB--notice the power clips holding the battery holder to the PCB. If you want to remove the PCB, you can do this by gently rotating the PCB counterclockwise so as to slide it out from underneath the power clips.
You can reassemble the RCX by following this process in reverse. You might want to bend the power clips down a little with a small pair of pliers before reattaching the PCB to the battery holder. This ensures that the power clips contact the pads on the PCB.