Put simply the rule of thirds is about subject placement in your image. Most amateur photographers put the subject of their photos smack bang in the middle. Sometimes this works, but often it would be better in a different position.
The rule of thirds breaks up your image into nine segments. Imagine a photo being cut up by two equally spaced horizontal lines, and two equally spaced vertical lines, creating nine equal sized squares. There are also four points where the lines cross over one another.
Say for example you are taking a photo of a lonely tree in a field, with the horizon far off in the distance. Almost all amateur photographers would place that tree in the middle of their image. Instead what we're going to do is use the lines from the rule of thirds to compose the shot.
Place the tree on one of the intersection points of the horizontal and vertical lines. For a shot like this you would most likely place the tree on one of the bottom two intersections, so that you have a big sky behind your tree. This also means you want to try and get the horizon line to sit on the lower horizontal line.
This placement of the tree and horizon in this example will add much more interest and 'tension' to use a photographic term, to the image. It will look more professional than if you had placed the tree and horizon in the middle of the image.
As with all rules of photography, they're not really rules, more like guidelines. So don't think you must use the rule of thirds all the time. Often it produces a nicely composed, professional looking picture. But sometimes you'll want to 'break' the rule to get a better composed shot.
When to use the rule of thirds is something you will learn with practice. I suggest that you try it for almost every picture you take, as most of the time it works. If it didn't work, just delete the image and try again. That's the beauty of digital cameras, you can keep photographing until you get it right!