Both leader and tippet can be and often are monofilament. Fly fishermen more often use fluorocarbon, though. For trout at least, line diameter and visibility are important. You don't want the trout to see it, and although fluoro has its flaws (that's a whole different discussion), it is stronger and less visible than mono for its diameter.
But back to the question. The leader (which is tied to the end of fly line) is usually tapered from thicker to thinner so your line will "turn over" and not plop on the water to scare fish (as Simms said). You can get tapered leader by building your own with different diameters of mono or fluoro. I usually do this just because it's easier and cheaper for me, but most folks (especially newbies) would prefer to just buy a leader that's already tapered. I bought mine for about 15 years before building my own.
As catman said, the tippet is just a section of line (mono or fluoro) you tie to the end of your leader because you don't want to be cutting back your leader every time you tie on a new fly. For trout... Usually, you will use a mono tippet if you are fishing dry flies on the surface because mono floats. For flies that are designed to be fished below the surface, you usually fish fluoro because it sinks.
For trout, the diameter of the tippet usually depends on the size of the fly you are using, not the size of the fish you expect to catch or the strength of the line. In fact, unlike conventional tackle, I rarely take into consideration the strength of the line and couldn't even guess at what pound test a certain size line is - you usually just don't think of it that way in trout fishing. One main reason is that your drag and especially your bending rod are what will absorb the brunt of the pressure, not the line. A good general rule is to divide the size of the fly's hook by 3 to get the size (diameter) of the tippet. A size 18 fly hook might be used with a size 6x leader (see - I divided by 3).
And on that note, just to add more interesting fly fishing trivia, most of the time, the size and strength of the fly rod usually have to do with the size of the fly rather than the size of the fish you want to catch. If you want to cast a big fly a long way, you better have a big rod to help the fly cut through the air. This is only a general rule as I use a decent size rod (not big) to fish for huge carp even though the flies are relatively small.
Again, back to the question and already answered by Simms, if you are fishing for bluegill or largemouth, you can probably get away with a short non-tapered monofilament because these species actually like it sometimes when you splash your fly in front of them. I would opt to still go with a tapered line anyway just because when a line "turns over" properly, you get more distance on your cast. You can still make your fly "plop" if you need to.
Lastly, I know there are a lot of folks that read this that are considering fly fishing. I threw out some stuff that makes it seem like fly fishing is complicated. Just like anything else, it takes practice. But I swear it's not too complicated. If I can do it, freaking ANYBODY can! You really need to add it to your fishing arsenal. It can be soooo much fun.
