The first purchases most new divers want to make are fins, a snorkel, and a mask. The mask and the fins are necessary for diving, but while snorkeling is easy in the clear, warm, shallow waters of the Caribbean, local conditions do not favor it, as the bottom is seldom visible from the surface. None of these items should be very expensive.
2016 Update
Not much has changed in the 20 years since I first wrote this guide. These were glass, silicone, and plastic back then, and they still are. Modern stuff may be sleeker than the old models that are pictured here, but really not much has changed. I still have the first mask I ever bought, and still use it. That comes from taking good care of your gear.
More: The Basics ...