Once you get there, suit up at your car and walk down the small stretch of beach till you get near the jetty. Enter the water at your leisure. Once in the water, you can swim north along the jetty to the tip. You may see fish, lobster, crabs and the sort. There are large openings in the rocks that form the jetty. Large enough in fact that you can almost swim into one. I don't advise doing this because getting stuck might be a frightening experience.
There are two actual Ponquoque bridges, the new one and the old one. You crossed the new bridge to get to the former Foster Road. The old bridge is where you will probably dive, although nothing is stopping you from diving the new bridge. Don't get caught in the channel between the two bridges, however, because it is considered a channel and it is illegal to dive in a channel in the town of Southampton. With that in mind, there is usually good parking at the bridge, but you will have to do some walking in order to get into the water.
The Shinnecock Inlet is located in the Hampton Bays on Long Island. The inlet is one of the only major inlets on the south shore and sees a great deal of boat traffic during the summer months. With that in mind, this inlet also sees a great deal of diver traffic as well.
Now, most people would look at the heading on the page and think that I was crazy for diving near a nuclear power plant. Just to calm your fears, this power plant was never brought online, so there is no chance of getting radiated at this location (barring a nuclear war while you're diving). With that said, Shoreham can be a very interesting dive. Located on the north shore of Long Island (I don't know the name of the town, look on a map), it is a shallow dive with an average depth of around 16 ft. Visibility tends to be anywhere from 3 to 10 ft.