There are two actual Ponquoque bridges, the new one and the old one. You crossed the new bridge to get to the former Foster Road. The old bridge is where you will probably dive, although nothing is stopping you from diving the new bridge. Don't get caught in the channel between the two bridges, however, because it is considered a channel and it is illegal to dive in a channel in the town of Southampton. With that in mind, there is usually good parking at the bridge, but you will have to do some walking in order to get into the water.
It is not at all uncommon for small songbirds to be blown out to sea in a strong west wind ( or a north wind, for Long Island. ) Unable to make their way back to land, these tiny fliers eventually tire and drop into the water, or get picked off by seagulls. In desperation, they will alight on anything solid to rest. An anchored dive boat makes an easy target and may collect several hitchhikers over the course of a day. This little guy was so exhausted that he even perched on a finger for photographs. Generally, though, you should just leave them alone and not frighten them further, and they will fly away on their own once the boat gets back to land.