On (or just off) Fifth Avenue between the Empire State Building and Madison Square (where the jumpers were summer of 2010) and the home of the Museum of Sex, you will find two of New York City's many many houses of worship.
On the West side of Fifth is Marble Collegiate Church. This is the church where Norman Vincent Peale preached and Liza Minnelli got married the last time around. This is a vibrant and welcoming place. There is often a tour of the church after services on Sunday. Yes that is the Empire State Building in the background.
The church is related to one of my East Village favorite churches - Middle Collegiate. For the last couple of years, I have attended a second seder - Maundy Thursday interfaith service here and the choir is really outstanding.
Across the street, just East of Fifth Avenue is another amazing church with history. The Little Church around the Corner. Since 1870, this church has had a relationship with New York's theatre community. It is the home of the Episcopal Actors' Guild.
Continue down Fifth Avenue and stop in at the shop and bar at the Museum of Sex or visit the new kid on the Avenue - the Italian Market Eataly. I stopped in on my way to a tour of the Masonic Temple (another post perhaps) and the hot chocolate at the coffee stand as you enter from the 5th Avenue side is to die for but don't buy a medium, a small is more than enough.
There's a great book about the religious buildings and congregations of Manhattan called "From Abyssinian to Zion". It provides wonderful maps. You can plan your own walking tour of Spiritual Paths in Manhattan.