After all that walking, a ferry ride across to the Asian side ($2.70 each way unless you stay on the ferry for the return trip) is a great way to relax with views up the Bosphorous and across the Sea of Mamara, and mosques dotting the skyline. The Blue Mosque, a central part of Muslim life in the city, is worth more than one visit and is free. It costs about $16 to enter the museum at the nearby Hagia Sophia.
As you would expect the queues are shorter first thing (it opens for visitors at 9am on most days). In the area around the mosque you will run into Baradan or one of his mates offering to shine your shoes ($6, but haggle). Baradan has five children in Kurdish Iran and spends three months there, then two months in Istanbul, and has been shining shoes in the city for 20 years.
The nearby Basilica Cistern is another bargain (about $6). The water storage system in Byzantine times, it's vast and cavernous but cool and peaceful on a hot day (until lots of tourists crowd in — as usual the morning is your best bet).
Back at Taksim Square you could investigate one of the Orthodox churches, the Hagia Triada (Holy Trinity), for instance, with its Rococo towers, and if you stay for a service you will be able to ponder the similarities between the chanting of the Orthodox and Muslim prayers.
Istanbul takes time to celebrate its past (the archaeological museum is a bargain at $6) and the Palatium cafe is a special treat, it's around the corner from the Four Seasons hotel in Sultanahmet, a few minutes' walk from the Blue Mosque. The owners have excavated ruins from Roman and Byzantine times and after you have enjoyed a coffee or apple tea you can climb down the stairs and find yourself in a domed space that would have been familiar to people 1500 years ago. You can view the ruins even without buying a coffee, but they are more than worth the price. And the pizzas are worth trying as well.