He also paid tribute to PC Keith Palmer, who died after being stabbed on Wednesday afternoon.
Speaking on Sky News, Khan said: "The important thing is we mustn't cower, we mustn't allow the terrorists to succeed and to destroy our way of life, or to divide communities.
"We have got to show them that we are resilient, that democracy, justice, civil liberties, our way of life, tolerance, are values they hate.
"They hate the fact that here in London, Christians, Jews, Muslims, Sikhs, Buddhists, Hindus, those who are members of organised faith and those who aren't, don't simply tolerate each other, we respect each other, we celebrate each other, we embrace each other.
"They hate the fact that we've got elections and elected politicians that go about their job making laws, they hate the fact that we've got a free press, they hate the fact that we're such a vibrant, successful city.
"It's not the first time terrorists have attacked our city, it's not the first time, I'm afraid, Londoners have tragically lost their lives and been seriously injured, it's not the first time people have tried to divide our communities and destroy our way of life.
"But I tell you what, in the past we have risen, we've shown our resilience, we've returned to work and returned to normality. We have done it in the past and we will do it again now."
He praised the Metropolitan Police's "expertise and excellence" for thwarting 13 terror attacks over the last few years: "It's because they practice regularly - they're the best in the world - that they responded so quickly yesterday."
He added he was impressed by the bravery, courage and innovation of the police and security services and how they have adapted and evolved to find new ways to make citizens safe.
