The document has been found in the east of the Israeli capital and appears to threaten “revenge” on Christians.
It is signed by a group calling itself “ISIS [Islamic State] Palestine”, according to local press.
“Vengeance” will be brought upon followers of Christ, the leaflet said.
Christian journalist Paul Calvert is in Israel and told Premier there was “a lot of fear concerning IS” amongst the Christian community.
Recent figures suggest there are 12,000 Christian residents in Jerusalem, and many more Christian tourists every year.
It’s thought a large number of Palestinians have left the area to go and fight for Islamic State in Iraq and Syria, although there is no way to verify this.
IS has made vocal attacks on the Christian faith in the past, with one its magazines even showing the group’s black flag flying above the Vatican.
Aryah King, a Jerusalem City Councillor, said Islamic fundamentalism was “waking up” and becoming “like a trend”.
“We’ve seen more Muslims, not just in Jerusalem, but in Northern Israel, who have gone to join IS,” he said.
He warned that IS was attempting to create a presence in Israel.
In December a militant from Islamic State said the group would not stop until Jerusalem had fallen: “Convert to Islam or pay the ultimate price when your armies are expelled from Muhammad’s peninsula, from Jerusalem, and all Muslim lands.”
The jihadi added: “The Jews and the Crusaders [Christians] are scared and weak”.