Halloween began as an ancient Celtic ceremony called Sawhain. It was the Celts' major celebration, a festival of the dead but also the beginning of winter.
It was time for them to harvest their crops and store them, and round up the cattle and sheep and move them in closer. They believed at this time the ghosts of the dead mingled with the living. I think this is archetypal (in our bones) because things are dying this time of year. Plants are dying, leaves fall off the trees ...
Others think it was the harvest, why keep it to yourself, invite all your ancestors and dearly departed to join in.
Sawhain was exactly 6 weeks after the Autumnal Equinox and 6 weeks before the Winter Solstice. In other words, right in between the two.
Now here's how one tradition developed. It was New Year's, a new beginning. Cold, too, so they'd have a fire. Someone from each household would be sent to collect embers from the community fire for the fire in their own home. The easiest way to carry home the coals was in a hollowed out turnip, sometimes with a carved face on it to scare away evil spirits.
Why some of the traditions?
This is from bigchill.net:
Apples ... Find out the identity of a future spouse by cutting an apple in half around the middle and eating half the apple in front of a mirror at midnight on Halloween. Your intended's face will appear.
Cakes and sweets are also essential. Ghosts are especially benevolent towards children and everyone knows that the best way to lure children to the table is with sweet stuff. Combine the cakes and the apples with apple tarts or warming apple crumble or caramel apples.
Anyway, the Celts were celbrating Sawhain and then the Christian missionaries arrived to convert them to Christianity. In one of the sanest edicts I've read, Pope Gregory I issued an edict that it was to be done this way: the missionaries should not obliterate the native religion, but rather use it. If they found the Celts worshipped a tree, they should not cut it down, but rather consecrate it to Christ, and let the worship continue. Christian holidays were also set to coincide with local holy days. All Saints Day was assigned November 1st and meant to substitute for Sawhain, but still it continued. Jack Santino says, “The powerful symbolism of the traveling dead was too strong, and perhaps too basic to the human psyche, to be satisfied with the new, more abstract Catholic feast honoring saints.”
The Church tried again in the 8th century, naming November 2nd as All Soul’s Day, but Sawhain continued, and with time, All Hallows Eve (the night before) became Hallowe’en. People began to dress in costume to seek gifts of food and drink, which had originally been set out to appease spirits, and to masquerade as spirits, and our traditions of Halloween evolved.
There’s a lesson in multiculturalism here; a way to change culture and blend beliefs and traditions. “Worship” was the constant in the equation, and dealing with dark spirits, which they kept and modified. The new was accommodated without changing the old too much. Remember this when you’re changing a household or office custom. It's an emotionally intelligent way to do things.