Why does The Giver have all of the memories instead of the community in The Giver?

Generations back, when the community went to Sameness, they decided that individuals in the community did not need individual memories of their communal past. While they do contain memories of joy, they also contain memories of pain, and these are burdensome to the community. Also, in having access to the memories, individuals are better able to make personal choices, and that would be counter-productive to a world where everyone needs to be the same.

In...

Generations back, when the community went to Sameness, they decided that individuals in the community did not need individual memories of their communal past. While they do contain memories of joy, they also contain memories of pain, and these are burdensome to the community. Also, in having access to the memories, individuals are better able to make personal choices, and that would be counter-productive to a world where everyone needs to be the same.

In Ch. 14 the Giver explains that Jonas and The Giver must hold all the memories because:

"It gives us wisdom. Without wisdom I could not fulfill my function of advising the Committee of Elders when they call upon me" (Ch. 14).

Even the Elders who make all of the decisions for the community do not have these communal memories. They depend on the Giver to advise them based on his memories of what has happened before. 

Ultimately, the Giver explains, "everyone would be burdened and pained" if the community held onto the memories and "they don't want that." So, one unlucky person, the Receiver of Memory, gets to be the memory bank for the entire community.