It kind of depends on the child as to how much they can learn and retain, even if they are severely mentally retarded. Some of these kids can learn to swim and do other physical activities, while others just cannot.
My sister volunteered with a day placement center for mentally retarded young adults and she said that some of them could handle sorting tasks and so forth, while others were content to sit in the chairs. Not that it mattered. They were around other people and were having more enriched lives because of it. So it was a good activity for them, even if they didn't do anything for the couple of hours they were at the center.