Moderate to high intensity aerobic and strength exercise training did not slow cognitive impairment, researchers found.
This sort of exercise may even worsen the condition, the authors suggested.
The study, published in the British Medical Journal (BMJ) found that among patients with mild to moderate dementia an exercise program did not halt the progress of their condition but it did make them physically fitter.
Indeed participants who had taken part in an exercise program had slightly worse scores in an Alzheimer's assessment when they were tested a year later.
Patients from memory clinics across 15 regions of England were invited to take part in the study.
Almost 500 people with dementia took part, with 329 embarking on a special exercise program and 165 receiving usual care.
The exercise program consisted of group sessions of 60 to 90 minutes in a gym twice-a-week for four months, plus home exercises for one additional hour each week with ongoing support.
The team of researchers, from the Universities of Oxford and Warwick as well as John Radcliffe Hospital and Coventry and Warwickshire Partnership Trust, found that patients' score on an Alzheimer's disease assessment had declined across both groups when they were followed up after a year.
Indeed patients who participated in the exercise program showed slightly worse scores.
"This indicates greater cognitive impairment in the exercise group, although the average difference is small and clinical relevance uncertain," the authors wrote.
The exercise group did show improvements in physical fitness.
The authors wrote: "People with mild to moderate dementia can engage and comply with moderate to high intensity aerobic and strengthening exercise and improve physical fitness.
"These benefits do not, however, translate into improvements in cognitive impairment, activities in daily living, behaviour, or health-related quality of life.
"The exercise program might possibly have worsened cognitive impairment."
