John Gruber at Daring Fireball has the definitive post on why you shouldn't force quit apps on your iOS device (double-pressing the home button and swiping them away):
The single biggest misconception about iOS is that it’s good digital hygiene to force quit apps that you aren’t using. The idea is that apps in the background are locking up unnecessary RAM and consuming unnecessary CPU cycles, thus hurting performance and wasting battery life.
That’s not how iOS works. …[U]nfreezing a frozen app takes up way less CPU (and energy) than relaunching an app that had been force quit. Not only does force quitting your apps not help, it actually hurts. Your battery life will be worse and it will take much longer to switch apps if you force quit apps in the background.
The only reason you should force quit an app is if it is misbehaving, such as not responding. All of this reasoning and advice applies equally to Android, which operates in a similar way.
If you force quit apps to keep the app-switcher clean, know that you are hurting your phone's battery life and your experience. At the time of writing there is no alternative way to keep the app switcher clean.