Yes, it's generally a good idea, even if it won't protect against most breaches.
There are also certain types of encryption at rest where it buys you absolutely nothing though, e.g. using AWS' builtin encryption at rest for S3. No one is going to break into the AWS datacenter and steal the data from the physical disks.
AWS decommissions hardware like anyone else. While I'm sure their processes include a secure-delete, no process is perfect and things are missed. It's worth setting the flag, especially since it's free.
There are also certain types of encryption at rest where it buys you absolutely nothing though, e.g. using AWS' builtin encryption at rest for S3. No one is going to break into the AWS datacenter and steal the data from the physical disks.