There is a regular automated remote cache deletion to keep your Mastodon server storage space usage (both database and media) as low as possible.
Currently, that is applied to remote media attachments, preview cards and avatars/headers. This may change as Mastodon introduces more hosting sustainability tools and new features.
Please read for details.