Memory is trainable. Simply put, if you endeavor and take action to remember every little thing, you will manage to do so. A skill, the same as all else. How does one begin? Simple. Remember. At the end of the day, meditate on your day, from start to finish, everything you did, in order, what did you eat, small things that were noticeable. This practice is to see it, a picture is worth a thousand words, visualize your day back to yourself, turning it into words is how we lose the memory. Language is a learned arbitrary rulebook for communication, the brain remembers with the senses, thus remember the image of what you read, not the words themselves, you can always reread it in your mind.
So take the 20-30 minutes before sleeping, and go back through the day without judgement, remember it, and visualize it. After some months, look to remembering the day back 2 days before hand, then 3, 4, a week, to a month, 2 months, and so on and so forth until the next year. Some skills require greater upfront investment of time before the fruits are found, this is as such. It is worth it, and the time will pass whether you spend it learning or not.
Give it a shot. Meditate.