Hi!
I’m no expert but I just tested this case. First, I installed Gpg4win 2.3.3 and created a key pair and imported a public key. Then I installed Gpg4win 4.1.0 and saw that all keys were still available. So it should be not necessary to backup the keys just for the upgrade. But, of course you always should have a backup of your keys 
What I noticed was that the default install directory was a different one. So if you use GnuPG on the command line you should edit your environment variables.
You don’t need to update Kleopatra separately.
I hope I could answer at least some of your questions. Maybe someone else has additional information about what you have to watch out for.
With regards,
Christoph