Sexual activity or no sexual activity is not going to have any noticeable effect on the testicle size of a healthy man... no more than reading or not reading will affect the size of your eyeballs.

If you notice a change in your testicle size, larger or smaller, you should go to your doctor IMMEDIATELY.