What motion causes the rattling? Are the sides of the blade rattling against the kydex (lateral movement), or is it rattling from spine to edge, or is it rattling longitude (tip to ricasso)?
If the problem is lateral rattle, use a heat-gun to warm up the sides of the sheath (or just one side) until semi-pliable, then insert the blade (which should have been withdrawn before heating) and use a spring-clamp (with foam feet) to press the sides of the sheath against the blade until the kydex cools firm. If you are worried that this fit will be too tight, wrap the blade in a layer of masking-tape before inserting into the sheath.
For other problems, proceed similarly, keeping in mind that kydex is thermo-form, i.e. malleable when properly warmed. You can mold it to your desire even by hand, but wear gloves. Avoid heating too much around rivets/eyelets. Good luck!