If there's a bit of slack in the sheath then cutting templates out of overhead transparencies (thin, clear plastic sheets in case they have another name) provide stiffness which you can wrap these in cling film. (I thought about using thin card but then I remembered that some paper releases acids that attack metals and the plastic is thinner.) For that matter, you could probably just wrap the templates in some sort of plastic tape which would be less tricky than cling film but thicker. This both stops leather from soaking up the oil and stops salts and whatnot from the leather attacking the steel. At least one plastic, PVC?, can actually break down and release acid or something so it's not used by coin collectors but I'm reasonably confident that transparencies are not this sort of plastic.
Since only the part of the knife in the sheath is covered in this way I don't know if there needs to be an air gap or if it's better to keep this extra sheath tight. I know that different oxygen concentrations can affect metal corrosion. Oil wearing off one part of the blade might cause it to corrode preferentially (but it's not too much to oil a knife regularly). I've never done this with a tight sheath.
It is quite a bit of effort to go to just to store a knife in its sheath--why not just wrap the knife in an oily rag like the old days?--and it's a really dumb thing to do if you ever plan to sheath the knife without cleaning it. It would pay to clean this internal sheath periodically.
I hear that beeswax used to be used commonly to preserve coins and it won't soak away like oil but it does contain some acids. Nowadays many museums and collectors use an acid free cryptocrystalline wax like Renaissance Wax (do a websearch) to protect coins. Any of these waxes will completely block air from the blade if coated properly. If you don't handle them the wax shouldn't come off.