Yes you are on the right track. Chromium, vanadium, niobium, titanium, etc. all form carbides easier than iron, and will bond with available carbon and replace iron carbide with (insert element) carbide. Nickel and manganese generally don't form carbides, nor does silicon. These will mostly be found dissolved in the iron matrix, be it martensite, ferrite, or austenite.
As to where you find the carbides, it depends on what kind. Some form at the grain boundaries, some form at the grain interior. Grain boundaries are very disorderly, and it's easier form carbides to form there in many cases. Particularly the large versions of Chromium Carbide, Cr23C6. Vanadium carbide tends to be a grain boundary carbide, which is nice as it basically acts as a speed bump for grain growth. Chromium carbide does too, but it dissolves at a lower temperature than vanadium carbide, so once it's gone, it doesn't help anymore.
As an example, let's use the 1095 above. Now this particular piece is not a good piece for knives, but it will illustrate the point. The gray/brown parts in the 1095 grains are areas of pearlite. Pearlite is made of alternating layers of ferrite and cementite, like layers in Micarta. So here we have cementite in the grain interior. The white parts around the grains are grain boundary cementite.
When we heat the 1095 to austenizing range, the ferrite will change to austenite and the carbides will dissolve, both at the grain boundaries and in the layers of pearlite. All of it won't dissolve until we get above the normal austenizing range for 1095, so whatever is left is undissolved carbide. For 1095, there will be some in the interior and some in the grain boundaries. For steel in the condition of the picture, we need to make sure the grain boundary carbide is not continuous, for reasons mete pointed out above.