Oh, you're absolutely right: a strong acid and a strong base will neutralize eachother. But the reaction can be violent and dangerous. You can get a little taste of this by adding lemon juice (a mild acid) to baking soda (a mild base). In this case, the gas evolved is carbon dioxide which is relatively harmless in the quantities we're talking about.
For today's science lesson: this is how baking soda levens food. When it mixes with a mild acid in the food (and there are many mild acids in our food, lemon juice is just one example), it releases carbon dioxide. Those little bubbles of CO2 leven the food. This reaction is accellerated by heat. Baking soda is commonly used in cookie dough. In the heat of the oven, the baking soda reacts with a mild acid inthe dough and evolves CO2 which makes your cookies lighter and softer. This is why, if you watch through the window in the oven, your cookies bubble as they bake. Leave the baking soda out some time and see what you get: hard, flat lumps... with chocolate chips in them. This is also why homemade cookie dough needs to be baked within a few hours of being mixed. The reaction is accellerated in the heat of the oven, but it happens, albeit slowly, at room temperature. Within a few hours sitting at room temperature, the baking soda will all have reacted. Putting the dough in the fridge or freezer can slow it dramatically.
In foods that don't have a mild acid, cooks use baking powder instead. Baking powder is baking soda, which is a mild base, mixed with a mild acid in it in powder form. The two solids don't react very much. Add water and they mix and react. This is why baking powder goes bad over time, though. The two chemicals don't react much as solids... much. High humidity will speed that reaction which is why baking powder used to come in a metal can with a screw-on lid, then a cardboard container with a metalized interior and a tight-fitting lid. Today it comes in a plastic container. The goal is to keep out humidity.