ok, we think we've tracked this down and it's (surprise) related to the search.
When someone makes a new post on the site, the posts index table is locked while the search index updated. Because of the size of the databases, this is taking longer and longer.
So, we've got a couple options here.
Option 1 is that we can add another column to the db and use it to speed up the indexing. This will require a minimum of 48 hours downtime for the forums.
Option 2 is that we can switch over to MYSQL Fulltext searching which is more efficient than the organic vB search, but I'm not sure how much more. This will probably require a week or so of downtime.
Option 3 is installing a third party search app such as Sphinx, which is entirely unsupported by vB at this time.
Option 4 is splitting the db and creating, effectively, another site - archives.bladeforums.com which will have the first 8-9 years of BFC and everything newer will be here on the "live" forums. The only problem with this is that the live forums and the archives won't probably be able to be married back together in the future.
I'm debating on what to do, but I'm open to opinions.