It might help your investigation to know from my experience that it seems to happen only when you are selling things for a price where the proceeds of the sale if consummated would exceed the available open inventory.

For instance, if I have Inventory (8/12) and one of my items is selling for 6 keys, the bot apparently internally reserves space for the 6 keys I might potentially receive from the sale of that item, so it deems my inventory to be overloaded when it really isn't. All I know for sure is that if I cancel that 6 key item trade, I can then deposit additional items to the shop, so this is what I assume is happening.

This is how I resolved the issue I reported having here: http://dispenser.tf/forum/view/3230/

Hope that helps.