Few things irritate office workers more than booking a meeting right over lunch. Sure, it may be the only time most people's calendars are open, but that's probably because everyone's trying to get out of the office and have a meal. The solution is simple: don't schedule lunch meetings unless you're buying lunch.
At my old job, I had to organise more meetings than I prefer to remember, but one thing was certain — if you booked a meeting in that time block between 11am and 1pm, you should expect someone to walk in and tease you with "so, when's lunch getting here?"
Obviously if it's important and everyone agrees that it's the best and only time for the meeting to happen, then it needs to happen, but considering that lunch is the only time that most people working in offices actually get to get up, walk around, leave the office for a little bit, or have a little time to themselves, it's best avoided if at all possible — and if you do need to intrude on that time, the least you can do if you're the meeting organiser is to provide or organise a lunch delivery, even if you ask everyone to chip in for it.
For the rest of us though, if you have that particular manager or team member who just can't get away from booking people over lunch, may we suggest the old "block off time for everything on your calendar" technique to keep them from booking you? Your calendar will look busy (especially if you make it a private appointment) and they will have to find some other time you're available.