ALDI's special offers for Saturday January 19 include gym balls for $7.99 each. As well as being a bargain purchase for fitness fanatics, that's also a cheap way to test out using a gym ball as your office chair.
Lifehacker's own Melanie Pinola made that switch herself last year, and saw a definite reduction in back pain as a result. If you decide to make the switch permanently, then spending on a more robust model would make sense, but $7.99 is a very cheap entry point. (Note the upper weight limit of 120 kilograms; if you exceed that, shedding some kilos is going to do more for your back pain, I'd suggest.)