I get my pots from thrift stores and garage sales. A lot of time it's just pottery that's not meant to be for plants, but it usually works. It's usually pretty easy to drill a hole or two in the bottom to let excess water drip out.

I usually put regular plates under the pots to catch the excess water, also usually collected from thrift stores and garage sales. I try to collect interesting ones when I can, but I also just have some in bulk for when I need a bunch.

For the plants themselves I often grab clippings from people I know if they have something I like. They usually like sharing. If I'm looking for something specific I'll usually buy seeds online.