Some other users mentioned new 8 bit/retro computers, and another one that comes to mind is the Agon Light. I've never used it and I don't know too much about it, but it looks like it costs only about $58 US, which makes it quite a good deal.

Last year I went on an ebay buying spree of vintage 8 and 16 bit computers. I think generally the cheapest I typically found were for TI-99 4/A's.

However, I would agree with the other suggestions to buy a Commodore 64, if you are looking for an actual vintage system, that is. This is still such a popular platform, with good parts availability, lots and lots of documentation, active communities, etc. You mentioned BASIC, and while it might not have the very best BASIC, there are lots of books and online documentation and countless thousands of BASIC programs written for this platform. I think overall it is just the most sensible starting point.

I'd expect the commodore 64 as the most sold computer ever to be the most affordable. It's definitely my favorite machine. Prices have been going up, but I picked up my latest one for $40 at a boot sale around 2016. You can always emulate all of these machines on just about any PC.

If you're okay with some kit building, the RC2014 is relatively inexpensive and can be built with all new chips (except graphics, which I can't find a solution I like for that doesn't use a retro chip).

But it's small, easy to understand, and can store and load data from CF cards. Ships with BASIC in ROM, but there are other environments available, like at least one FORTH interpreter.