"Inexpensive" is a relative term. You'll need to be more specific. Compared to a grand piano, all harmonicas are inexpensive. Compared to a McDonald's Happy Meal, not so much.
That said, a good place to start is the Hohner Blues Band. It's about $5 and most music stores (and your local Cracker Barrel restaurant) have them. As the price implies, it's not the greatest harmonica out there but it's better than many no-name brands that cost more and it's functional to start out with and see if you like it.
Once you play that one for a little while then you can decide how much you'd like to invest and in what type of harmonica.
The Hohner Marine Band, Special 20, Blues Harp, and Big River are good choices if you don't mind spending $25-50ish. Lee Oskar is pretty good in that price range too as is the Suzuki Folkmaster (even cheaper but not as easy to find). Just be sure to get your first one in C since most tutorials are written in C.
If you decide you'd like to learn melodic harmonica instead of 'blues harp' then you might prefer the Hohner Golden Melody or one of their Echo Harps. Tremolo harmonicas are ideally suited for playing tunes and melodies but not so hot for bending the blues. The Hohner tremolos have a wetter (sounds like a French cafe accordion) than do the Suzuki tremolos. The Golden Melody is great for tunes and can also do a tolerable job on some blues though it's not tuned as well for use as a blues harp. A lot of players love the Golden Melody for playing gospel music.
Th other option is a chromatic but you said "inexpensive" and the bottom on a decent chromatic is the Hohner Educator 10 at about $50 with some of those chromatics going into the $100's. Chromatics have a lot of advantages over a 10-hole diatonic but aren't as easy to learn on.
add: on wooden vs plastic vs sealed wooden combs. The deal is that many traditionalist players prefer both the feel and the sound of a wooden comb. The problem is that when wood gets wet it tends to swell and then crack. That's a problem for beginner harp players because beginners tend to play wet as well as to own only one or a couple harps which then get more use and become wetter. The solution used to be to lacquer the comb or seal it with wax - that keeps out most of the moisture but adds to the cost. It also, some feel, deadens the tone. Then came plastic - cheaper than sealed wood and results in about the same timbre. It's easier to clean and it lasts A LOT longer than woods. So you have a compromise to look at; a Marine Band with a wooden comb, maybe a little better tone once you learn how to play it but a risk of wood warping and a potentially shorter life. A Special 20 or Blues Harp with a plastic comb, maybe a little loss of tone quality if you were good enough for that to matter, no warping risk, easier to maintain, and a longer life. Both about the same price.
I have some of each. I prefer the wood comb's sound. On the other hand, I also like that my plastic combs get be played on a rainy day with no worries. They each have their advantages and disadvantages depending on how you're going to use it. Anyone that tells you that one is automatically "better" than the other hasn't weighed all of the possible playing styles and circumstances you may find yourself in. That's why we get a choice - there is no single "best" for all circumstances.
Always remember the old saying though "it's a poor musician indeed who blames his instrument for his inabilities." A good player can wow you with a $1 toy from the dollar store. A bad player can't make music with a $1,000 custom made instrument.