Like anything in life, it takes money to ride a motorcycle. It doesn't have to take tens of thousands of dollars, though. Actual costs will vary, depending on where you live, but here's an idea of what it will take to get started, both properly kitted out as well as on the cheap.
Get Your Motor Running:
The absolute cheapest way to start riding is off-road. You don't need a license or insurance, and beater dirt bikes are cheap. For our purposes, however, we're going to assume you want to ride on the street. We'll circle back to dirt later.
License
Although many states don't require a motorcycle endorsement for mopeds or bikes under 50 cc, you're probably going to want to ride bigger bikes, and for that, you'll need a license. While it's possible to learn on your own and just take the test at the DMV, we strongly recommend spending the extra money on the MSF Basic Rider Course, no matter what your budget. This course will actually teach you how to ride, from basic clutch control to emergency maneuvers. These skills will save your hide on the street and are worth many times more the cost of the course.
Some states run it through private organizations, while others run it directly through the state government. In many states, passing the course will automatically earn you your license, avoiding the trip to the DMV altogether. This, alone, is another great incentive to do it this way.
The cost depends on where you are but figure $200-300 or for this. Going straight to the DMV is cheaper, but don't skimp on this part.
Bike
We've covered beginner bikes in great detail. If you have some money to spend, buy someone else's lightly used bike that they started on. People are always looking to upgrade, and with a used bike you won't fret about putting the first scratch in the paint. Go as new and nice as you can afford, but nothing too crazy. While I know people who started with a $20,000 Harley as their first bike, you're better off on an affordable small or mid-size bike. One without a lot of luggage or bodywork is worth considering, too, since these will break if/when you drop the bike. Figure on $2,000 to $4,000 for this.
If you don't have much to spend, and especially if you're handy with a wrench, you can get a bike for really cheap. My first bike, a 1982 Suzuki GS650L, was free. It had sat at my friend's house for three years and had issues, but the price was right. Old UJMs that need work are quite common to find for under $1,000. It's also a great way to learn your way around maintaining your own bike.
There's also the cost of registration, as well as inspection if required where you live, but these are fixed costs that are always going to exist. You can look them up and add them in.
Gear
This can be a contentious subject, particularly in places where helmets aren't required. Of course, we recommend wearing gear, whether the law tells you to or not. You can fudge some things, but you need a good helmet. The saying "If you have a $10 head, buy a $10 helmet" applies, but you don't have to spend more than $200 to protect your noggin. More expensive helmets have more features, better comfort, and can be worth it if you have the money to spend.
The MSF course requires a helmet (loaners are often available), a jacket, long pants, over-the-ankle footwear, and gloves. Basically, all of your skin should be covered. You can fake much of this, but motorcycle-specific clothing is designed to protect the parts of you that can be hurt the most in a spill. It can also have ventilation to keep you cool, or extra insulation to keep you warm on cool days, and sometimes both. Spend as much as you can afford. The good news is that you don't have to splurge for all the best gear all at once, though you'll want to upgrade when you can.
Insurance
Some states don't require insurance, but basic liability insurance is always a good idea. You don't want to be paying for the fence or shrubbery you took out instead of fixing or replacing your bike if you crash. If you spent a lot on your bike, or it has extensive bodywork or luggage, you might consider comprehensive insurance to cover the cost of repair.
The cost of insurance varies widely depending on where you live, the type of bike you get, and its engine displacement. You can often get a payment plan so you don't have to pay the entire policy up front, though you can save some money in the long run by paying it all up front.
Total
We've been pretty vague with exact dollar amounts because there are so many variables involved. If you buy a lightly used beginner bike and good gear, you can get on a good reliable bike for $5,000 to $7,000, all-in. If you cut some corners, get an older project bike, buy some affordable gear and promise to upgrade later, you could get on two wheels for $1,000 to $2,000. This is just a starting cost, though. You'll need to keep putting money into the old project, and you'll want to save and invest in better riding gear for yourself.
What About Dirt?
As we said, you can avoid the license and insurance costs by getting a dedicated off-road motorcycle. Used dirt bikes are often cheaper than road bikes. You'll still want decent off-road riding gear. Especially starting out, you're likely to crash more often than on the street, though at slower speeds with fewer consequences. Even top racers crash a lot, so it's nothing to be ashamed about. You're just going to want protection for when that happens.
The downside is that unlike a dual-sport, you can't ride to the trails or the track, so you'll need a way to get your bike there. If you already have a pickup truck or trailer, you're all set. Otherwise, you may need to invest in one. The cost of that could be more than simply getting licensed for the street so you can have the best of both worlds.