Excellent question, in my case I had a 2007 Hyundai Sonata which I got rid of in favor of a 2016 Ninja 300. In the short overlap where I owned both vehicles I found the following: the ninja needed more frequent maintenance trips but the trips where 5* cheaper, $167 in San Jose California for the Ninja compared to ~$800 for the Hyundai, unless you need tires. As a daily commuter and weekend fun toy, I was taking the bike in for servicing about once a quarter, having a decade old car I was getting that serviced about once a year so the ninja saved ~$100. Ninja parts wore out more quickly but they cost about 90% less. In addition to this the Hyundai got ~30mpg, good for a car, the Ninja, which I got as a commuter and 'rode like an old lady' (direct quote from my brother), got ~60mpg. Having said that, I also used the bike as a weekend fun toy and rode much hard, something I never did with the car, we'll call the gasoline a wash. As a new rider insurance was about the same, however, California registration was less than half for the bike and as a green vehicle I gained free access to HOV/toll lanes (easily saved $500 a year). Since California, might be an anomaly, although many other states are following suit, we can ignore that. One thing I won't ignore is the cost of repairs. My girlfriend who is both an inexperienced rider and inexperienced driver managed to drive my car into pole and drop the bike. Luckily the pole only damaged the cars fender, unluckily I can't repair a car's fender. What I can do is replace a bikes fairings, thanks to some good advice from our friend CycleCruzer, I had already swapped out my OEM fairings for a $400 pair of Chinese fairings, cost of repairing the body damage to the car $1700. Cost to repair the bike, a Saturday afternoon buffing out cheap aftermarket fairings. Even if I had to replace the fairings I just checked on bike bandit and an entire set of new fairings is $1000, I don't even want to think about how expensive it would be to redo the entire cars body!
Bottom Line: If you compare a $4000 car to a $4000 bike (sold the car for about $4000 bought the bike for about $4000) and you ride a lot there is a very small chance it might be a wash. However, when something goes wrong on a car it's orders of magnitude more expensive, if you have a $4000 car something IS going to go wrong.
Comparing new to new: The best deal I have seen anyone get on a brand new car is $15,000+taxes and fees, and that was NOT this decade. Even if you buy a ninja300 at MSRP, after taxes and fees you're still looking at less than $6000. That's a minimum savings of $9000. If you have to pay $200 to service your bike AND you service your bike 4 times a year, it's going to take you 11.25 years to spend $9000 servicing your bike.