Cargo or the lack of shouldn’t cost you much as it doesn’t really cause any additional costs other than for the few needed handlers. The “losses” you see under flight info are owed to the way costs are distributed among classes there. Those are no actual losses caused by offering cargo.
On servers with dynamic turnarounds, cargo can cause longer turnaround though. Which in turn results in higher capital costs due to a lower possible utilization.
“Cargo handling” and “Cargo controllers” are the only costs directly attributable to cargo. All the other costs “attributed” to cargo are costs required for the plane to travel at all, regardless of what’s in it.
The game simply displays the total cost divided according to how much space each class takes on the plane, but if you didn’t offer cargo those costs would still be required anyway, and would be distributed by the passenger classes. So no, you’re not taking big revenue losses because of cargo, it’s some other reason.