I'll caveat and say I wish I had known this before I transferred, as I would have put all my money in the C fund. All in all, if you are putting everything in the "C" fund (TSP's S&P 500 fund), it looks like the expense ratio is 0.059%. If you put it in VTI, it's around 0.03%, which is negligible; it's about the cost of a Chipotle burrito once a year. I would think about a couple of things. You can access the TSP at 55 if you retire early, taking advantage of the rule of 55. If you rolled into an IRA a 72(t), from what I understand. If it's Roth, there is more nuance, but the rule of 55 remains.
Another consideration is that Federal Employees also get access to a TSP as a retirement vehicle, so if you get a federal job, it may be worth keeping in the TSP.
Summary: If consolidating your retirements into one brokerage is important for simplicity, I don't see a reason not to. You could potentially save marginally on the expense ratios. If you plan to access this money before 59 1/2, you could have more flexibility keeping in the TSP due to the rule of 55, particularly if you get a federal job.