You will need to continually pay your taxes to avoid losing the land in a tax sale.
You will lose out on the growth and compounding that the funds could be getting you if invested elsewhere.
You need to maintain the land to avoid liens or other issues.
Depending on location, your land might even lose value over time
If you leave land as a legacy, your family will be responsible for all the expenses and title issues.