Two countries, two very different cost bases
The mismatch starts with how each country sets your cost. Under US law, when you inherit property your cost, or basis, is stepped up to the fair market value on the date the previous owner died, under Internal Revenue Code Section 1014. So if you sell soon after, the US sees only the gain since the death, which is small or nil.
India works the other way. There is no inheritance tax, so receiving the property is not taxed, but when you sell, your cost is the original owner's cost, carried over under Section 49(1), and your holding period includes theirs under Section 2(42A). If the property was bought before April 2001, you may use its value as on 1 April 2001 instead, under Section 55(2)(b), but that is still far below today's price. So India taxes the gain from decades ago, not just the gain since you inherited.
As an NRI you get no indexation to soften it
For a long-term sale after 23 July 2024, an NRI pays 12.5% on the gain without indexation, plus surcharge, capped at 15% on capital gains, and 4% cess, which comes to about 14.95% of the gain. Section 112 becomes Section 197 from FY 2026-27, but the rate is the same.
The part that stings is that the option to pay 20% with indexation, which would lift your cost for inflation and cut the gain, was kept only for resident individuals and Hindu undivided families. An NRI does not get it. So you are taxed at 12.5% on the full gain from the old cost, with no inflation relief at all.
Why the foreign tax credit gets stranded
You would expect the US foreign tax credit to cancel the Indian tax, but it cannot here. The credit is limited, under Internal Revenue Code Section 904, to the US tax on the same income. Because your US gain is tiny, thanks to the step-up, the US tax on it is tiny, so the credit is capped at that small figure. The much larger Indian tax has almost no US tax to sit against.
The excess credit is not lost forever on paper, it can be carried back a year and forward ten, but it can only be used against US tax on other foreign passive income, which most people do not have, so it usually expires unused. In practice the bulk of the Indian tax is a real cost you bear. On top of that, the 3.8% NIIT applies to your US gain with no credit at all.
A worked example
Sameer, a US citizen, inherits his father's Delhi flat. His father bought it in the 1990s, so its cost, taken as the 1 April 2001 value, is about twenty lakh rupees, and Sameer sells it soon after inheriting for one crore.
India taxes the gain from twenty lakh to one crore, about eighty lakh, at 12.5% plus surcharge and cess, roughly twelve lakh rupees of Indian tax. The US, using the step-up, sees almost no gain because he sold near the inherited value, so his US tax on the sale is small. His foreign tax credit is capped at that small US tax, so most of the twelve lakh has nothing to offset it and is stranded. Planning the sale, and knowing this before he sells, is the only way to manage it.