When you ¨Remove 3 Health worth of Invaders”, can you Remove 3 damaged Towns?

When you ¨Remove 3 Health worth of Invaders”, can you Remove 3 damaged Towns?

No, the Health of an Invader doesn't change when it takes Damage. Damaged Towns still count for 2 Health each.
A few effects (e.g., Kingdom of England Level 5, or some Event/Fear cards) modify Health, and this does affect how many Invaders you can Remove. But Damage does not reduce Health. (Health is simply the total amount of Damage it takes to destroy something.)