Clean finish while ironing

I've been tuning to print at a 0.1mm layer height, the ironing turned out like this. The left side of the Z is a little textured and more shiny.

Why did this happen and how could I fix it?