TLR - wet sanded on mineral oil and now it has a weird velvet like finish. Any ideas how to fix?

I made some mistakes, and now I am having to figure out how to live with them. I didn't do enough research prior to finishing my walnut island top.

1st mistake was not wet sanding at all, I never raised the grain. this is odd, because I do this on every bowl i make while it is still on the lathe, but for some reason i completely neglected to do it on the island top.
2nd mistake was not realizing that mineral oil is great for cutting boards (this is where my research stopped, it was like, "oh, people finish cutting boards with mineral oil...that will be perfect for an island countertop, looks great and easy to maintain!"
3rd mistake, i am not sure where it really went wrong. but here's what happened:
I installed the countertop and it looked great.
then my kid spilled some water on it which I thought would be totally fine. i didn't care. until it went from being 220 smooth to raised grain rough.
I wet sanded with water and 220, it got smooth again. Yeah!
I cleaned it as thoroughly as i could (there was a lot of sanding sludge).
I dried the surface by hand with paper towels.
I let it completely dry.
I used a tack cloth to remove any left over residue.

now the issue...It is like it has a velvet finish. depending on which way you rub it, it looks clean or it looks dusty.
to fix it, I then:
washed with mild soap and water, dried, tack cloth --> no change, still the dusty velvet look.
wiped with micro fiber cloth and mineral spirts --> no change, still the dusty velvet look.

pics to show what it looks like, and a quick video clip showing it
https://youtu.be/_0-nmn2oe5U

first pic is what it looked like before i tried to "fix" it! ahhhhhh
PXL_20210123_025557278.jpgPXL_20210213_160107872.jpgPXL_20210213_160113740.jpgPXL_20210213_160122295.MP.jpgPXL_20210213_160132867.MP.jpg

Any ideas what to try next?