Grimes, who pretends she's not involved with the far right, was seen with renowned fascist Curtis Yarvin after inauguration