[Spock's human side is attempting to intervene and tell him that he's done something wrong... but the Vulcan cannot see it.

He looks down at Coulson...]

I am amenable to that suggestion... I will see you soon Mister Coulson. [He gives the cat one more pat on the head before looking again at the two agents.]

I will hand you over to Miss Romanoff at this time. [Spock, pointedly not answering Clint, hands the kitten over to Natasha.

And as much as Spock would have liked to have stayed... he could also tell that he was not welcome by all...]

I will take my leave...