Here, the term “best” is actually a noun. It is short for “best of luck.”
“Best” is a confusing term because it can function as a verb (“she bests him at tennis”), adverb (“the best singing”), adjective (“the best singer”), or noun (“he did his best”). However, in “all the best,” it is a noun.
The reason why it is singular is because, like “luck,” it is uncountable.