of course the traditional symbol is the six-pointed asterisk $*$ or $\ast$. of course, the problem with that is that all of these programming languages use the asterisk for multiplication.
MattL likes to use the five-pointed $\star$. dunno if i have seen it anywhere else.
http://web.mit.edu/2.14/www/Handouts/Convolution.pdf uses $\otimes$ which i really do not recommend.
my preference is to extend the idea of $\otimes$ but put inside the circle the traditional six-pointed asterisk. LaTeX has a symbol for this $\circledast$. it is sufficiently different from the naked asterisk, to mean something different, yet recognizable.
what do you folks think? we could be establishing a new standard for the lit (hope so).
another symbol i have seen used is $\triangleq$ (an equal sign with a little triangle on top) for "equal by definition to". it's better than that three-line equal sign $\equiv$ which means "logical equivalence". should we encourage usage of that symbol for definitions?